Enhanced Single Image Dehazing Technique based on HSV Color Space

1. INTRODUCTION

Digital image processing (DIP) is important in many fields including medical image processing, image in-painting, pattern recognition, biometrics, content-based image retrieval, image dehazing, and multimedia security [1], [2]. In DIP, visibility is a major problem that image vision-based systems must deal with. Weather condition makes a scene less visible, which has an impact on how well outdoor image processing-based systems work such as detection and recognition of objects, visual surveillance, traffic monitoring, intelligent transportation, and etc. [3]. The camera captures a small portion of the light reflected directly from the surface of an object as well as a significant portion of the light reflected by the atmosphere. A light that is reflected from the surface of an object is scattered and absorbed by atmospheric particles [4]. In bad weather, these scattering and absorption increase causing the irradiance to be measured incorrectly [5]. In addition, inclement weather makes the images and videos degrade and this leads objects lose their contrast and visibility [6]–[9]. Applications based on computer vision operate perfectly when the input is noiseless. Many applications perform unsuccessfully in bad weather due to the fading images and videos. Therefore, image dehazing is required for computer vision-based applications. Dehazing techniques turn a hazy image into a haze-free image [6], [9]. Typically, an atmospheric scattering model (ATSM) is used to do dehazing [6]. The creation of images in inclement weather is described by ATSM. The depth of the scene point affects the haze concentration according to ATSM [6]. Image dehazing can be broadly divided into three categories: 1 – Based on additional information, 2 – based on numerous images, and 3 – based on a single image. Early dehazing techniques are based on additional information [10], [11]. These techniques call for further details such as depth cues and degree of polarization. User engagement or other camera positioning procedures can supply this additional information. Therefore, real-time vision applications are not an appropriate for these approaches. For approaches based on multiple images, numerous images of a scene under various weather conditions obtained from differing degrees of polarization are required [12]–[16]. These techniques require additional hardware or resources; therefore, they are more expensive than single image dehazing, which has caught researchers’ attention [6]. Strong priors and assumptions are the foundation of the majority of single image dehazing approaches. Only if the presumptions are accurate will these strategies work. Because these priors or assumptions were incorrect, the single image dehazing approaches were inaccurate. Typically, smoothing increases the transmission accuracy, which slows computing of the dehazing process. On the other hand, certain algorithms take atmospheric light into account [17], [18].

This paper concentrates on single-image dehazing which additional information is not required and also numerous images of different scene under various weather conditions are not required. It presents a new approach for selecting the haziest opaque regions of an image and using them to estimate the airlight. The rest of the paper contains the following sections: Section 2 presents the literature review. Section 3 explains the background. Section 4 describes the proposed approach. Section 5 shows the experimental results. Finally, the paper concludes in Section 6.

2. LITERATURE REVIEW

Usually, the unclarity of the images is produced due to the impact of weather conditions such as haze and fog. Haze/fog removal is considered a significant issue and hot topic by the researcher since the clarity of the degraded images is required for a variety of computer vision-based applications. Many techniques are introduced and existed for the haze/fog removal purposes in the area of DIP. The main competition in this research area is increasing the quality of the dehazed image; mainly, peak signal-to-noise-ratio (PSNR) and structural similarity index (SSIM) image quality measurements are used. This section reviews the most important and related existing works on image dehazing using DIP. In general, all the existing techniques either contributed to improve dark channel prior (DCP) and/or worked on proposing another technique to estimate the airlight and the transmission map. Furthermore, they used either PSNR, SSIM, or both to evaluate their performance.

This area of research was first explored by Narasimhan and Nayar in 2003, who offered a technique for reducing haze that made use of multiple images of the same location taken in various weather conditions [6]. They dealt with the issue of restoring contrast in images and videos that had been negatively affected by the atmosphere. This work discussed ways to identify depth discontinuities and determine a scene’s structure using data from two images taken in various weather conditions. It demonstrated how to recover contrast from any image of the scene captured in inclement weather using either depth segmentation or the scene structure. This work did not use a specific dataset for the experimental result and it selected some images from the internet. The drawback of this approach is that it is dependent on weather variations to provide a number of images. In 2008, Raanan Fattal, suggested an image dehazing technique that requires only one input image [19]. Object surface shading and transmission signals are thought to be unrelated. The transmission map was calculated using independent component analysis. Later, a Markov random field was used to infer the color. For the experimental results, this work did not use a specific dataset, and the evaluation was performed on randomly selected images from the internet. Tarel and Hautiere in 2009, introduced a new image dehazing technique [20]. As reported in this work, since the ambiguity between the presence of fog and the items with low color saturation is resolved by assuming only small objects can have low color saturation, the ability to handle both color images and gray-level images is the achievement of this work. This work did not use a specific dataset for the experimental results and images are selected from different image datasets. Lu et al., in 2015, proposed a powerful single-image dehazing technique [21]. Based on the color lines, airlight is calculated by applying a compensated filter to the white-balanced image, the highlight regions were eliminated as a pre-process for the airlight estimation. White-balanced image refers to the procedure of correcting colors to get objects that are white in reality to appear correctly white in your desired image by removing unnatural color casts. The airlight is estimated after the highlight regions have been subtracted from the image. The transmission map is then estimated using DCP. This work presented a semi-globally adaptive filter (SAF) to reduce the formation of gradient reversal artifacts on a rough transmission map. White-balanced image serves as the starting point for SAF’s filtering procedure. To evaluate the performance of this work, the AMOS-outdoor dataset was used. As authors reported, this technique achieved 15.2440 for PSNR and 0.7565 for SSIM. Salazar-Colores et al., in 2019, introduced an image dehazing technique that significantly reduced the recurrent artifacts that are produced due to using the traditional DCP [22]. Both airlight and transmission map were estimated using DCP. This work provided a quick and an efficient way of altering the DCP computation, which greatly reduces the artifacts produced in the restored images when utilizing the standard DCP. It used the pixel-wise maximum operation to reduce the underestimated values in heterogeneous regions near from edges. However, the effect of the pixel-wise maximum operation values is essentially unaltered in homogenous regions, far from the edges, where the dark image neighbors are very comparable. For the experimental results, a dataset of 100 images has been created using Middlebury Stereo datasets. This technique obtained 18.50 of PSNR and 0.810 of SSIM. Dai et al., in 2019, suggested a robust ATSM [23]. The actual image was breaking down into incident light as well as reflectance components and adding a noise term to the conventional model. For the airlight estimation, this work uses the same way as DCP which is picking 0.1% of brightest pixel in the dark channel of an image. Furthermore, it chooses eligible input image pixels as the candidate pixels because they have the same coordinates as the top 0.1% of the brightest dark channel pixels. This work also takes into account that the region with the greatest hazy opacity has a marginal three-channel difference. As a result, among the candidate pixels, the pixels with the smallest absolute change across the three channels are chosen to represent atmosphere light. To reduce over-enhancement in locations with thick haze, a compensation term with regard to transmission map is implemented after they estimate the transmission map using the DCP basis. For the performance evaluation, RESIDE SOTS (outdoor and indoor) and O-Haze datasets were used. Regarding to the RESIDE SOTS-outdoor dataset, 18.264 for PSNR and 0.855 for SSIM were obtained and for the RESIDE SOTS-indoor the obtained PSNR is 18.860 and SSIM is 0.831. Moreover, for the O-Haze dataset, 16.4 for PSNR and 0.75 for SSIM were gained. Gao et al., in 2020, developed a non-local consistency assumption to eliminate the “halo” effects caused by standard image dehazing techniques and produce a haze-free image from a single hazy image [24]. When an image has been heavily edited, especially through the use of high dynamic range editing, a bright line known as a halo may form in places of high contrast on the image. This work dealt with each pixel separately instead of a block of pixels. It began by enhancing the technique for obtaining atmospheric light value and adapting their algorithm to a range of unique situations. Consequently, the brightness and saturation data were used for a single pixel to define a special energy function. The revised transmission map was then created by introducing propagation and random search into the image dehazing field. For the experimental results, 11 hazy images were selected from the internet and the obtained SSIM was reached 0.69. The limitation of this work is not perfect in some synthetic hazy image scenes which are caused by unaccurate airlight estimation. Zhang et al., in 2020, developed a unique saliency-based and bright channel prior (BCP)-based single image dehazing technique [25]. A supper pixel-based atmospheric light estimation method was suggested to increase atmospheric light estimation accuracy in the stage of estimating atmospheric light. Furthermore, initially, the BCP model was suggested based on their observation to manage bright spots in the hazy images at the transmission map estimation stage. The automatic fusing of the DCP and BCP models is accomplished through a fusion-based transmission map estimation technique that is subsequently provided. In the refinement process, saliency analysis was applied to improve the rough transmission map. Middlebury Stereo dataset was used for testing this algorithm and the obtained PSNR and SSIM was 15.96 and 0.8287, respectively. In 2020, Yang and Wang. designed a new strategy to address DCP flaws [26]. This technique consists of two modules: Transmission map estimation and piece-wise function. In addition to acquiring a new dark channel map, a delicate function was used to replace the minimal filter operation. A nonlinear compression was then applied afterward to enhance accuracy and optimize the transmission map. When the ATSM used in conjunction, this technique can restore a clear image. For the experiment results, the RESIDE SOTS dataset was employed. Consequently, this technique achieved the PSNR of 17.021 and SSIM of 0.778.

Recently, Sun et al., in 2021, suggested the new image dehazing technique [27]. The atmospheric light value is first calculated using the K-means clustering technique, which may successfully reduce the atmospheric light estimation inaccuracy brought on by the appearance of white objects in the image. Second, the transmission map is improved using the quick weighted guided filtering technique to eliminate of discontinuity and halo artifacts. The dehazing image’s contrast and brightness are then adjusted using gamma correction and automatic contrast-enhancement methods. Middlebury dataset was utilized for the experimental outcome. Furthermore, for evaluating the proposed work, certain quality metrics were used such as information entropy, the rate of new visible edges, the mean of normalized gradients of visible edges, and average gradient. This technique effectively removes halo artifacts while restoring clear images. In 2021, Raikwar and Tapaswi developed a method that utilized a difference channel (DCH) for estimate of the initial transmission map to nonlinearly translate the minimum channel of a hazy image into a minimum channel of a haze-free image [28]. This method employs a quad-tree subdivision-based method for the airlight estimation. It repeatedly divides an image into four rectangular parts. Based on the threshold, the brightest zone is selected as a region of atmospheric light. Contextual regularization enhances the smoothness of the initial transmission map. It has been demonstrated that estimation of the initial transmission map using DCH is more precise and reliable in variable haze concentration than existing methods. The suggested method can recover information from a distance while producing differing visual outcomes. However, regularization, a computationally slow approach is used to further smooth the initial transmission map obtained by the proposed method. The RESIDE SOTS and Dense-Haze datasets were used for the performance evaluation purposes. As reported in this work, for the RESIDE SOTS dataset, the obtained PSNR was 17.74 and the obtained SSIM was 0.83. Moreover, for the Dense-Haze dataset, the obtained PSNR was 12.26 and the obtained SSIM was 0.20.

Riaz et al., in 2022, introduced a straightforward but efficient method of image restoration using multiple patches [29]. It fixed DCP’s flaws and increased its computation speed for high resolution images. For the airlight, this work uses the same techniques as DCP. The smallest number of patches of various sizes is used to estimate a coarse transmission map. The transmission map is then improved using a cascaded rapid guided filter. This work provides the advantage of very little performance reduction for a high resolution image by introducing an effective scaling technique for the transmission map estimation. The standard Middlebury stereo vision dataset was utilized for the performance evaluation. Furthermore, this work reached a SSIM value of 0.9689.

The proposed approach presented in this paper focuses on single-image dehazing and it improves the DCP by enhancing the airlight estimation, details are discussed in the next sections.

3. BACKGROUND

This section concerns with the explanation of the atmosphere scattering model (ATSM) to understand how the hazy image is formed. In addition, the DCP is also discussed as one of the most mechanism used by the majority of researchers as a base for improving it or develop their technique for image dehazing.

3.1. ATSM

The ATSM was introduced by McCartney in 1976, aims to explain how hazy images are formed [30]. Later, the ATSM was significantly improved by Narasimhan and Nayar [6]. An imaging model of a hazy scene, as illustrated in Fig. 1, essentially consists of two factors under the principle of atmospheric scattering: (1) The technique of attenuating the light that is reflected from an object’s surface onto a camera is the first factor, (2) the second factor is how the airlight is dispersed as it approaches the camera. Theoretically, hazy and blurry images are based on both components [31].

Fig. 1. Imaging model of hazy scene [31].

Consequently, the scattering model to represent hazy images in the field of computer vision can be expressed as:

Where x is the distance coordinate, I(x) denotes an image with the haze, J(x) denotes an image without haze, A denotes an atmospheric light, and t(x) is the medium’s transmission rate which is also known as transmission map. To recover J(x) from I(x), image dehazing is used. The deterioration model has several unidentified parameters, which creates an imprecise problem. J(x) can only be reconstituted from I(x) after estimating the parameters A and t(x).

3.2. DCP

The DCP is based on the observation that most non-sky areas have at least one color channel with very low intensity at certain pixels in haze-free outdoor images [12]. In other words, the minimum intensity of the patch/block should be quite low. Formally, for an image J, the DCP can expressed as:

Where _Ω(x) is a local patch centered at x, y is one pixel in the local patch _Ω(x), J^c is a color channel of J. According to the observation, if J is a clear outdoor image, the intensity of J_dark is low and typically zero, with the exception of the sky region. The above statistical finding or information is referred to as the DCP, and J_dark referred to as the dark channel of J. According to [12], the following three reasons contribute to low intensities in the dark channel:

3.2.1. Shadows

For instance, the shadows cast by vehicles, buildings, and the interiors of windows in cityscape images, or by leaves, rocks, and trees in landscape images.

3.2.2. Bright surfaces or items

For instance, any object such as green grass, trees, plants, red or yellow flowers, or blue water surfaces lacking color in any color channel will produce low values in the dark channel.

3.2.3. Dark items or regions

The black channels of some images are incredibly dark because the natural outside images are typically colorful and full of shadows. For instance, stone or a black tree trunk.

In general, the DCP consists of two stages [12]. First stage is the airlight estimation which is selecting the top 0.1% of pixels in J_dark with the highest brightness found, and then the maximum value of the pixels that match the pixels in the original image is chosen as the atmospheric light. The second stage is the transmission map estimation, which is according to [12] can be expressed as:

I_c and A_c represent input hazy image and airlight in color channel c, respectively. In practice, to preserve the sense of depth in the image, a correction factor ω (0<ω≤1) is added to keep the partial haze. Then, Equation (3) can be rewritten as follows:

Since the regional transmission map is assumed to be constant, the block effect often exists in the transmission map. To further improve t(x), soft-matting [12] or guided filtering [16] are applied. The final scene radiance can be recovered using Equation (5) under the ATSM once the transmission map t(x) and the atmospheric light A have been acquired:

4. PROPOSED APPROACH

Based on DCP, two essential components of single image dehazing are airlight estimation and transmission map estimation [12]. In DCP, before transmission map estimation, the input hazy image must be first normalized by the estimated airlight. Airlight estimation is crucial for recovering haze-free scene radiance. Consequently, estimating the airlight improperly leads to an erroneous transmission map estimation and incorrect scene radiance recovery. Based on the DCP strategy, the dark channel of the image can be calculated using Equation (2). Then, 1% of the brightest pixel in the dark channel is selected and then calculating the average of the matching pixel in the input hazy image is used as an airlight. This estimation fails when there are white items in the image and when these white things are chosen as the scene’s most opaque haze region. In this study, the proposed approach improves the airlight estimation that uses the HSV color space. The brief detailed of the proposed approach is presented in the following subsection.

4.1. Airlight Estimation

Due to the impact of haze, certain regions of the hazy image are caused to have high brightness and low saturation. For a haze-free region or light haze in a hazy image, the scene’s saturation is rather high, its brightness is moderate, and the difference between brightness and saturation is almost near to zero [32]. However, in [32], the authors discovered that for the places with moderate haze, the saturation of the region drastically falls while the color of the scene fades due to the haze, and the brightness increases at the same time causing the large value of the difference. It is more challenging for human eyes to distinguish the scene’s natural color in areas with strong haze, and the difference is even greater. According to [32], it appears that the three characteristics (brightness, saturation, and difference) are likely to change frequently in a single hazy image. Consequently, in this study, the HSV color spaces is utilize to pick the region of the hazy image that contains the haziest and then use that region to estimate the airlight. The steps of the proposed approach for the airlight estimation are as follows:

1. The input hazy image I is converted from RGB to HSV color space producing I’

2. The image I’is divided into blocks of size (32 × 32) pixels

3. For each block, the difference between brightness V and saturation S is calculated, D = | V – S |

4. The block that has a maximum difference value, i.e. maximum D, is selected as the haziest opaque region

5. The dark channel for the selected block needs to be produced using Equation (2)

6. From the produced dark channel, 1% of the brightest pixels in the block (i.e., 0.01 × 32 × 32 = 10 pixels) are selected

7. Based on the location of the selected 10 brightest pixels, select 10 pixels of the same location in the original block of the input hazy image, and then calculate the average A of them for each channel separately producing A_R, A_G, A_B

8. Finally, A_R, A_G, and A_B are considered the values of airlight.

Fig. 2 illustrates the block diagram of proposed airlight estimation.

Fig. 2. Block diagram of the proposed airlight estimation.

The advantage of the proposed airlight estimation over the airlight estimation in DCP is that DCP can fail to select the haziest opaque region by the influence of white object. While in the proposed airlight estimation, the region with haziest can be selected, and hence, the airlight can be estimated from that region properly and more accurately. Furthermore, instead of calculating the dark channel for the entire image, the proposed approach calculates the dark channel of only one block of the input image. This leads to reduce the time consumption, since calculating dark channel for the entire image is time consuming.

4.3. Scene Radiance Recovery

On the bases of Equation (5), the haze-free scene can be reconstructed once the airlight and transmission map have been estimated. Fig. 3 illustrates the block diagram of the scene radiance recovery, that is, dehazing technique.

Fig. 3. Block diagram of the Dehazing technique.

5. EXPERIMENTAL RESULTS

This section concerns with the performance evaluation of the proposed image dehazing approach. First, it contains details about the dataset that was used in the experimental results. Second, to evaluate the influence of the proposed approach in terms of image quality, objectively as well as subjectively, extensive experiments are carried out. Finally, the results of the proposed approach are compared to the results of the most recent relevant approaches.

6. EXPERIMENTAL ENVIRONMENT

To evaluate the effectiveness of the suggested strategy, experiments are conducted under the Intel i7-6600U 2.8 GHz CPU and 8 GB RAM, using Matlab 2018. The proposed approach uses RESIDE SOTS dataset which contains 500 images for each outdoor and indoor scene [33]. Figs. 4 and 5 show some image examples for each indoor and outdoor dataset, respectively.

Fig. 4. Example of outdoor images.

6.1. Objectively Assessment

Image quality evaluation is crucial in image analysis systems to analyze techniques and assess their performance. Consequently, it is essential to analyze the experimental findings objectively. Furthermore, two common objective evaluation methods are used, which are:

6.1.1. SSIM

SSIM is one of the most common image quality measurement. For employing this measurement, two images from the same acquired image must be considered, the original image and the processed image. SSIM can be calculated using the following Equation [34].

l(x,y) is a luminance comparison function, c(x,y) refers to contrast comparison function, and s(x,y) is structure comparison function. Moreover, x and y are two images to be compared. In addition, α > 0, β > 0, ϑ > 0 denote the relative importance of each of the metrics.

6.1.2. PSNR

The PSNR ratio measures how much noise can degrade an image’s representational quality in comparison to its highest achievable power. Calculating the PSNR, it is necessary to compare that image to an ideal clean image with the maximum possible power. The Equation 7 can be used to compute PSNR [35].

Where, L is the value of maximum possible intensity levels. MSE is the mean squared error and it is defined as:

Where:

I: represents the matrix data of the original image

R: represents the matrix data of the reconstructed/degraded image

m: represents the number of rows of pixels

i: represents the index of that row of the image

n: represents the number of columns of pixels

j: represents the index of that column of the image.

The obtained results are compared to the results of four other existing techniques such as [12], [23], [26], and [28], Table 1.

TABLE 1: Objectively performance evaluation of the proposed approach and the existing approaches

Table 1 presents the average value of SSIM and PSNR for the all images in the tested dataset. Table 1 makes it abundantly clear that the proposed approach outperformed the existing approaches. Moreover, the PSNR and SSIM of the proposed approach are significantly increased in comparison to other existing techniques.

The reason behind selecting reference [12] is, this work is considered as one of the most successful and earliest work in the area of single image dehazing. Moreover, the majority of the existing works are based on [12] that known as DCP in the literature. They either present their contribution by making improvements to DCP or by defining new techniques for making DCP work better. In addition, the reasons behind selecting the references [23], [28], and [26] are because these works are recently published, their contribution is significant, and they were published in the well-known and high-quality journals.

6.2. Subjectively Assessment

Figs. 6 and 7 illustrate the reconstructed images using the proposed image dehazing approach and other existing approaches for both SOTS-outdoor and SOTS-indoor, respectively.

Fig. 5. Example of indoor images.

Fig. 6. Reconstructed outdoor images: (a) Hazy images, (b) Reconstructed by [12], (c) Reconstructed by [23], (d) Reconstructed by [26], (e) Reconstructed by [28], (f) Reconstructed by proposed approach.

Fig. 7. Reconstructed Indoor Images: (a) Hazy images, (b) Reconstructed by [12], (c) Reconstructed by [23], (d) Reconstructed by [26], (e) Reconstructed by [28], (f) Reconstructed by proposed approach.

From Fig. 6, one can notice that the resulted images of the proposed approach clearer and the haze removed properly compared to other tested techniques. In addition, it can be noticed that results of [12] and [28] are still hazy and the haze not removed completely, while results of [23] and [26] are over dehazed.

From Fig. 7, it is quite obvious that in the resulted images of the proposed approach, the haze is completely removed and the colors of the scenes are rendered naturally. In contrast, the resulted images of [23] are mostly over dehazed and for the techniques [26], [28], and [12] the haze is not removed completely.

7. CONCLUSION

Images acquired under hazy environment require processing for improving their contrast and color fidelity. Haze removal or dehazing is a significant pre-processing stage in the area of computer vision and video applications. Many techniques have been proposed in the literature for dehazing outdoor/indoor images. This study is presented an approach to enhance atmospheric airlight by exploiting HSV color space. The proposed approach can discover the most haze opaque region in the input hazy image by finding the difference between brightness and saturation of each region of the input hazy image. Consequently, from the selected region, the airlight is estimated. Regarding to the transmission map, the proposed approach uses the traditional DCP technique. In other words, the proposed approach focused on improving the airlight. The proposed approach is implemented on RESIDE SOTS dataset for both outdoor and indoor images. The performance of the proposed approach is assessed objectively and subjectively. Regarding to the objectively evaluation, the proposed approach achieved the PSNR and SSIM of 18.37 and 0.90 for the outdoor images, respectively. For the indoor images, it achieved the PSNR and SSIM of 20.59 and 0.89, respectively. The obtained results are compared to the results of other existing image dehazing techniques in terms of PSNR and SSIM and it outperformed existing techniques. In terms of subjectively evaluation, the proposed approach again outperformed the existing techniques. The future direction of this research will concentrate with strengthening and improving transmission map estimation to obtain/reconstruct the better dehazed image quality.