# Adaptive coherence-enhancing diffusion flow for color images.

Color image restoration is one of the fundamental problems in image processing pipelines. Variational regularization and diffusion partial differential equations (PDEs) are widely used in solving these low-level image smoothing and noise removal problems. In this paper, we consider a new adaptive coherence enhancing diffusion (CED) filter which combines anisotropic diffusion and structure tensor derived diffusion functions. By exploiting isotropic smoothing in homogeneous regions and anisotropic diffusion tensor filtering in edges and corners we obtain a PDE flow which can removing noise while preserving important image details. Compared to the original CED approach our proposed adaptive CED (ACED) obtains stable smoothing results. Experimental results on synthetic and real color images show that the proposed filter has good noise removal properties and quantitative measurements indicate it obtains better structure preservation as well.Keywords: coherence, structure tensor, anisotropic diffusion, image restoration

Povzetek: Predlagan je nov algoritem za obnavljanje barv slik.

1 Introduction

Image restoration is an important low-level image processing which is still an active area of research in computer science. Among a wide variety of image noise removal methods two important classes of techniques are variational regularization and partial differential equation (PDE) based filters [1]. Perona and Malik [2] proposed an anisotropic diffusion filtering based on a nonlinear PDE for image denoising and edge detection. Though the Perona-Malik (PM) PDE obtained edge preserving restorations under noise, it is known to create blocky artifacts in homogenous (flat regions where the pixel values do not vary much) regions in the resultant images.

Various modifications and adaptations of PM PDE in particular and other PDEs in general have been proposed in the last two decades. One of the important class of improved PDE based filter is due to Weickert who provided a unified theory of anisotropic diffusion [4], The structure tensor provided better orientation estimation and edge discrimination for steering the diffusion process away from image discontinuities and to make smoothing strong in homogenous regions. Weickert [5] proposed an elegant formulation which handles coherence enhancement for color images with tuning the eigenvalues of the structure tensor in a controlled manner. Structure tensors provide a geometric analysis of digital images via eigenvalues and vectors and there have been applications in edge detection and image denoising literature.

In this work, we base our new PDE based filtering approach on Weickert's coherence-enhancing diffusion [5] with an adaptive choice of diffusion functions for better edge and corner preservation while smoothing out random noise. For this, we utilize structure tensor eigenvalues for controlling anisotropic smoothing according to geometric content of the images. This adaptive choice facilitates isotropic diffusion in homogenous regions and anisotropic diffusion near sharp edges, corners. Our proposed filter is robust to noise and we conduct detailed experimental results on noisy synthetic and real images to prove the effectiveness. Comparison results with related filters show that we obtain better restoration results visually as well as based on peak signal to noise ratio and structure similarity.

We conduct experimental results on synthetic and various corrupted real color test images and test our method against some related filters from the literature. Our experiments show that both visually and quantitatively our proposed adaptive approach obtained better restoration results.

2 Adaptive coherence enhancing diffusion

2.1 Preliminaries

We start with the basic assumptions and notations of coherence-enhancing diffusion filtering framework. Let [u.sub.0] : [OMEGA] [subset] [R.sup.2] --> R be the input (possibly noisy) grayscale image. Weickert [4] provided a unified tensor diffusion formulation which is given by the following parabolic nonlinear PDE,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. (1)

The resultant sequence of images [{u(* ,t)}.sup.T.sub.t=0], for a finite time T represents a nonlinear scale space. Here the diffusion tensor V is dependent on the image information via the structure tensor [J.sub.[rho]]([nabla][u.sub.[sigma]]). Structure tensors encode local image information with first order directional derivatives and is given by,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (2)

where [[([u.sub.[sigma]]).sub.x], [([u.sub.[sigma]]).sub.y].sup.T] ([X.sup.T] denotes the transpose of vector/matrix X) is the gradient of [u.sub.[sigma]] (pre-smoothed image [u.sub.[sigma]] = [G.sub.[sigma]] * u), [G.sub.p] = [(2 [pi][[rho].sup.2]).sup.- 1] exp (- [absolute value of x].sup.2] /2 [[rho].sup.2]) is the Gaussian kernel and * denotes the convolution operation. Let the eigenvalues and eigenvectors of the structure tensor be ([[lambda].sub.+], [[lambda].sub.-]), and ([[upsilon].sub.+]+, [[upsilon].sub.-]) respectively. Weickert's unified tensor diffusion formulation is given by,

D = [f.sub.+] ([[lambda].sub.+], [[lambda].sub.-]) [[upsilon].sub.+] [[upsilon].sup.T.sub.+] + [f.sub.-] ([[lambda].sub.+], [[lambda].sub.-]) [[upsilon].sub.-] [[upsilon].sup.T.sub.-], (3)

where where [f.sub.+], [f.sub.-] are the diffusivities perpendicular and parallel to structure orientations. The eigenvectors of the structure tensor [J.sub.[rho]] matrix can be calculated as,

[[lambda].sub.[+ or -]] = 1/2 (trace([J.sub.[rho]]) [+ or -] [square root of [trace.sup.2] ([J.sub.[rho]] + 4def([J.sub.[rho]])). (4)

For vector valued (multichannel) images u : [OMEGA] [subset] [R.sup.2] [right arrow] [R.sup.N] with u = ([u.sup.1], [u.sup.2], ..., [u.sup.N]) channels the PDE (1) can be written using a common diffusion tensor,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. (5)

For vectorial images the common structure tensor is given by,

[J.sub.[rho]] ([nabla] [u.sub.[sigma]]) = [n.summation over (i=1)] [w.sup.i] [J.sub.[rho]] ([nabla] [u.sup.i.sub.[sigma]]), (6)

with [[summation].sup.N.sub.1=1] [w.sub.i] = 1, and [w.sup.i] > 0 are the averaging factors. Interpretation of this tensor for vectorial images in terms of eigenvalues and eigenvectors carries over from grayscale case, see [5] for more details. A simple choice is to chose [w.sup.i] = 1/N for all i = 1, ... ,N representing all channels have similar meaning, range and reliability. We restrict ourselves here to color images (RGB, N=3), and the formulation holds true for multispectral imagery as well.

Weickert [5] proposed the following particular choices for steering smoothing for coherence enhancement diffusion (CED),

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. (7)

with [alpha] > 0 is known as the coherence factor (if the coherence is inferior to [alpha] the flux is increasing with the coherence while if the coherence is larger then a the flux decreases as the coherence grows), [gamma] > 0 a small parameter added to keep the tensor diffusion matrix D in Eqn. (3) positive definite. Note that [([[lambda].sub.+] - [[lambda].sub.-]).sup.2] measures the coherence within a window of scale [rho]. This particular choice obtained good diffusion results when the structures are oriented in one particular direction, however can smooth out corners and other singularities as multiple directional information is lost, see Figure 1.

2.2 Adaptive coherence-enhancing diffusion

In this work, to control the filtering better and to preserve image singularities better we chose the following adaptive diffusivities,

[f.sub.+] = exp (- [[[lambda].sup.2.sub.+]/[[beta].sub.1]])

[f.sub.-] = (1 - exp (- [[[lambda].sup.2.sub.+]/[[beta].sub.1]])) exp (- [[[lambda].sup.2.sub.-]/[[beta].sub.2]]) (8)

With this choice of diffusivities we observe the following salient points:

[check] If either of the eigenvalues [[lambda].sub.+] or [[lambda].sub.-] is high the diffusion is now in the direction of [[upsilon].sub.+] or [[upsilon].sub.-], which in turn means at corners (where [[lambda].sub.[+ or -]] is high) anisotropic diffusion is applied.

[check] Original CED formulation's diffusion oriented in the direction of [[upsilon].sub.+] is kept intact and the diffusivity [f.sub.-] is now incorporates orientation direction [[upsilon].sub.-] (coherence orientation).

[check] The parameters [[beta].sub.1], [[beta].sub.2] control the diffusivities along [[upsilon].sub.+], [[upsilon].sub.-].

[check] In homogeneous (flat regions where the pixel values do not vary) areas the diffusion is still isotropic.

The diffusion PDE in Eqn. (5) where the diffusion matrix in Eqn. (3) given with this diffusivities (8) obtains adaptive coherence enhancing diffusion (ACED) for smoothing color images with salient edges, comers better preserved as we will see in the experimental results next.

3 Experimental results

3.1 Setup and parameters

The PDE based filters (CED) are implemented using the implicit finite differences method with coherence parameter [alpha] = 5 x [10.sup.-4], [gamma] = 0.01 (to keep D positive definite), pre-smoothing Gaussian standard deviations [sigma] = 4, [rho] = 1, and step size [DELTA]t = 0.24. The new parameters in our ACED [[beta].sub.1] = 20, and [beta] = 20 are set in all the experiments reported here.

3.2 Comparison results

Figure 1 shows a comparison of Weickert's original CED [5] (Eqn. (1) with (7)), and our proposed adaptive improvement ACED (Eqn. (1) with (8)) at the same terminal time T = 25. We show the residue [absolute value of u(x, T) - [u.sub.0]] to highlight the amount of noise removed in both these methods. As can be seen, our proposed ACED preserves the central square's edges through the diffusion flow and smaller texture regions are grouped better than the original CED formulation.

To show visually the qualitative differences of the flows we utilize the Starry Night, a painting by van Gogh (Saint-Remy, 1889, Courtesy of The Museum of Modern Art, NY, USA) color image of size 606 x 480. Figure 2 shows CED and our proposed ACED at iterations T = 10, 100, and 300. As can be seen, we obtain two different behaviors with respect to the coherency of long level lines. CED obtains a long flowing structures whereas our ACED obtains long lines interspersed with small flowing lines inside big structures. This property shows that our adaptive choice of diffusivities (8) helps the flow retain comers and singularities better than the original (7).

Next in Figure 3 we compare CED flows on noise-free and noisy (additive Gaussian noise of standard deviation [[sigma].sub.n] = 30 added in all three channels independently) Baboon color image of size 512 x 512. Figure 3(a) shows comparison on noise-free image and the corresponding CED, proposed ACED results at the iteration T = 25. Our proposed ACED obtains better coherency as can be seen by mouth and surrounding whiskers. A similar visual analysis shows in noisy case, Figure 3(b), indicate we obtain better noise removal while maintaining all the salient edges and thin linear structures. These are further corroborated by the corresponding residue images showing how much of structure and random noise are removed.

We note that for a fair comparison to the original CED formulation we kept all the parameters in our proposed ACED the same including the terminal time T of the corresponding PDE flows. The convergence result for the discretized versions, as iterations increases t [right arrow]> [infinity], of both CED and proposed ACED are the same and we defer the discussion of deeper theoretical results of the corresponding nonlinear PDEs for future.

To quantitatively compare the noise removal and structure preservation we use two standard error metrics utilized widely in the image processing literature. Peak signal to noise ratio (PSNR) is given by,

PSNR = 20 * [log.sub.l0] (3 x [u.sub.max] / [square root of MSE]) dB,

where MSE = [(mn).sup.-1] [SIGMA][SIGMA] [(u - [u.sub.O]).sup.2], mean squared error, with [u.sub.O] is the original (noise free) image, m x n denotes the image size, [u.sub.max] denotes the maximum value, for example in 8-bit images [u.sub.max] = 255. A difference of 0.5 dB can be identified visually. Mean structural similarity (MSSIM) index is in the range [0,1] and is known to be a better error metric than traditional signal to noise ratio. It is the mean value of the structural similarity (SSIM) metric [6], We use the default parameters for SSIM and the MATLAB code is available online (1). The SSIM is calculated between two windows [[omega].sub.1] and [[omega].sub.2] of common size N x N, and is given by,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII],

where [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] the average of [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] the variance of [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] the covariance, and [c.sub.1], [C.sub.2] stabilization parameters. The MSSIM value near 1 implies the optimal denoising capability of a method and we used the default parameters.

Table 1 and Table 2 show PSNR (dB) and MSSIM values for CED and our proposed ACED methods compared on some standard color test images which are synthetically perturbed by additive Gaussian noise of standard deviation [[sigma]sub.n] = 30. Though our proposed ACED is not the top performing method in all the tested images in terms of PSNR, we note that the purpose of adaptive CED is to obtain smoothed images with coherent structures in tact. Further, PSNR is known to be not the right metric in evaluating the performance of denoising methods and MSSIM is more apt. We remark that the optimal stopping time T > 0 for denoising is determined based on best possible MSSIM value in these synthetically noise added cases.

Finally, we show in Figure 4 some smoothing results from mobile phone imagery (12 mega-pixel). Figure 4(a) shows a picture taken in day-light conditions with no flash and our proposed ACED obtains flow like small structures while keeping the bigger regions intact. A similar result is observed in Figure 4(b) where a night time image is captured with an in-built flash. The smoothing property of our flow provides visually pleasing results in both cases.

4 Conclusions

In this work, we considered a new PDE based filter for color image coherence enhancing smoothing and noise removal. By a combination of anisotropic diffusion with structure tensor driven adaptive functions, our method obtains edge preserving smoothing results which result in better noise removal capabilities. Experimental results on a variety of noisy images indicate the potential of our proposed approach and compared with other original coherence enhancing diffusion filter we obtained better restoration results as well. One of our important future work is in extending the proposed method by incorporating other adaptive diffusive regularizers and to handle mixed noise removal [3] and consider multispectral imagery [7].

V. B. Surya Prasath

Computational Imaging and VisAnalysis (CIVA) Lab, Department of Computer Science

University of Missouri-Columbia, USA

E-mail: prasaths@missouri.edu

Received: July 7, 2016

References

[1] G. Aubert and P. Kornprobst (2006) Mathematical problems in image processing: Partial differential equation and calculus of variations, Springer-Verlag.

[2] P. Perona and J. Malik (1990) Scale-space and edge detection using anisotropic diffusion, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 12, No. 7, pp. 629-639.

[3] D. N. H. Thanh, S. D. Dvoenko, D. V. Sang (2016) A mixed noise removal method based on total variation, Informatica, Vol. 40, pp. 159-167.

[4] J. Weickert (1998) Anisotropic diffusion in image processing. B.G. Teubner-Verlag, Stuttgart, Germany.

[5] J. Weickert (1999) Coherence-enhancing diffusion of colour images, Image and Vision Computing, pp. 201-212.

[6] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612.

[7] V. B. S. Prasath and A. Singh, Multispectral image denoising by wellposed anistropic diffusion with channel coupling, Inter. J. Remote Sens, Vol. 31, No. 8, pp. 2091-2099, 2010.

(1) https://ece.uwaterloo.ca/ z70wang/research/ssim/

Table 1 : Mean structural similarity (MSSIM) metric values for results of various schemes with noise level [[sigma].sub.n] = 30 for standard test color images from USC-SIPI Miscellaneous dataset (size ([dagger]) = 256 x 256 and ([double dagger]) = 512 x 512). MSSIM value closer to one indicates the higher quality of the denoised image. The top result in each color test image is indicated by boldface. Image CED [5] Ours Baboon ([double dagger]) 0.6047 0.6087# Barbara ([double dagger]) 0.7129 0.7964# Boat ([dagger]) 0.6786 0.7013# Car ([double dagger]) 0.7214 0.7853# Couple ([dagger]) 0.7016 0.7020# F-16 ([dagger]) 0.8699 0.8898# Girl1 ([dagger]) 0.7081 0.7174# Girl2 ([dagger]) 0.8210 0.8522# Girl3 ([dagger]) 0.8020 0.8309# House ([dagger]) 0.7024 0.7812# IPI ([dagger]) 0.8335 0.8929# IPIC ([dagger]) 0.7899 0.8125# Lena ([double dagger]) 0.7581 0.7824# Peppers ([double dagger]) 0.7737 0.8026# Splash ([double dagger]) 0.7938 0.8136# Tiffany ([double dagger]) 0.7633 0.8107# Tree ([dagger]) 0.7099 0.7325# Note: The top result in each color test image is indicated by boldface is indicated with #. Table 2: PSNR (dB) values for results of various schemes with noise level [[sigma].sub.n] = 20 for standard images of size ([dagger]) = 256 x 256 (Noisy PSNR = 22.11) and ([double dagger]) = 512 x 512 (Noisy PSNR = 22.09). Higher PSNR value indicate better denoising result. The top result in each color test image is indicated by boldface. Image CED [5] Ours Baboon ([[dagger] dagger]) 20.48 20.53# Barbara ([double dagger]) 24.59 25.94# Boat ([double dagger]) 25.21# 24.98 Car ([double dagger]) 26.01# 25.83 Couple ([dagger]) 27.18# 26.97 F-16 ([dagger]) 24.56 26.50# Girl1 ([dagger]) 26.66 27.21# Girl2 ([dagger]) 29.34# 28.05 Girl3 ([dagger]) 29.92# 29.68 House ([dagger]) 28.95# 28.39 IPI ([dagger]) 30.38# 29.32 IPIC ([dagger]) 29.05# 28.64 Lena ([double dagger]) 28.56# 27.89 Peppers ([double dagger]) 28.61# 28.03 Splash ([double dagger]) 31.65# 31.18 Tiffany ([double dagger]) 29.67# 28.99 Tree ([dagger]) 25.73# 25.10 Note: The top result in each color test image is indicated by boldface is indicated with #.

Printer friendly Cite/link Email Feedback | |

Author: | Prasath, V. B. Surya |
---|---|

Publication: | Informatica |

Article Type: | Report |

Date: | Sep 1, 2016 |

Words: | 2940 |

Previous Article: | Statistic-based dynamic complexity measurement for web service system. |

Next Article: | Rough-mereology framework for making medical treatment decisions based on granular computing. |

Topics: |