This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.
Various types of wavelet transforms are available for image fusion. The most widely used discrete wavelet transform is the critically-sampled DWT. The critically sampled DWT suffers from shift sensitivity and directionality. Improved performance can be obtained by using an expansive, over complete or redundant transform. An expansive transform, undecimated discrete wavelet transform (UDWT) is shift-invariant, which has expansive factor J + 1 for J level of scale of decomposition. The double-density discrete wavelet transform (DDWT) provides a compromise between the UDWT and the critically-sampled DWT. None of these expansive transforms increase the sampling with respect to scale of decomposition. An expansive dyadic wavelet transform, namely Higher Density Discrete Wavelet Transform (HDWT) oversamples both scale and frequency by a factor two. This transform has intermediate scales, i.e. one scale between each pair of scales of the critically-sampled DWT. Similar to DDWT, it uses three filters, one scaling and two wavelet filters. However, one of the wavelet filters is band pass instead of high-pass filters. And also the high pass filter is not down sampled and up sampled during the analysis and synthesis. In this chapter, HDWT is used to fuse two images focusing different parts of the same scene of size 480 x 640 taken by the digital camera.
5.1 Higher Density Discrete Wavelet Transform
The most commonly used wavelet transform for image fusion is critically sampled Discrete Wavelet Transform (DWT) which can be implemented using perfectly reconstructed Finite Impulse Response filter banks. But, critically sampled DWT suffers from four shortcomings namely Oscillations, Shift variance, poor directionality and aliasing. Shift variance in critically sampled discrete wavelet transform exists due to down sampling during analysis and up sampling during synthesis.
Figure 5.1: Frequency and Scaling plane of DWT, DDWT and HDWT
Improved performance can be obtained by using an over complete or redundant transform. The over complete and redundant wavelet transforms are called as expansive transform. An expansive transform is one that expands an N point signal to M transform coefficients with M > N. For example, the Undecimated Discrete Wavelet Transform (UDWT) is expansive by the factor J+1, when J scales are implemented for 1D signals and expansive by the factor 3J+1 for 2D signals. It shows improved results for enhancement of images due to its shift invariant property.
Complex Wavelet Transform (CWT) is also an alternate and uses complex valued filtering that decomposes the real or complex signal into real and imaginary parts in transform domain. It is approximately shift invariant and directionally selective in higher dimensions. It achieves this with a redundancy factor of only 2d for d-dimensional signals, which is lower than the UDWT.
The double-density discrete wavelet transform (DDWT) which provides a compromise between the UDWT and the critically-sampled DWT is two-times expansive, regardless of the number of scales implemented. Even so, the DDWT is approximately shift-invariant. To construct a DDWT with perfect reconstruction using FIR filters, it is necessary to have one low-pass filter and two high-pass filters.
These above said expansive transform do not increase the sampling with respect to frequency or scale. An expansive dyadic wavelet transform, namely High Density Discrete Wavelet Transform (HDWT) over samples both scale and frequency by a factor two. Like DDWT, at each scale of HDWT, there are twice as many coefficients as the critically sampled DWT. HDWT also has intermediate scales, it has one scale between each pair of scales of the critically-sampled DWT as shown in figure 5.1.
5.1.1 Filter Bank Structure of HDWT
Similar to DDWT, it uses three filters, one scaling filter 'Φ(t)' and two distinct wavelet filters namely 'Ψ1(t)' and 'Ψ2(t)'. The spectrum of the first wavelet Ψ1(ω) is concentrated between the spectrum of the second wavelet Ψ2(ω) and the spectrum of its dilated version Ψ2(2ω). However, the associated wavelet filter is band pass instead of high-pass filters. And also the high pass filter is not down sampled and up sampled during analysis and synthesis. As this wavelet transform samples the scale-frequency plane with a density that is three times the density of the non-expansive wavelet transform, this transform is expansive by a factor of three. The analysis and synthesis filter bank structure of HDWT is shown in figure 5.2.
Figure 5.2: Filter Bank structure of HDWT
Following the multi resolution frame work, the scaling function 'Φ(t)' and two wavelet functions 'Ψ1(t)' and 'Ψ2(t)' are defined through the dilation and wavelet equations as
The scaling function 'Φ(t)' and two wavelet functions 'Ψ1(t)' and 'Ψ2(t)' are defined in the above equations by the low pass scaling filter 'h0(n)', band pass wavelet filter 'h1(n)' and high pass wavelet filter 'h2(n)'. If hi(n) satisfy the perfect reconstruction condition and if Φ(t) is sufficiently regular, then the dyadic dilations and translations of Ψi(t) form a tight frame. Specifically, if Φk(t)= Φ(t-k), Ψ1,j,k (t)= Ψ1 (2jt-k) and Ψ2,j,k (t)= Ψ2 (2jt-k/2). Then for any square integrable signal f(t) is given by
For the input signal x(n) and output signal y(n) as shown in figure 5.2, the Z transform of y(n) is given using the standard multi rate identities as,
This can be written as, Y(z) = T(z) X(z) + V(z) X(-z) where
Then the condition for perfect reconstruction is given by,
To derive the low pass scaling filter 'h0(n)', band pass wavelet filter 'h1(n)' and high pass wavelet filter 'h2(n)' satisfying the perfect reconstruction condition, consider the factorization of H0(z) and H1(z) as
where L is the degree of Q(z) and α = 0 if L is odd and 1 if L is even. Then α + L is odd. The factorization of H0(z) and H1(z) follows that
so that the second condition for perfect construction is satisfied. To find H2(z) so that it satisfies the first condition for perfect reconstruction, it can be written as
If the right hand side of the equation 5.10 is positive on the unit circle z = ejω , the filter H2(z) can be obtained by spectral factorization.
5.1.2. 2-D Extension of HDWT
To use the higher density discrete wavelet transform for 2-D signal processing, it is necessary to implement a two-dimensional HDWT. This can be simply done by alternately applying the transform first to the rows, then to the columns of an image, as usual separable 2D implementation. This is shown in figure 5.3, in which 1-D over sampled filter bank is iterated on the rows first and then on the columns. This gives rise to nine 2-D sub bands namely LL, LB, LH, BL, BB, BH, HL, HB and HH where L stands for Low pass filter, B stands for Band pass filter and H stands for High pass filters. The relative sizes of the sub bands for an input 2D signal of size N X N is tabulated in table 5.1. In comparison, the critically-sampled two dimensional DWT has three sub bands namely LH, HL, HH and each of size N/2 X N/2. Therefore, the two dimensional extension of HDWT is only 5-times expansive. This is substantially less than the two dimensional form of the undecimated DWT, which is (3 J + 1)-times expansive, where J is the number of stages.
Figure 5.3: 2D Higher Density Discrete Wavelet Transform
Table 5.1: Relative Size of Sub bands of HDWT
N/2 X N/2
N/2 X N/2
N/2 X N
N/2 X N/2
N/2 X N/2
N/2 X N
N X N/2
N X N/2
N X N
6N X 6N
5.2 Image Fusion Based on HDWT
In this approach, three methods are applied to fuse two images focusing different parts of the same scene of size 480 x 640 taken by the digital camera using HDWT.
Method 1: This method uses average for low frequency sub bands and uses absolute maximum fusion rule for medium and high frequency bands. After taking HDWT transforms, the transformed image contains one low frequency sub bands and eight numbers of medium and high frequency sub bands. The low frequency sub bands contain the average information whereas the medium and high frequency sub bands contain directional and edge information. So, a good image fusion algorithm should deal the low frequency and high frequency sub bands separately. Since low frequency approximation sub bands contain the average image information, average of low frequency approximation sub band of both source images are taken to form low frequency approximation sub band of the fused image. To transfer the directional and edge information present in the high frequency sub bands to the fused image, a good integration rule is to use maximum absolute value of the coefficients over 3 x 3 or 5 x 5 window of each image patch as an activity measure. The coefficient from the source image, whose activity measure is larger, is chosen to form the fused wavelet coefficient of the medium and high frequency sub band at the corresponding locations.
Method 2: This method uses the absolute maximum fusion rule for low frequency sub bands whereas the salience match measure based fusion rule is applied to medium and high frequency sub bands. Since low frequency approximation sub bands contain the average image information, larger absolute transform coefficients of these sub bands correspond to sharper brightness changes. So, maximum absolute value of the coefficients over 3 x 3 or 5 x 5 window of each image patch is used as an activity measure. The coefficient from the source image, whose activity measure is larger, is chosen to form the fused wavelet coefficient of the low frequency sub band at the corresponding locations. For each high frequency sub band pair, the salience of a feature is computed as a local energy in the neighborhood of a coefficient.
where w(q) is a weight and w(q)=1. At a given resolution level j, this fusion scheme uses two distinct modes of combination namely Selection and Averaging. In order to determine whether the selection or averaging will be used, the match measure M(p) is calculated as
If M(p) is smaller than a threshold T, then the coefficient with the largest local energy is placed in the composite transform while the coefficient with less local energy is discarded. The selection mode is implemented as
Else if M(p) T, then in the averaging mode, the combined transform coefficient is implemented as
where . A binary decision map of same size as the wavelet sub band is then created to record the selection results. This binary map is subject to consistency verification. Specifically in the transform domain, if the center pixel value comes from image A while the majority of the surrounding pixel value come from image B, the center pixel value is switched to that of image B. In the implementation, a majority filter is applied to the binary decision map. A fused image is finally obtained based on the new binary decision map. This selection scheme helps to ensure that most of the dominant features are incorporated into the fused image.
Method 3: The structure of proposed HDWT based image fusion is shown in figure 5.4. In this method, the focus measure spatial frequency is used as an activity measure for fusing wavelet coefficient of all sub bands of HDWT transformed source images to form the fused image.
Figure 5.4: Structure of proposed HDWT based Image Fusion
The pyramid image using HDWT is created, and then canny edge detector is applied to all sub bands of the image. After the edge detection, region segmentation is performed based on the edge information using region labeling algorithm. In the labeled image, zero corresponds to the edges and other different values represent different regions in the image. Then the focus measure namely Spatial Frequency using the formula given by the equation 5.15 is calculated in all regions as an activity measure to form the fused image.
where RF and CF are row and column frequency respectively and given by,
The spatial frequency of the corresponding regions of the two input source images are compared to decide the coefficient of which source image should be used to construct the wavelet coefficient of the fused image. The wavelet coefficient of the source image whose spatial frequency is higher for the particular region is selected to form the wavelet coefficient of the fused image as shown by the equation 5.18.
Finally, inverse HDWT transform is applied to get the fused image.