A High-Resolution, Large-Area, Digital Imaging System

Charge-coupled-device (CCD) is the current dominant technology for both user and industry applications. CCD sensors provide higher resolution, wider dynamic range, and higher sensitivity, with which conventional photography can not compete. However, CCD sensors also cost more. In some applications like medical imaging which require both high resolution and large area, the device can be even more expensive. How to achieve large area and high resolution, while at the same time remaining cost-effective is the main concern of this research.

One of the economic solution to large area is to use several CCDs side by side. This method is technically called butting. Butting technique reduces system cost dramatically, but also causes additional distortions to the captured images. In this project, we are to develop algorithms to recover distortions, which include blur, noise, radiometric and geometric distortion, and missing data along the butting edge to achieve high-resolution, large-area imaging with low cost.

We first analyze the sources of the distortions in our imaging system. We then propose different approaches and present experimental results for image correction and image restoration. In the end, a list of publications and presentation are provided.

In our project, we use devices from EEV, Inc. EEV developed a light-guide system to butt CCD arrays together. The light-guide system uses fiber optics where a bunch of fibers connect each point on the scintillator with a corresponding cell on the CCD detector. An array of 1 x 2 CCDs (1152 rows and 1242 columns each) are used for experimental purpose (Fig. 1).

  

Figure 1: Two scintillator/fiber/CCD combinations butted together.

It is the use of fiber-optic tapers that causes radiometric and geometric distortions. No matter how precise the compression process, the cutting surface of the taper can not perfectly match the size of the CCD array. Usually, the taper is more compressed than it should be in order to avoid missing data sensed at the boundary of the outer-surface of the taper. The compression thus causes the pincushion distortion, a well-known type of geometric distortion. The distortion is not symmetric, exhibiting some degree of shear effect (Fig. 2).

Furthermore, because of the shape of the taper, the fibers at the boundary need to travel a longer distance than those at the center, and the light transfer is less efficient when the fiber is not normal to the CCD surface. Thus, there is an intensity gradation from the center to margin like a cross section of a tree, or a set of concentric ellipses (Fig. 2). This kind of distortion is called the vignetting-type radiometric distortion, as it resembles the vignetting which results from an imperfect lens.

  

Figure 2: Vignetting-type radiometric distortion and geometric distortion.

Other distortions include

• blur: comes from the finite-sized X-ray source
• noise: comes from three sources
  • dark current noise: which can be reduced by cooling;
  • quantum noise: a signal-depend noise which can be modeled by a Poisson process. When the readout rate is very high and the amount of the readout data is very large, this kind of noise can be neglected;
  • fixed noise: due to the electrical system noise, quantization noise, etc., which is the major noise source in our system.
• CCD detector induced errors: butting technique can cause data at the boundary of adjacent CCDs missing. Defects in the metallization or fibers can introduce single point defects as well.

Image correction and restoration have similar objectives. They both try to undo distortions and degradations caused by imperfections or physical limitations of the imaging system. Here, image correction refers to the radiometric and geometric corrections.

We extend the traditional polynomial approximation method to model the radiometric distortion by a set of elliptical equations, and to correct the geometric distortion by using a higher degree polynomial. We also implement the thin-plate spline interpolation technique, and are the first to adopt it for geometric correction.

Following are the experimental results of image correction. Fig. 3 shows the correction results of a grid frame and a blank frame.

  

Figure 3: Results from radiometric correction of a grid template (left) and a flat frame (right).

To compare the results more precisely, we draw the histograms of these two images, shown in Fig. 4. We can see that the histogram of the measured grid template has two peaks: one is from dark grid lines; the other smoother peak is from white pixel areas which shows the shape of a Gaussian because of radiometric distortion. The histogram of the corrected grid template also has two peaks: one is from dark grid lines; the other peak is an impulse (Dirac delta function) instead of a Gaussian, which indicates the successful correction. Histograms of the measured flat frame and the corrected frame show similar characteristics as those of the grid template except that the smaller peak is from the boundary of the image area.

  

Figure 4: Comparison of histograms of the measured images (top), and the corrected images (bottom). In each figure, the left plot is for the image from the left sensor, and the right plot is that from the right sensor.

Fig. 5 is a set of profiles used to measure performance of the algorithm in further detail. These profiles are from the same columns/rows of the measured grid template image and the corrected image. Profiles on the left column are from the measured image which show a Gaussian envelope; while the profiles to the right are from the corrected image that show a line envelope.

  

Figure 5: Comparison of profiles from the same columns and row of the measured grid template image and the corrected image.

We implement both polynomial approximation and TPS interpolation to correct pincushion distortion. Control points are chosen to be equally distributed across the whole image. Fig. 6. displays the corrected results by both methods with different numbers of control points chosen. It is not that apparent as to which graph shows better result. Therefore, we need to use some quantitative measurement to compare the results. Fig. 7 and Fig. 8 are two plots of error rate comparison and cross correlation coefficients comparison.

  

Figure 5: Correction results from both polynomial approximation (left column) and TPS interpolation (right column) with respect to different numbers of control points. From top to bottom: 33 x 33, 17 x 17, 9 x 9, and 5 x 5. The artifacts in (b) and (d) are explained later.

  

Figure 6: Comparison of error rates of polynomial approximation and TPS interpolation as a function of the number of control points.

  

Figure 7: Comparison of cross correlation coefficient between different correction results and the template.

From the above comparisons, we draw the following conclusions: (1) In general, TPS interpolation performs better than polynomial approximation. Except that with 5 x 5 control points, TPS has a lower error rate and a higher cross correlation coefficient than those of the polynomial approximation. (2) The quality of the corrected image relates to the number of control points being chosen. Surprisingly, it is not true that the more control points, the less the error rate. The corrected image has the lowest error rate and highest correlation coefficient with 9 x 9 control points. The reason is stated as follows:

• The interpolation approach can achieve exact mapping based on the assumption that the control point positions are known exactly. In real applications, however, the positions of the control points in the measured image can only be determined approximately. Therefore, the more control points chosen, the more the potential error might accumulate, resulting in the severe local distortion in the corrected images when choosing 33 x 33 control points, and 17 x 17 control points, as can be observed in Fig. 5. More local distortions exist in the correct image with 33 x 33 control points than that with 17 x 17 control points.
• For approximation approach, the more control points chosen, the more tendency the mapping functions show towards discontinuity. In other words, the mapping function might fit well for the control points, but not globally for all the points in the image. Therefore, unless we choose enough control points (e.g. all the points in the image), or we will get more errors when continue increasing the number of control points.

Here's the image correction result of a real x-ray image.

  

Figure 8: Image correction result of a real X-ray image (animal tail bone).

The image restoration problem can be formulated as Fig. 4. An original image f is slightly blurred by a point spread function (PSF) h from the X-ray source. The blurred image is further corrupted by fixed noise (n) and defects (d) from the CCD detector. g is the so called measured image.

  

Figure 8: System model.

We propose two approaches to estimate missing data along with deblurring and denoising: the consistency method using separable deblurring, and a maximum a-posteriori probability (MAP) method, in which the optimization is solved using Mean Field Annealing (MFA).

The basic idea behind both of our approaches is to make use of the point spread function (PSF): before a pixel is missed, it has already distributed its information to its neighbors through the effect of blur. Fig. 5 illustrates this process, where each block indicates a pixel, and the blocks marked with "0"s are the missing pixels. The figure shows that before pixel C is missed, it spreads its information to the neighbors g1, g3, g4, g6, g7, and g9.

  

Figure 9: Information distribution by blur.

If we can obtain a measurement of the PSF, in theory, we can reconstruct the missing data.

By putting some restrictions on input data, and the blur operator, we are able to transform the missing data estimation problem into a well-posed one. The consistency method is proposed based on the assumption that the blur kernel is separable, such as Gaussian. The separability property of Gaussian leads to the separable deblurring where two N x N sparse matrices are generated based on the separated blur kernels. We demonstrate that this transformed problem is well-conditioned. We also analyzed the conditioning of the problem when relaxing the restrictions to various degrees.

MFA based MAP is another approach we proposed to solve the problem of missing data estimation with denoising and deblurring. The MAP approach may be regarded as a Bayes interpretation to the regularization theory. It eventually formulates an image restoration problem into finding the optimal solution of an objective function which is very similar to the traditional regularization method, but derived from a different point of view.

With the complete set of assumptions (the blur kernel is exactly known, the blur kernel is separable, the original image is of integer type, and no noise is inserted) satisfied, the consistency method using integer criterion can recover the missing column exactly from the measured image. The consistency method using neighbor least square error (NLSE) criterion shows better results than the MFA method in deblurring, as shown in Fig. 10.

  

Figure 10: Restoration of integer images blurred by an exactly known kernel.

A cross-correlation comparison between the original image and the restored one using different methods is another way to compare the algorithm performance, as shown in Fig. 11.

  

Figure 11: Cross-correlation comparison.

The consistency method with integer criterion always has the highest cross-correlation coefficient, which is 1. The consistency method with NLSE criterion shows better correlation than the MFA method most of the time except for sinusoid images.

Fig. 12 exhibits a stripe of the grid template image, and its restoration results using both the consistency method and the MFA method.

  

Figure 12: Restoration results of a stripe of grid template image with flash light source. From left to right: the measured image, restored image by NLSE, restored image by MFA.

The restored image from the consistency method is very similar to that from the MFA method, but the consistency method is more efficient according to the complexity analysis.

To summarize, the consistency method works better when the noise is very small, and the blur kernel is estimated with high accuracy. Performance of the consistency method may be largely affected by perturbations in the estimated blur matrix, or by noise, with perturbations in noise playing a more important role. The MFA method behaves more stable for large noise or inaccuracy in the blur kernel estimation.

• H. Qi, A High-Resolution, Large-Area, Digital Imaging System, (pdf version)(html version) Ph.D. Dissertation, North Carolina State University, August, 1999.

• H. Qi, Powerpoint Slides of Dissertation Oral Defense, August, 1999.

• H. Qi, W. E. Snyder, G. L. Bilbro, "Missing data estimation by separable deblurring", Proceedings for the IEEE International Joint Symposia on Intelligence and Systems, May, 1998, pp348-353.

• H. Qi, W. E. Snyder, G. L. Bilbro, "Comparison of mean field annealing and multiresolution analysis in missing data estimation", Computer Vision - ACCV'98: Third Asian Conference on Computer Vision, v1, pp722-729, Hong Kong, China, January 8-10, 1998.