JP2006211218A - Apparatus and method for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a method for image processing changing the direction and magnitude of edge enhancement according to the radial direction or the azimuth angle direction from the center of the screen, and correcting image deterioration produced in the meridional direction and the sagittal direction caused by the design and manufacturing reasons of an optical imaging system. <P>SOLUTION: The image processing apparatus for processing signals from an imaging apparatus having an axisymmetric optical imaging system includes a means for obtaining a coordinate on an imaging plane having a cross point of the optical axis of the above optical imaging system and an imaging plane of the above imaging apparatus as an origin. Also, the image processing apparatus includes an edge enhancement means for performing edge enhancement of an edge perpendicular to the radius direction of the above coordinate, and an edge enhancement means for performing edge enhancement of an edge perpendicular to the azimuth angle direction of the above coordinate. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for processing a signal from an imaging apparatus having an axially symmetric imaging optical system such as a camera and an endoscope.

  In an imaging apparatus such as a camera or an endoscope, an imaging optical system such as a lens having an axially symmetric shape with respect to one optical axis is generally used. The imaging characteristics of such an imaging optical system change on the imaging surface only in relation to the distance from the center of the screen in the radial direction, and specifically, this is spherical aberration, astigmatism, coma. Aberration, image surface distortion, chromatic aberration, etc. appear as image degradation. Conventionally, in order to correct image deterioration due to these aberrations, an imaging optical system such as a lens is optimized and configured to minimize these aberrations. In image processing, methods such as edge enhancement are used to prevent image contrast deterioration.

  However, with the downsizing and increase in the number of pixels of the image sensor, the pitch per pixel is becoming smaller and it is becoming difficult to correct aberrations of the lens. In particular, when the F number of the imaging optical system is reduced, image degradation becomes conspicuous due to light diffraction. This is particularly noticeable near the periphery of the screen. FIG. 5 illustrates this state. When the imaging optical system shown in FIG. 51 forms an image on the screen 52, the light beam collected on the center of the screen, the so-called axis, is 53 if the cross section of the lens, the diaphragm, etc. constituting the lens barrel is circular. The pupil shape spreads out evenly in a circle as shown in FIG. On the other hand, the light beam collected off the periphery of the screen, that is, off-axis, has a pupil size that is generally smaller than that on the axis due to the effects of vignetting, and the shape is shown in the radial direction of the screen as shown in FIG. The so-called meridional direction is different from the azimuthal direction, so-called sagittal direction. In the case of a general imaging optical system, the meridional direction is more susceptible to vignetting, and the solid angle of the light beam collected in the meridional direction is larger than the solid angle of the light beam collected in the sagittal direction. It tends to be small.

This means that, off-axis, the F-number tends to darken in the meridional direction, and image degradation due to diffraction tends to occur in the meridional direction, that is, the screen direction. Furthermore, the downsizing and the increase in the number of pixels of the image pickup element make the manufacturing accuracy of the image pickup optical system more severe, and easily cause image degradation due to the manufacturing error of the image pickup optical system. This is because, for example, the best focus plane obtained in the meridional direction of the screen, the so-called meridional image plane 55, and the best focus plane obtained in the sagittal direction of the screen, the so-called sagittal image plane 56 deviate from the designed ideal image plane 52. Appears as In general, the influence of the manufacturing error is more easily received on the meridional surface than on the sagittal surface. Therefore, the image is more likely to deteriorate in the meridional direction. However, depending on the manufacturing state, the meridional surface 55 is more ideal than the sagittal surface 56. The image may be close to the surface 52 and image degradation in the sagittal direction may occur. Furthermore, the influence of the manufacturing error of the imaging optical system does not occur evenly in the azimuth direction of the screen, but a phenomenon called “single blurring” occurs in which the degree of image deterioration differs depending on the azimuth angle of the screen. In such a case, the degree of degradation of the image varies between the meridional direction and the sagittal direction depending on the azimuth direction of the image. Japanese Patent Application Laid-Open No. 2003-172873 discloses a technique for correcting image degradation in an imaging optical system. In this configuration, a rotationally symmetric filter has been proposed as a filter for repairing a deteriorated image, and the degree of repair of deterioration in the meridional direction and the sagittal direction cannot be changed. In addition, the proposal proposes a method for defocusing the focus position of the imaging optical system from the ideal image plane so that the image degradation in the meridional direction and the sagittal direction can be made comparable to enable image restoration to be rotationally symmetric. Has also been proposed.
JP 2003-172873 A

  However, this method has a problem that the image near the center of the screen is deteriorated.

  In order to achieve the above object, an image processing apparatus and an image processing method of the present invention have the following configurations.

  That is, an image processing device that processes a signal from an imaging device having an axially symmetric imaging optical system, on the imaging surface with the intersection point between the optical axis of the imaging optical system and the imaging surface of the imaging device as an origin It has means for obtaining coordinates, and has two edge enhancement means: edge enhancement means for enhancing edges perpendicular to the radial direction of the coordinates, and edge enhancement means for enhancing edges perpendicular to the azimuth direction of the coordinates An image processing apparatus and an image processing method are provided.

  As described above, according to the embodiment of the present invention, the direction and size of edge enhancement can be changed from the center of the screen according to the radial direction or the azimuth angle direction. It is possible to repair the deterioration of the image that occurs in the meridional direction and the sagittal direction.

  Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the configuration of the present invention.

  In the image processing apparatus and the image processing method shown in FIG. 1, a subject is imaged on an imaging element in an imaging unit 2 by an imaging optical system 1 that is symmetric with respect to the optical axis. The imaging unit 2 is provided with a parameter storage unit 3 in which various parameters for outputting signals from the image sensor to the image are stored. The imaging performance of the imaging optical system 1 in the meridional direction and the sagittal direction is provided. In addition, information indicating the center position (x0, y0) on the imaging surface is also included in this. (X0, y0) is the intersection of the optical axis of the imaging optical system and the imaging surface of the imaging device. The signal obtained from the imaging unit 2 is the coordinate detection unit 4, and the coordinates (x, y) on the imaging surface with the position of (x0, y0) as the origin from the center position (x0, y0) and the address of the image signal Is calculated. The coordinates (x, y) are determined by the radial azimuth calculating unit 5 based on the radius and the azimuth, and based on these sizes, the imaging performance data of the imaging optical system 1 in the meridional direction and the sagittal direction. Can be obtained. Here, the radius and the azimuth are obtained by the following equations.

Radial radius: r = SQRT (x ² + y ² )
Azimuth: φ = arctan (y / x)
Let this be the square of the radial length r ² and the tangent value tanφ of the azimuth,
r ² = x ² + y ²
tanφ = y / x
Thus, if this value is obtained by the radial azimuth calculating unit 5 and the magnitude of the radial and azimuth is determined, the amount of calculation can be reduced. The radial and azimuth angles obtained by the radial azimuth calculation unit 5 are input to the BPF creation unit 6, and a bandpass filter used for edge enhancement at image coordinates (x, y) from these values. Is created. FIG. 2 shows an example of the characteristics of the bandpass filter for edge enhancement used in this embodiment. 2A and 2B show the characteristics of the bandpass filter when the coordinates (x, y) are away from the center of the imaging surface to the upper right. FIGS. 9A and 9B show the characteristics of a band pass filter that is suitable when the influence of image degradation is greater in the meridional direction than in the sagittal direction. When the coordinates (x, y) move away from the center of the imaging surface to the upper right, the deterioration of the image in the meridional direction, that is, in the radial direction increases as shown in (A) to (B) of the figure. In addition, the bandpass filter for edge detection is created in the BPF creation unit 6 so that the bandpass filter for further emphasizing the frequency component in the same direction as the radial direction is gradually switched from the characteristic uniform to the direction. FIGS. 2C and 2D show the characteristics of the bandpass filter when the coordinates (x, y) are separated from the center of the imaging surface to the upper right. FIGS. 3C and 3D show the characteristics of a band pass filter that is suitable when the influence of image degradation is greater in the sagittal direction than in the meridional direction. As the coordinates (x, y) move away from the center of the imaging surface to the upper right corner, when the sagittal direction, that is, the azimuth direction of the image deteriorates greatly, as shown in (C) to (D) of the figure In addition, the edge enhancement band-pass filter is created in the BPF creation unit 6 so that the band-pass filter gradually emphasizes the frequency component in the same direction as the azimuth direction from the characteristic uniform to the direction. As described above with reference to FIG. 2, a bandpass filter that changes edge enhancement in a certain direction in accordance with the size of the radius is calculated by the radius azimuth calculation unit 5 and the radius of the coordinates (x, y) and An azimuth angle is obtained, a bandpass filter for edge enhancement in the radial direction is created by the radial direction BPF creation unit 8, and a bandpass filter for edge enhancement in the azimuth direction is created by the azimuth direction BPF creation unit 9. Can be created by adding them together at an appropriate mixing ratio in the mixing unit 10. This mixing ratio is determined by the imaging performance of the imaging optical system 1 stored in the parameter storage unit 3 in the radial direction, that is, the meridional direction and the azimuth direction, that is, the sagittal direction, depending on the size of the moving radius of the screen, that is, the image height. Thus, a bandpass filter having the characteristics shown in FIGS. 2A, 2B, 2C, and 2D is created. If the degree of image degradation differs depending not only on the radius but also on the azimuth angle due to the effects of manufacturing errors, etc., the parameter storage unit 3 changes depending on the azimuth angle in addition to the radius. Information on the imaging characteristics to be held may be held, and the mixing ratio given by the mixing unit 10 may be determined by a combination of the radius vector and the azimuth angle. FIG. 6A shows an example of a bandpass filter that performs edge enhancement in the radial direction of coordinates, and FIG. 6B shows an example of a bandpass filter that performs edge enhancement in the azimuth direction of the same coordinates. When the bandpass filter shown in (A) and (B) of the figure is configured with a 5 × 5 tap digital filter in the azimuth angle θ = 0 °, that is, in the horizontal direction of the screen, the following coefficients are taken. Can be realized.

The radial enhancement direction and the edge enhancement filter in the azimuth direction at an arbitrary azimuth angle θ can be obtained by rotating the original bandpass filter at an arbitrary angle θ as follows. Here, the original band-pass filter is an M × N tap, and the i-th coefficient in the horizontal direction and the j-th coefficient in the vertical direction are a [i, j]. That is,
BPF = (a [i, j])
(I = 1, ..., M, j = 1, ..., N)
It is. At this time, the following functions are defined.

h (x, y) = Σ (i = 1, M) Σ (j-1, N)
{a [i, j] · sinc (x + i- (M + 1) / 2) · sinc (y + j- (N + 1) / 2)}
(However, sinc (x) = sin (πx) / (πx))
If this is sampled at coordinates rotated clockwise by θ, an edge-enhanced bandpass filter BPF (rot = θ) rotated by θ in the direction of the azimuth angle can be created. That is,
BPF (rot = θ) = (h (i · cosθ + j · sinθ, -i · sinθ + j · cosθ)
(I = 1, ..., M, j = 1, ..., N)
And it is sufficient. FIGS. 6C and 6D show examples in which the bandpass filters of FIGS. 6A and 6B are rotated by 60 °. For example, when the original bandpass filter is a 5 × 5 tap bandpass filter with the azimuth angle θ = 0 °, the edge enhancement filter rotated by 60 ° using the above method is as follows: .

  As described above, the bandpass filter shown in FIG. 6 (B) may calculate the coefficient of the digital filter having the direction of the frequency to be emphasized from the azimuth angle of the coordinate each time. Accordingly, a finite number may be prepared in advance. In addition, the mixing ratio of two or more digital filters may be changed so that the peak position moves in a desired direction from such a limited number of digital filters. . The edge enhancement bandpass filter created by the BPF creation unit 6 is sent from the center according to the radius and azimuth of the coordinate at the coordinate position away from the center of the imaging surface by the edge enhancement processing unit 7. Alternatively, the deterioration of the image in the meridional direction or the sagittal direction can be corrected by enhancing the edge having a structure perpendicular to the azimuth direction by a desired amount.

  FIG. 3 shows a second embodiment of the present invention.

  In this figure, the imaging surface is divided into 16 areas B to Q according to the vicinity of the center A and the range of the azimuth angle from the center, and the radial value calculated by the radial azimuth calculation unit 5 is less than a certain value. In some cases, the region is considered to be the A region, and the range of coordinates B to Q is determined by the azimuth value. In determining the azimuth angle, if the tangent value of the azimuth angle is used instead of the azimuth angle value itself, the amount of calculation can be reduced. If it is determined that the coordinates are in the area A in the figure, the image is near the center of the screen, and the deterioration of the image is small regardless of the radial and azimuth directions, and edge enhancement processing in this direction is not performed. If it is determined that the region is in the B to Q region, the bandpass filter rotated to have a peak in the direction of the radius of the B to Q region in the shape of FIG. 2A or the shape of FIG. A band pass filter rotated so as to have a peak in the direction of the azimuth angle of the B to Q region is adopted and used in the edge enhancement processing unit 7. By adopting such a configuration, it is possible to significantly reduce the amount of computation and storage of the BPF creation unit.

  If the tangent value of the azimuth angle of coordinates is used, the regions symmetric to the center of the B to Q regions, such as B and J, F and N, etc., are determined as the same tangent value. Since the pass filter may have the same characteristics, a good determination can be made with a small amount of calculation.

  FIG. 4 shows a third embodiment of the present invention.

  In the figure, in order to simplify the configuration of the BPF creation unit 6, the horizontal bandpass filter (H-BPF) 41, the vertical bandpass filter (V-BPF) 42, and the diagonal right bandpass filter (DR-BPF) 43 are used. , An oblique left band pass filter (DL-BPF) 44 and four multipliers 45, 46, 47 and 48 adders 49. The horizontal bandpass filter (H-BPF) 41 is for enhancing the horizontal component of the edge of the image. For example, the coefficient [-1/2, 0, 1, 0, -1/2] is used. A digital filter is used. Similarly, the vertical bandpass filter (V-BPF) 42 is used to emphasize the vertical component of the image edge, and a digital filter having the same coefficient as the horizontal bandpass filter (H-BPF) 41 is used. Used. The diagonal right band pass filter (DR-BPF) 43 is a digital filter having the following coefficients, for example:

The edge to the upper right of the image is emphasized. Similarly, the diagonal left bandpass filter (DL-BPF) 44 is a digital filter having the following coefficients, for example.

This emphasizes the diagonally upper left edge of the image. The diagonal right bandpass filter (DR-BPF) 43 and the diagonal left bandpass filter (DL-BPF) 44 are used to perform edge enhancement in the meridional and sagittal directions of an image. The role of edge enhancement varies depending on the azimuth angle of coordinates. In the multipliers 45, 46, 47, and 48, a horizontal bandpass filter (H-BPF) is selected in accordance with the radial and azimuth values of the coordinates calculated by the radial azimuth calculating unit 5 shown in FIG. ) 41, vertical bandpass filter (V-BPF) 42, diagonal right bandpass filter (DR-BPF) 43, diagonal left bandpass filter (DL-BPF) 44 coefficients to be multiplied by KH, KV, KDR, KDL Is determined. When the azimuth is nearly horizontal, the horizontal direction is the meridional direction and the vertical direction is the sagittal direction. If you want to emphasize the edge in the meridional direction more, the KH value will be larger than the others, and the edge in the sagittal direction If you want to emphasize more, the value of KV will be larger than the others. When the azimuth is close to vertical, the vertical direction is the meridional direction, and the horizontal direction is the sagittal direction. If you want to emphasize the edge in the meridional direction, the KV value is larger than the others, and the edge in the sagittal direction If you want to emphasize more, the value of KH will be larger than the others. When the azimuth angle is close to 45 °, the diagonal right direction is the meridional direction and the diagonal left direction is the sagittal direction, so if you want to emphasize the edge in the meridional direction, the KDR value will be larger than the others. If you want to emphasize more sagittal edges, the KDL value will be larger than the others. When the azimuth angle is close to 135 °, the diagonal left direction is the meridional direction and the diagonal right direction is the sagittal direction. Therefore, if you want to emphasize the edge in the meridional direction, the KDL value is larger than the others. If you want to emphasize more sagittal edges, the KDR value will be larger than the others. If the azimuth angle is between these, for example, between the horizontal direction and 45 ° direction, and you want to emphasize the edge in the meridional direction, KH and KDR should be larger than the others, and the ratio of both Determined by azimuth. Similarly, in other cases, the ratio of two bandpass filters that emphasize edges in the direction near the azimuth may be changed according to the azimuth. The ratio of the size of KH, KV, KDR, and KDL can be changed according to the size of the radius, and if the degradation of the image increases from the center to the periphery of the screen, The larger the diameter, the larger the ratio of KH, KV, KDR, and KDL, and the smaller the radius, the more uniform.

As described above, the magnitude of the radius is determined not by the radius r of the radius but by the square of the radius r ² , and the magnitude of the azimuth is determined by the tangent value tanφ of the azimuth φ. If it is determined, the amount of calculation can be reduced. The configuration of this embodiment is simpler than the above-described two embodiments, and is a good configuration when simplification of the hardware configuration is prioritized over the discrimination accuracy of the direction in which the edge should be emphasized.

The block diagram which shows the structure of the image processing apparatus and image processing method according to embodiment of this invention The figure showing the characteristic of the band pass filter for edge emphasis used by the present invention The figure explaining the edge emphasis method of 2nd Example of this invention. The figure explaining the structure of the 3rd Example of this invention Diagram explaining off-axis characteristics of imaging optical system The figure explaining the structure of the band pass filter shown in FIG.

Claims (16)

  An image processing device for processing a signal from an imaging device having an axially symmetric imaging optical system, wherein coordinates on the imaging surface with an intersection point between an optical axis of the imaging optical system and an imaging surface of the imaging device as an origin Two edge emphasizing means including an edge emphasizing means for emphasizing an edge perpendicular to the radial direction of the coordinates, and an edge emphasizing means for emphasizing an edge perpendicular to the azimuth direction of the coordinates. A featured image processing apparatus.   The image processing apparatus according to claim 1, wherein the two edge enhancement units change the ratio of edge enhancement according to the radius of the coordinate.   The image processing apparatus according to claim 1, wherein the two edge enhancement units change the ratio of edge enhancement according to the magnitude of the azimuth angle of the coordinates.   The image processing apparatus according to claim 1, wherein the image processing apparatus obtains information on a moving radius of the coordinates by obtaining a square of the moving radius of the coordinates.   The image processing apparatus according to claim 1, wherein the image processing apparatus obtains information on the azimuth angle of the coordinates by obtaining a tangent value of the azimuth angle of the coordinates.   The image processing apparatus according to claim 1, wherein the two edge enhancement units change an edge enhancement ratio according to a field angle of an imaging optical system.   The image processing apparatus according to claim 1, wherein the two edge enhancement units change an edge enhancement ratio according to a size of a stop of the imaging optical system.   The image processing apparatus according to claim 1, wherein the two edge enhancement units change an edge enhancement ratio according to a subject distance.   An image processing method for processing a signal from an imaging device having an axially symmetric imaging optical system, wherein coordinates on the imaging surface with an intersection point between an optical axis of the imaging optical system and an imaging surface of the imaging device as an origin Two edge emphasizing means including an edge emphasizing means for emphasizing an edge perpendicular to the radial direction of the coordinates, and an edge emphasizing means for emphasizing an edge perpendicular to the azimuth direction of the coordinates. A featured image processing method.   The image processing method according to claim 9, wherein the two edge enhancement means change the ratio of edge enhancement according to the radius of the coordinate.   The image processing method according to claim 9, wherein the two edge enhancement means change an edge enhancement ratio according to the magnitude of the azimuth angle of the coordinates.   The image processing method according to claim 9, wherein the image processing method obtains information on a moving radius of the coordinates by obtaining a square of the moving radius of the coordinates.   The image processing method according to claim 9, wherein the image processing method obtains information on the azimuth angle of the coordinates by obtaining a tangent value of the azimuth angle of the coordinates.   The image processing method according to claim 9, wherein the two edge enhancement means change an edge enhancement ratio in accordance with a field angle of an imaging optical system.   The image processing method according to claim 9, wherein the two edge enhancement means change an edge enhancement ratio according to a size of a stop of the imaging optical system.   The image processing method according to claim 9, wherein the two edge emphasizing units change the ratio of edge emphasis according to the size of the subject distance.

Images