JP2015091072A - Imaging apparatus, imaging system with the imaging apparatus, and false color removal method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus that is able to substantially prevent occurrence of false color when a blurred color image is restored.SOLUTION: An imaging apparatus 2 according to the invention comprises: an imaging optical system 3 that images a target 1 to be imaged; a color imager 4 that image-receives the target 1 as a blurred, defective image via the imaging optical system 3; a restoration processing section 5C configured such that, for each color obtained from the imager 4, a defective image is formed from pixel data output from each pixel, then, the defective image is subjected to an imaging process, thereby restoring an image, from which blur is removed; and an effective information nullifying section 5B configured such that, in order to prevent occurrence of false color, effective information used for restoring process on the saturated area side of the pixel data is nullified.

Description

  The present invention relates to an imaging apparatus capable of acquiring a high-definition color image by performing image processing on a blurred color image, an imaging system including the imaging apparatus, and a false color removal method.

  Conventionally, in an imaging apparatus, a phase plate is inserted in the optical path of the imaging optical system to expand the depth of focus, and image processing called restoration filter processing is performed on an image (degraded image) blurred by the expansion of the depth of focus. There has been proposed a technique for restoring the original high-resolution image by performing (see, for example, Patent Document 1).

  In this restoration filter process, a filter obtained by performing an inverse transform process on a point spread function (or a point spread function (PSF) or a so-called blur function) is used. Further, in order to accurately restore the original high-resolution image, the amount of light from each part to be imaged must be accurately acquired in each pixel of the image sensor corresponding to each part.

  However, when a light amount equal to or greater than the threshold is incident on the image sensor from a part of the imaging target, the output of the pixel of the image sensor corresponding to that part is saturated. For this reason, the pixel corresponding to the part of the image sensor cannot accurately acquire the amount of light from the subject, and accurate information on the amount of light corresponding to the part of the subject is lost.

  The original high-resolution image can be restored by the restoration process as long as each pixel corresponding to the part can accurately acquire the amount of light from the part to be imaged over the entire blur function spreading region. However, the pixels in the neighboring area where the pixel output is saturated (pixels included in the size of the filter used for image restoration) lacks accurate information on the amount of light. Cannot be restored.

As a similar technique, a technique for reducing the blur of a color image as much as possible by a color image restoration process has been proposed (see Patent Document 2).
By the way, a monochrome image is perceived by human eyes as a slight deterioration in resolution. However, in the case of a color image, for example, green (red) among red (R), green (G), and blue (B) images. If the restoration process is not accurately performed on the image of G) and the restoration process is correctly performed on the red (R) and blue (B) images, it is false when a color image is created by combining the images for each color. Color is generated. Since the human brain reacts sensitively to color, when a false color occurs, a sense of incongruity occurs.

  The present invention has been made in view of the above circumstances, and an imaging apparatus capable of preventing generation of false colors as much as possible when a color image having blur is restored, an imaging system including the imaging apparatus, and It is to provide a false color removal method.

  An image pickup apparatus according to the present invention is obtained from an image pickup optical system that picks up an image pickup target, a color image pickup element that receives the image pickup target as a deteriorated image having blur via the image pickup optical system, and the image pickup element. For each color, after generating the deteriorated image from the pixel data output from each pixel, a restoration processing unit that performs image processing on the deteriorated image to restore the image from which blur has been removed, and prevention of false colors In order to achieve this, the present invention is characterized by comprising a role information invalidation processing unit for converting the role information used for the restoration processing on the saturation region side of the pixel data to role information.

  According to the present invention, it is possible to prevent the generation of false colors as much as possible when a blurred color image is restored.

FIG. 1A is an explanatory diagram schematically illustrating an example of an imaging target. FIG. 1B is a schematic diagram illustrating an image corresponding to the imaging target illustrated in FIG. 1A and having no deterioration. FIG. 1C is a schematic diagram illustrating a deteriorated image having a blur corresponding to the imaging target illustrated in FIG. 1A. FIG. 2 is a schematic diagram showing a one-dimensional profile image obtained by the horizontal scanning line shown in FIG. 1B. FIG. 3 is a schematic diagram showing a one-dimensional profile image obtained by the horizontal scanning line shown in FIG. 1C. FIG. 4 is a schematic diagram showing a one-dimensional profile image when the luminance value of the deteriorated image shown in FIG. 1C is saturated. FIG. 5 is a schematic diagram showing a one-dimensional profile image that should be originally obtained when restoration processing is performed on the deteriorated image shown in FIG. 1C. FIG. 6 is a schematic diagram showing a one-dimensional profile image actually obtained when restoration processing is performed on the deteriorated image shown in FIG. 1C. FIG. 7 is a block diagram illustrating an example of an imaging system according to an embodiment of the present invention. FIG. 8 is an explanatory diagram showing the detailed configuration of the image processing unit according to the first embodiment of the present invention, and is a block diagram of the image processing unit shown in FIG. FIG. 9 is a block diagram conceptually showing a part of the internal configuration of the control unit 1 shown in FIG. FIG. 10 is an explanatory diagram for explaining the operation of the active information invalidation processing unit shown in FIG. 8. FIG. 10A shows the degraded image shown in FIG. FIG. 10B shows the one-dimensional profile image before performing the restoration process when not saturated, and FIG. 10B shows the deteriorated image shown in FIG. 1C after the restoration process when each pixel is not saturated for red and blue. The one-dimensional profile image is shown. FIG. 11 is an explanatory diagram for explaining the operation of the active information invalidation processing unit shown in FIG. 8. FIG. 11 (a) shows a part of the pixel saturated for green in the degraded image shown in FIG. 1C. FIG. 11B shows a primary image after the restoration process when a part of the pixel is saturated with respect to the green color of the deteriorated image shown in FIG. 1C. An original profile image is shown. FIG. 12 is an explanatory diagram for explaining the operation of the active information invalidation processing unit shown in FIG. 8. FIG. 12 (a) shows the degraded image shown in FIG. FIG. 12B shows a one-dimensional profile image obtained by discarding pixel data of a predetermined value or more on the saturation region side, and FIG. 12B shows a pixel having a predetermined value or more on the saturation region side shown in FIG. A one-dimensional profile image after performing restoration processing for red and blue using information obtained by truncating data is shown. FIG. 13 is a diagram for schematically explaining the concept of the restoration processing, and is a diagram for conceptually explaining the convolution of the deteriorated image and the inverse transform processing filter. FIG. 14 is an explanatory diagram showing a detailed configuration of the image processing unit according to the second embodiment of the present invention, and is a block diagram of the image processing unit shown in FIG.

  First, an image pickup apparatus capable of preventing generation of false colors as much as possible when a color image having blur according to the present invention is restored, an image pickup system including the image pickup apparatus, and a method for removing false colors are described. The relationship between a deteriorated image, its restoration process, and false color will be conceptually described.

  FIG. 1A is a schematic diagram of the imaging target 1. The imaging target 1 may be a person, a monitoring target object, a barcode, or a character string, but here it is a part of a barcode where the boundary 1C between the black area 1A and the white area 1B is clear.

When the imaging object 1 shown in FIG. 1A is imaged, if the image taken by the imaging device is clear, an image without deterioration shown in FIG. 1B is obtained. If the image captured by the imaging device is blurred, the blurred degraded image shown in FIG. 1C is obtained. FIG. 2 is a one-dimensional profile image obtained when the image without deterioration shown in FIG. 1B is scanned by the horizontal scanning line SC. FIG. 3 is a one-dimensional profile image obtained when the degraded image shown in FIG. 1C is scanned by a horizontal scanning line. 2 and 3, the vertical axis represents the luminance value (Y) as the pixel data, and the horizontal axis represents the horizontal number of the pixel X.
Note that the one-dimensional profile image is a curve representing a change in intensity of an image by one horizontal scanning line SC among a plurality of horizontal scanning lines SC constituting a two-dimensional image as a luminance value.

  The one-dimensional profile images shown in FIGS. 2 and 3 show changes in luminance values of the pixels on the horizontal scanning line SC shown in FIGS. 1B and 1C. Luminance value (Y) = 255 means a saturation value HO. Even if a light amount exceeding the saturation value is incident from each part of the imaging target 1, the light amount information is useless information that is not useful for use in the restoration process. It is. When each pixel is saturated, each pixel has pixel data with a saturation value “255”.

  When the image is clear, as shown in FIG. 2, the pixel region 1A ′ on the low luminance side corresponding to the black region 1A and the pixel region (pixel region on the saturation region side) 1B ′ on the high luminance side corresponding to the white region 1C. The slope of the boundary pixel area 1C ′ between the two becomes steep.

On the other hand, in the case of a degraded image, as shown in FIG. 3, a low-luminance pixel region 1A ′ corresponding to the black region 1A and a high-luminance pixel region (pixel region on the saturation region side) 1B ′ corresponding to the white region 1C. The low-luminance side and the saturation region side of the boundary pixel region 1C ′ are gentle curves 1C ″.
The gentle slope means that the portion of the boundary 1C is received across a plurality of pixels, that is, the boundary region 1C ′ is blurred.

  In the one-dimensional profile image shown in FIG. 3, the luminance value (Y) is not saturated. Therefore, pixel data can be used as useful information for all pixels. Therefore, in principle, when the restoration process is performed on the pixel data shown in FIG. 3 using the inverse function of the point spread function, the one-dimensional profile image shown in FIG. 2 can be restored. That is, by restoring the deteriorated image shown in FIG. 1C, a clear image without deterioration shown in FIG. 1B can be obtained.

  However, as shown in the one-dimensional profile image of FIG. 4, when the luminance value (Y) of the deteriorated image is saturated, the pixel data information (light quantity information) of the broken line portion that is originally obtained when it is not saturated. Is lost. That is, a pixel having the saturation value HO cannot be used as useful information when performing the restoration process.

  As a result, the one-dimensional profile image shown in FIG. 5 that should originally be obtained when the restoration process is performed cannot be restored. As shown in the one-dimensional profile image of FIG. The curve is a gentle curve 1C "on the low luminance side, and when the restoration process is performed, the boundary region 1C on the low luminance side is blurred.

  As described above, the degraded image restoration processing is described using the luminance value (Y) for the one-dimensional profile image of the image. However, the pixel data is saturated for each color of red (R), green (G), and blue (B). Then, similarly, an accurate restoration process cannot be performed.

  For example, among red (R), green (G), and blue (B), some pixel data of the green (G) pixel is saturated, and all of the remaining red (R) and blue (B) It is assumed that the pixel data is not saturated for the pixels.

  As described above, when the pixel data of some pixels has a saturation value HO for one of the three colors (for example, green), when the degraded image is restored, the green (G) pixel is As shown in FIG. 6, the boundary pixel region 1C ′ on the low luminance side becomes a gentle curve 1C ″.

  On the other hand, the red (R) and blue (B) pixels have a steep slope in the low luminance side pixel region 1C ′ as shown in FIG. When restoration processing is performed, there is a difference in restoration processing results between green (G), red (R), and blue (B), and a color image is generated using pixel data of each color subjected to the restoration processing. , False colors are generated in the QR region indicated by diagonal lines.

(Principle of the present invention)
Next, the principle of the present invention will be conceptually described.
In order to accurately restore a deteriorated image (blurred image), as described above, it is premised that all pixel data is useful information for performing the restoration process. However, if the pixel data is saturated for some pixels, the pixel data becomes useless information that is not useful for performing the restoration process. Therefore, if the same restoration process is performed for each color, a false color is generated.

  Therefore, in order to prevent the occurrence of false colors, the useful information on the saturation area side of the pixel data is made non-use information for each color before performing the image restoration process. For example, for each color, if the useful information on the saturation area side is aligned to a predetermined value and converted to non-use information, a part of pixels of a certain color (for example, green) is saturated and another color (for example, for example, Even when the pixel data of the red and blue pixels is not saturated, the accuracy of the restoration process can be reduced to the same extent as the degree of restoration for a color in which some of the pixels are saturated. As a result, generation of false colors can be prevented when color images are combined.

  A false color is generated when a part of a pixel area of a certain color pixel is saturated and a pixel of another color is not saturated. Therefore, luminance data and color difference signal data are created for each color, a pixel area having a saturation value is searched, and the color difference data signal of the pixel area having the saturation value is made useless information. That is, the color difference signal data is brought close to 0.

  As described above, when the color difference signal data of the pixel area having the saturation value is made useless information, the color difference signal data is brought close to 0 in the vicinity of the pixel area having the saturation value when the color image is synthesized. Thereby, when a color image is synthesized, the color in the vicinity of the boundary region 1C is thinned, and generation of a false color is prevented.

Example 1
FIG. 7 is a block diagram of an imaging system according to the present invention.
In FIG. 7, reference numeral 2 denotes an imaging device. The imaging device 2 includes a lens unit 3 as an imaging optical system, a color imaging device 4 such as a CCD or CMOS sensor, an image processing unit 5, and a data communication unit 6. As the image pickup element 4, a Bayer array type of red (R), green (G), and blue (B) is used.

  The lens unit 3 is provided with a diaphragm member 7 between the front lens system L1 and the rear lens system L2. A phase plate 8 is provided immediately before the diaphragm member 7. This phase plate 8 is used to generate aberrations for expanding the depth of field. That is, the imaging optical system includes the phase plate 8 as means for reducing the change in the depth of field with respect to the change in the distance to the imaging target 1.

  The phase plate 8 can be attached to and detached from the lens unit 3, and when the phase plate 8 is removed from the optical path of the lens unit 3, the imaging device is an ordinary camera that can capture a clear image with almost no blur. 2 can be used. When the phase plate 8 is inserted into the optical path of the imaging optical system, the phase plate 8 can be used for a camera that supports so-called EDoF (Extended Depth of Field) that extends the depth of field.

  The part BU of the imaging target 1 is received by the imaging element 4 as a blurred degraded image corresponding to the point spread function via the imaging optical system. The image sensor 4 outputs pixel data corresponding to the deteriorated image to the image processing unit 5.

  Pixel data corresponding to each color is input from the image sensor 4 to the image processing unit 5. As shown in FIG. 8, the image processing unit 5 includes a white balance processing unit 5A, a role information disabling processing unit 5B, a restoration processing unit 5C, a gain adjusting unit 5D, and a Bayer interpolation processing unit 5D. It has.

  The white balance processing unit 5A performs a process for adjusting the white balance by a known process. For example, the white balance adjustment is performed by multiplying the pixel data of each pixel by a gain value held in a memory or the like for each color of red (R), green (G), and blue (B).

  The effective information revocation processing unit 5B and the restoration processing unit 5C will be described later. The Bayer interpolation unit 5D generates color image data after performing Bayer interpolation processing such as linear interpolation, gradient based interpolation, and adaptive color plane interpolation on the insufficient color information.

  The color image data is output toward the data communication unit 6. The data communication unit 6 is connected to Ethernet (registered trademark) or the like, and image data is input to the personal computer 9.

  The personal computer 9 has an input unit 10 and a control unit 11. A monitor device 12 is connected to the control unit 11. The control unit 11 has a function of displaying a color image output from the data communication unit 6 on the screen of the monitor device 12 and a function of controlling the image processing unit 5 via the data communication unit 6.

The useful information invalidation processing unit 5B is a function for rendering useful information on the saturation area side of the image data for each color before performing the process of restoring the image in order to prevent generation of false colors. Have
In the first embodiment, pixel data for each color is obtained by replacing the luminance value Y as pixel data with the predetermined value TH for the pixel X having pixel data as active information that exceeds the predetermined value TH lower than the saturation value. The useful information on the saturation area side of the.

  The control unit 11 includes, for example, a comparator CP1 as shown in FIG. The comparator CP1 compares the luminance value Y of each pixel with a predetermined value TH, and when the predetermined value TH is exceeded, outputs the predetermined value TH toward the memory (storage unit) 13 and exceeds the predetermined value TH. The address information of the selected pixel is output to the memory 13. The effective information revocation processing unit 5B performs a revocation information conversion process based on the information stored in the memory 13, and details thereof will be described later.

  The restoration processing unit 5C generates, for each color obtained from the color image pickup device 4, a degraded image corresponding to the point spread function from the pixel data output from each pixel, and then the inverse function of the point spread function on the degraded image. Has a function of restoring an image from which blur is removed.

  For the restoration processing, known inverse filter processing and Wiener filter processing can be used. In this embodiment, Wiener filter processing is used. The winner filter processing is described in, for example, non-patent literature (“Computer Image Processing” author Hideyuki Tamura). The formula for the Wiener filter is as follows.

Here, R is an image processing filter, H is a transfer function (OTF) of the optical system, S ² is a power spectrum of the subject, W ² is a power spectrum of intrinsic noise of the image sensor 4, u and v are vertical and horizontal. The spatial frequency of the direction is shown.

The optical transfer function H (u, v) can be obtained from design data of the optical system. The image processing filters W (u, v) and S (u, v) can be accurately obtained by actual measurement. However, in practice, W ² / S ² may be set as an appropriate constant and handled.

  If the Wiener filter is incorporated in the imaging device 2 as it is and processing is performed in the frequency space, the load of restoration processing becomes excessive. Therefore, the frequency space is Fourier transformed to be converted into an FIR filter in the real space, and the FIR filter is subjected to a restoration process by convolution with a degraded image. The concept of an example of the arithmetic processing will be outlined.

FIG. 13 is an explanatory diagram for schematically explaining the convolution of a filter and a deteriorated image in inverse transformation processing.
Reference numeral 20 denotes a pixel area corresponding to a partial image of the deteriorated image used for the restoration process, and reference numeral 21 denotes an FIR filter used for the restoration process of the pixel area 20. For example, the pixel region 20 is a 5 × 5 pixel region of the deteriorated image shown in FIG. 1C. Pixel data of the pixels in the pixel region 20 are indicated by reference signs A11 to A15, A21 to A25, A31 to A35, A41 to A45, and A51 to A55.

  The FIR filter 21 has a size corresponding to the size of the pixel region 20, and reference numerals a11 to a15, a21 to a25, a31 to a35, a41 to a45, and a51 to a55 are filter coefficients used for convolution. Is shown.

  Convolution of each pixel in the pixel region 20 and the FIR filter 21 is performed based on the equation shown in FIG. The convolution calculation is performed on the first horizontal scanning line from the left to the right, for example, as indicated by a broken line in FIG. 1C, and then the first horizontal scanning line SC is calculated. Is completed, the convolution operation is executed for the horizontal scanning line SC of the second row, and this inverse conversion process is performed until the convolution operation of the horizontal scanning line SC of the last row is completed.

  An image from which blur is removed is acquired by these inverse transform processes, and details thereof are disclosed in, for example, Japanese Patent Application Laid-Open No. 2009-89082. Since the restoration process itself is known, the details thereof will be omitted, and pixel data used for this convolution process will be described next.

  Based on the information from the memory 13, the useful information revocation processing unit 5B replaces the pixel data with the predetermined value TH for the pixel having the pixel data as the useful information that exceeds the predetermined value TH lower than the saturation value. By doing this, the useful information of the pixel region on the saturation region side is converted to the useless information for each color.

  For example, it is assumed that pixel data of a red (R) image and pixel data of a blue (B) image are less likely to be saturated and green (R) pixel data is likely to be saturated. In this case, when restoration processing is performed on the red (R) and blue (B) degraded images shown in FIG. 10A, a one-dimensional profile with a steep boundary region 1C ′ is obtained as shown in FIG. 9B. can get.

  On the other hand, when restoration processing is performed on the green (G) degraded image shown in FIG. 11A, as shown in FIG. 11B, the low luminance side image region of the boundary region 1C has a gentle curve. 1C ″ is obtained. For green (G), the useful information is lost in the pixel region on the saturation region side, and the information is useless in the restoration process, so that the low luminance side of the boundary region 1C is gently For this reason, if a color image is synthesized using image data restored for each color without performing any processing, a false color is generated.

  On the other hand, before performing the restoration process of the deteriorated image, as shown in FIG. 12A, for the red (R) and blue (B) image data, a predetermined value TH as a threshold value lower than the saturation value. And the pixel data is replaced with the predetermined value TH for the pixel having the pixel data as the useful information exceeding the predetermined value TH.

  As described above, when red (R) and blue (B) pixel data is subjected to restoration processing by converting pixel data having pixels of a predetermined value TH or more into useless information, as shown in FIG. In addition, similarly to the green (B) image data, the low luminance side of the boundary region 1C ′ cannot be normally restored during the restoration process.

As a result, the one-dimensional profile images of the respective colors can be made substantially the same in the low luminance side boundary region 1C, and the generation of false colors can be reduced when color images are synthesized.
In the first embodiment, the effective information of the pixel data is truncated at the predetermined value TH, and the effective information is converted to non-use information. Therefore, red (R) and blue (B) on the high luminance side are A deviation occurs between the saturation value (predetermined value TH) and the green (G) saturation value HO. For this reason, when a color image is synthesized, a sense of incongruity occurs.

Therefore, in the first embodiment, the gain for aligning the saturation value for each color by multiplying the red pixel data and the green pixel data by a multiple of the ratio (HO / TH) of the saturation value HO to the predetermined value TH. Make adjustments.
The gain adjustment data is input from the control unit 11 via the data communication unit 6 to the gain adjustment unit 5D.

  Note that the gain adjustment unit 5D only has a function of performing the process of adjusting the saturation value HO for each color. Therefore, in the gain adjustment unit 5D, pixel data of a predetermined value TH or more for green (G) is rounded down to a predetermined value. The pixel data may be replaced with pixel data having a value TH.

(Modification 1)
The useful information revocation processing unit 5B includes the saturated pixels after saturation of the pixel data as the useful information by adding a predetermined value TH to the pixels having pixel data lower than the saturation value HO. By subtracting the predetermined value TH for each pixel, the useful information on the saturation area side of the pixel data may be converted to non-use information for each color.

(Modification 2)
Further, the useful information invalidation processing unit 5B saturates the pixel data as useful information by multiplying the pixel data of each pixel for each color by a predetermined value TH as a coefficient larger than “1”. After that, by dividing each pixel including the saturated pixel by the predetermined value TH, the useful information on the saturated area side of the pixel data for each color may be converted to non-use information.

  As described above, in the imaging system according to the first embodiment, the role information is disabled for each color by the input unit 10 that inputs a predetermined value to the imaging device 2 and the predetermined value TH input from the input unit 10. And a control unit 11 that controls the processing unit 5B to convert the useful information on the saturation region side to non-use information. In addition, the effective information invalidation processing unit 5B has a memory 13 for storing pixels for converting the effective information on the saturation region side into invalid information and for storing a predetermined value TH.

(False color removal method)
Next, the false color removal method by this imaging system will be described below.
First, a deteriorated image shown in FIG.
Next, a predetermined value TH is input to the control unit 11 by the input unit 10 (input step).
Next, the control unit 11 causes the effective information invalidation processing unit 5B to convert the effective information on the saturated area side into invalid information for each color (effective information invalidation step).

  The restoration processing unit 5C of the image pickup apparatus 2 uses the restoration area 5C of the imaging apparatus 2 to generate a deteriorated image for each color using the pixel data on the saturation region side that has been rendered unusable by this role information unusage step. A color image is created by restoration. Then, this color image is output to the monitor device 12 (output step).

Next, the presence or absence of a false color is confirmed by confirming the color image displayed on the monitor device 12 (confirmation step).
If a false color exists, the predetermined value TH input to the input unit 10 is changed, and if it is determined that there is no false color, the effective information on the saturation area side is converted to the role information in the memory 13. The pixel to be stored is stored and the predetermined value TH is stored (storage step).
Thereby, the false color is removed.

(Example 2)
In the second embodiment, the useful information is converted to non-use information after the degraded image is restored. In the second embodiment, as shown in FIG. 14, the image processing unit 5 includes a white balance processing unit 5A, a restoration processing unit 5C, a Bayer interpolation unit 5D, a YCbCr conversion unit 5E, and a CbCr suppression processing unit 5F. Have

The functions of the white balance processing unit 5A, the restoration processing unit 5C, and the Bayer interpolation unit 5D are the same as those in the first embodiment.
The YCbCr conversion unit 5E performs conversion processing from the RGB color space to the YCbCr color space. Thereby, luminance signal data Y ′ and color difference signal data Cb and Cr are generated. That is, the YCbCr conversion unit 5E functions as a generation unit that generates luminance signal data Y ′ and color difference signal data Cb and Cr for each pixel from pixel data for each color.

  The CbCr suppression processing unit 5F includes a search unit that searches for a saturated pixel having luminance signal data Y ′ that is saturated using the generated luminance signal data Y ′, and the vicinity including each pixel that corresponds to the saturated pixel The color difference signal data Cb and Cr obtained by the pixels existing in the area of this area has a function as an effective information invalidation processing unit for converting to unusable information.

  The neighboring pixel region may be, for example, a 5 × 5 pixel range. For this 5 × 5 pixel range, a gain of about 0.1 is added to the values of the color difference signal data Cb and Cr, or the color difference signal data Cb. By setting the value of Cr to 0, the color of the image constructed by the pixels existing in the neighboring region including each pixel corresponding to the saturated pixel is reduced, and processing is performed so that the luminance signal stands out.

  The luminance signal on the saturation region side is constructed by blue (B), red (R), and green (G), and color difference signal data is used to indicate the neighborhood region of the pixel corresponding to the saturation region constituting the luminance signal. The pixel corresponding to the pixel region when the pixel is decomposed into two pixels includes the pixel in the boundary region 1C that extends from the pixel region on the low luminance side to the pixel region on the high luminance side, so that the color of the region where the false color may occur can be removed. As a result, generation of false colors can be eliminated.

1 ... Imaging object 1
2 ... Imaging device 3 ... Lens unit (imaging optical system)
4 ... Image sensor 5 ... Image processing unit 5B ... Active information non-use processing unit 5C ... Restoration processing unit

JP 2009-89082 A JP 2011-170777 A

Claims (16)

An imaging optical system for imaging an imaging target;
An image sensor for color that receives the imaging object as a degraded image having blur via the imaging optical system;
A restoration processing unit that, for each color obtained from the image sensor, generates the degraded image from pixel data output from each pixel, and then performs image processing on the degraded image to restore an image from which blur has been removed; ,
In order to prevent the occurrence of false colors, it is provided with a role information invalidation processing unit that renders role information used for restoration processing on the saturated area side of pixel data as role information. Imaging device.   The said useful information revoking process part converts the useful information of the said saturation area side into useful information for every color, before performing the decompression | restoration process of the said degradation image. Imaging device.   The effective information disabling processing unit replaces the pixel data with the predetermined value for pixels having pixel data as effective information that exceeds a predetermined value lower than a saturation value, so that a pixel for each color is obtained. The imaging apparatus according to claim 2, wherein the useful information on the data saturation region side is converted to non-use information.   The useful information revocation processing unit saturates pixel data as useful information on the saturation region side by adding a predetermined value for pixels having pixel data lower than a saturation value, and then adds saturated pixels. The imaging apparatus according to claim 2, wherein, by subtracting the predetermined value for each pixel, the useful information on the saturation area side of the pixel data for each color is converted to non-use information.   The effective information invalidation processing unit saturates pixel data as effective information on the saturation region side by multiplying pixel data of each pixel by a predetermined value greater than 1 for each color, and then 3. The effective information on the saturation area side of the pixel data for each color is converted into non-use information by dividing each pixel data by a predetermined value for each pixel including the processed pixels. The imaging device described in 1.   The effective information revocation processing unit generates a luminance signal data and a color difference signal data for each pixel from the pixel data for each color after the restoration process of the deteriorated image, and the generated luminance signal data A search unit that searches for saturated pixels having luminance signal data that is saturated by using, and color difference signal data obtained by pixels existing in the vicinity including the pixels corresponding to the saturated pixels are made useless information The imaging apparatus according to claim 1, wherein:   A white balance processing unit is provided that performs white balance processing on pixel data from the image sensor before performing the role information conversion of the role information by the role information revocation processing unit. The imaging device according to any one of claims 1 to 6.   The said effective information revocation processing part makes the pixel data of the saturation area side useless information about red and blue pixel data, The any one of Claim 2 thru | or 5 characterized by the above-mentioned. Imaging device.   The imaging apparatus according to any one of claims 1 to 8, wherein the imaging optical system includes means for reducing a change in depth of field with respect to a change in distance to the imaging target. An input unit for inputting a predetermined value to the imaging device according to any one of claims 2 to 5 or the imaging device according to claim 8,
A control unit that performs control for converting the effective information on the saturation area side into the unusable information in the effective information unserving processing unit for each color according to a predetermined value input from the input unit;
The said effective information revocation processing part has a memory | storage part which memorize | stores the said predetermined value while memorize | storing the pixel which changes the useful information of the said saturation area side into useful information.   The effective information invalidation processing unit replaces pixel data as effective information exceeding a predetermined value lower than a saturation value with the predetermined value for the pixel stored in the storage unit. The imaging system according to claim 10.   The effective information invalidation processing unit saturates pixel data as effective information by adding the predetermined value to the pixels stored in the storage unit, and then includes each pixel including the saturated pixels. 11. The imaging system according to claim 10, wherein the effective information on the saturation region side of the pixel data is converted into the unusable information for each color by subtracting the predetermined value for.   The effective information revocation processing unit saturates the image data as the effective information by multiplying each pixel stored in the storage unit by a predetermined value greater than 1, and then extracts the saturated pixel. 11. The imaging according to claim 10, wherein the useful information on the saturation area side of the pixel data is made non-useful for each color by dividing the pixel data by the reciprocal of the predetermined value for each pixel. system.   The imaging system according to claim 10, further comprising a monitor device that displays a color image restored by the restoration processing unit. An input step of inputting the predetermined value according to claim 10 to the control unit according to claim 10 by the input unit according to claim 10;
A role information invalidation step in which the role information invalidation processing unit according to claim 10 is converted into role information by the control unit according to claim 10 for each color according to claim 10;
The restoration processing unit of the imaging device according to claim 10 uses the pixel data on the saturation region side that has been rendered unusable in the role information unusage step and the pixel data that has not been rendered unusable information for each color. An output step of creating a color image by restoring the deteriorated image and outputting the color image to the monitor device according to claim 13;
A confirmation step of confirming the presence or absence of a false color by confirming a color image displayed on the monitor device;
When the false color is present, the predetermined value input to the input unit is changed, and when it is determined that there is no false color, effective information on the saturation region side is stored in the storage unit according to claim 10. A false color removal method comprising: storing a pixel for converting the predetermined value into a useless information and storing the predetermined value.   The false color removal method according to claim 15, wherein white balance processing is performed on the pixel data input by the effective information invalidation processing unit.