JP2013084067A - Image processing system, method for controlling the same, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform luminance correction processing by reducing a positional deviation between a recording image and a reference image for deciding a luminance correction coefficient.SOLUTION: A recording image having a preset number of pixels is generated from an image obtained by performing reduction processing for a processing object image. Further, a luminance image having a number of pixels less than the preset number of pixels is generated by performing the reduction processing by extracting the luminance image of an area corresponding to the recording image from the processing object image. When the area corresponding to the recording image contains the under one pixel, the corresponding area is changed into the area which does not contain the under one pixel, to extract the luminance image. Enlargement processing is performed for the luminance image after the reduction processing so as to generate the reference image for deciding the luminance correction coefficient of the recording image. When the area for extracting the luminance image is changed, a gravity center of the area corresponding to each pixel of the luminance image after the reduction processing is changed to a position where the positional deviation of a subject image becomes minimum and generates the reference image based on information on the change and magnification of the corresponding area.

Description

  The present invention relates to an image processing apparatus, a control method thereof, and a program, and more particularly to a technique for correcting luminance of an image.

  In an imaging device such as a digital camera, development processing including various correction processing or filter processing is performed on an image signal output from an imaging device by shooting, and the corrected image signal is output as a recording image. . The correction processing and filter processing include image scaling processing, color correction processing, camera shake correction processing, luminance correction processing, and the like.

  In recent years, even in so-called backlight scenes where a high-luminance subject exists behind the main subject in imaging devices, the high-luminance subject and the low-luminance main subject are not blown out or blacked out. In addition, there is a camera that can take an image with a wide dynamic range. One method for generating an image with a wide dynamic range is to perform luminance correction (tone mapping) in accordance with the luminance component of the image, for example, by increasing the luminance level of a dark portion with low luminance. In particular, luminance correction processing for low-luminance regions is called (digital) dodging processing, and it is possible to record an image with a wide dynamic range by performing dodging processing on the main subject in a backlight scene. .

  In the dodging process, the low-frequency component of the luminance image composed of the luminance component of the recording image is extracted in order to perform luminance correction on the low-frequency and low-frequency component region of the recording image. The brightness correction coefficient is determined with reference to the low-pass image (reference image). In order to realize the generation of the reference image with low cost and low resources in the imaging device, the luminance image is reduced once to reduce the number of pixels, and then the low-pass filter is applied, and the reduced image after filtering is made large for recording. An enlargement method may be used. When the reference image is generated through the scaling process using such a fixed reduction ratio, the following problem may occur.

  Since the image for recording can be recorded with various pixel numbers and aspect ratios depending on the settings of the image pickup device, depending on the number of pixels of the image for recording, when it is reduced at a fixed reduction ratio, it is calculated and reduced. The number of lines may not be an integer value. Because of the structure of the image data, pixel value information is held for the unit pixel, information less than one pixel corresponding to a fractional line that does not become an integer value after reduction is lost during the reduction process. Since the reference image generated through the enlargement process on the reduced image obtained in this way is enlarged to the same size as the recording image, there is information lost during the reduction process. Therefore, the reference image and the recording image are not matched for each pixel. That is, when the dodging process is performed with the luminance correction coefficient determined with reference to such a reference image, a suitable correction result cannot be obtained.

  In Patent Document 1, when there is a fractional area that cannot be converted in the reduction process, the reduction process is performed as a pixel without excluding the area, and excess pixels that are larger than the recording image at the time of enlargement are excluded. It is disclosed that a reference image having the same size as a recording image is generated.

JP 2010-215192 A

  However, Patent Document 1 does not consider that scaling processing is applied to a recording image. In order to record an image with a desired number of pixels, a reduction process is applied to an image (original image) output from an image sensor, and then various filter processes are performed to generate an image for recording. The following problems arise.

  For example, consider a case where the original image output from the image sensor is composed of 800 pixels × 600 pixels as shown in FIG. The original image 500 includes an OB (Optical Black) region 501 that is referred to in image processing, and an effective pixel region 502 having a pixel value corresponding to the subject. When an image for recording is generated using the entire effective pixel area 502, the filter process applied after the reduction process of the original image 500 is smaller than the effective pixel area 502 after the reduction process depending on the number of taps of the filter. An image having the number of pixels is output. Therefore, in order to make the recording image have the desired number of pixels, the reduction ratio of the reduction process is determined in consideration of the number of filter taps. For example, when the number of filter taps is 20 in both the vertical and horizontal directions, that is, when calculating the pixel value of one pixel with reference to 20 pixels in the vertical or horizontal direction, an image of 180 pixels × 160 pixels is recorded. Is generated, the original image 500 is subjected to reduction processing with a reduction ratio of 3/10. As a result of the reduction processing, the original image 500 is converted into a 240 pixel × 180 pixel reduced image 510 as shown in FIG. 5B, and the image for recording is 180 pixel × 160 pixel by performing filter processing on the image. 511 is obtained. In the example of the figure, the position of the recording image 511 is defined as the coordinate of the upper left end of the recording image 511 being (50, 10) in the coordinate system with the upper left end of the reduced image 510 as the origin. Yes.

On the other hand, a reference image for determining a luminance correction coefficient for such a recording image 511 is generated from a luminance image in an area corresponding to the recording image 511 from the original image 500 in order to improve the correction effect. It is preferable. An area corresponding to the recording image 511 in the original image 500 is shown as an area 503 in FIG. At this time, the coordinates of the upper left corner of the region 503 in the coordinate system with the upper left corner of the original image 500 as the origin are calculated as the reduction ratio in the reduction processing is 3/10.
x = 50 ÷ (3/10) = 166.666...
y = 10 ÷ (3/10) = 33.333
It becomes a decimal value like As described above, due to the structure of the image data, the coordinates are converted into integer values by rounding off. When a reference image is generated using an image extracted from the region 503 in accordance with the coordinates after such conversion, the extracted image does not include exactly the same subject image as the recording image. That is, when the dodging process is performed with the luminance correction coefficient determined based on such a reference image, a suitable correction result cannot be obtained. In other words, it is necessary to correct a positional deviation between the reference image and the recording image and generate a reference image in which correspondence for each pixel is taken.

  The present invention has been made in view of the above-described problems, and an image processing apparatus that reduces a positional deviation between a recording image and a reference image for determining a luminance correction coefficient and performs a suitable luminance correction process, It is an object to provide a control method and a program.

In order to achieve the above object, an image processing apparatus of the present invention comprises the following arrangement.
An acquisition unit that acquires an image to be processed, and a recording having a preset number of pixels from an image obtained by performing a reduction process at a first reduction ratio on the image to be processed acquired by the acquisition unit Generating means for generating an image for use, extracting means for extracting a luminance image of an area corresponding to the recording image generated by the generating means from the processing target image acquired by the acquiring means, and extracting by the extracting means A reduction unit that performs a reduction process with a second reduction ratio different from the first reduction ratio on the luminance image of the corresponding area, and generates a luminance image having a smaller number of pixels than a preset number of pixels Then, the luminance image after the reduction process generated by the reduction unit is subjected to the enlargement process at a value obtained by dividing the first reduction ratio by the second reduction ratio, and has a preset number of pixels. , Brightness of the image for recording An enlarging unit that generates a reference image for determining a positive coefficient, and a correcting unit that performs luminance correction with a luminance correction coefficient determined from the luminance value of a pixel at the same coordinate of the reference image for each pixel of the image for recording. And when the corresponding area includes pixels less than one pixel, the extraction means changes the corresponding area to an area not including pixels less than one pixel, extracts the luminance image of the corresponding area, and the enlargement means When the corresponding region is changed by the extracting unit, the center of gravity of the region in the reference image corresponding to each pixel of the luminance image after the reduction process is based on the information on the change of the corresponding region by the extracting unit and the enlargement ratio. Thus, the reference image is generated by changing the position of the subject image between the reference image and the recording image so that the amount of positional deviation is minimized.

  With such a configuration, according to the present invention, it is possible to reduce the positional deviation between the recording image and the reference image for determining the luminance correction coefficient, and perform a suitable luminance correction process.

1 is a block diagram showing a functional configuration of an image processing unit 100 according to an embodiment of the present invention. The flowchart which showed the dodging gain production | generation process which concerns on embodiment of this invention The figure which showed the relationship between the input image data, the image data for recording, and the luminance image data for determining the luminance correction coefficient according to the embodiment of the present invention The figure for demonstrating the gravity center of the area | region in the image data after the expansion process corresponding to the pixel of the image data before the expansion process based on embodiment of this invention Illustration for explaining the problems of the prior art

[Embodiment]
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following embodiment, an example in which the present invention is applied to a digital camera including an image processing unit that performs dodging processing on a developed image will be described as an example of an image processing apparatus. However, the present invention can be applied to any device capable of performing luminance correction processing according to the luminance value on the input image.

<Configuration of Image Processing Unit 100>
FIG. 1 is a block diagram showing a functional configuration of an image processing unit 100 according to an embodiment of the present invention. The image processing unit 100 is controlled by a control unit (not shown) included in the digital camera of the present embodiment. In the image processing unit 100, under the control of the control unit, input image data picked up by an image pickup unit (not shown), information indicating the number of pixels of image data for recording, and various image processing related to development processing are performed. It is assumed that correction parameters are input.

  The signal correction unit 101 performs predetermined signal processing such as shading correction, blackening noise correction, and flicker correction on the input image data. Note that the input image data of this embodiment is color image data composed of three types of data R, G, and B (red, green, and blue) output from the Bayer array image sensor. Since the input image data is image data output from the Bayer array image sensor, pixel data of any color of R, G, and B is arranged in a regular rule for the unit pixel. For example, in the input image data, pixel data is arranged in the order of R, G, R, G,... On the odd lines in the horizontal direction, and the order of G, B, G, B,. Pixel data is lined up. The signal correction unit 101 outputs image data obtained by performing predetermined signal processing to a developed image reduction unit 102 and a window area calculation unit 105 described later.

  The developed image reduction unit 102 performs reduction processing on the input image data in accordance with the number of pixels of the recording image data, and outputs the obtained image data to the development processing unit 103. Specifically, the developed image reduction unit 102 cuts out an area (effective pixel area) necessary for the development process from the input image data by window processing, and applies a low-pass filter to the image data in the area. Image data is reduced by performing a thinning process. As a coefficient of the low-pass filter applied in the developed image reduction unit 102, an appropriate coefficient that is set so as not to cause aliasing distortion according to the reduction ratio, such as [1 2 1], is used. Note that the reduction ratio (first reduction ratio) in the developed image reduction unit 102 is determined according to the number of pixels of image data for recording input to the image processing unit 100.

  The development processing unit 103 performs known synchronization processing on the Bayer-format image data after the reduction processing in the development image reduction unit 102, and converts each pixel into image data having information on luminance and color difference signals. . The development processing unit 103 performs predetermined image processing such as white balance adjustment on the converted image data based on the correction parameters input to the image processing unit 100, and covers the corrected image data described later. The data is output to the baking processing unit 104.

  The luminance image extraction unit 105 generates luminance image data in which only the luminance component is extracted from the image data input by applying signal processing in the signal correction unit 101. Specifically, the luminance image extraction unit 105 performs conversion processing from color data of each color into luminance values for Bayer-format image data. The calculation formula used for conversion to the luminance value at this time may be a conversion formula for converting to an arbitrary luminance value such as a conversion formula from the RGB format to the YCbCr format or a conversion formula from the RGB format to the HSV format. Then, the luminance image extraction unit 105 outputs the obtained luminance image data to the window area extraction unit 106.

  The window area extraction unit 106 extracts image data included in an area corresponding to image data to be subjected to dodging processing in the dodging processing unit 104 described later, that is, image data included in the image data for recording, from the input luminance image data. Cut out by window processing. Note that the region extracted by the window processing is extracted in units of pixels of luminance image data. The area corresponding to the image data for recording in the luminance image data may include pixels less than one pixel depending on the first reduction ratio. In such a case, the window region extraction unit 106 changes the region to be extracted so as not to include pixels less than one pixel. The window area extraction unit 106 extracts the luminance image data of the area corresponding to the recording image data, and then outputs the extracted luminance image data to the first luminance image reduction unit 107 and the second luminance image reduction unit 111.

  The first luminance image reduction unit 107 performs a reduction process on the input luminance image data corresponding to the recording image data. In the present embodiment, the first luminance image reduction unit 107 performs a reduction process using a method for normalizing a bit shift operation. Specifically, for example, when the reduction ratio (second reduction ratio) is 1/10 in both the vertical direction and the horizontal direction, the first luminance image reduction unit 107 first converts the input luminance image data to 10 in each direction. A pixel value of a total of 100 pixels is acquired and integrated for each pixel. Further, the first luminance image reducing unit 107 performs a reduction process by multiplying the integral value by 10 as a preset gain coefficient and then shifting it to the right by 10 bits. Note that the reduction processing method in the first luminance image reduction unit 107 is not limited to this, and it can be easily imagined that any method may be used. The first luminance image reduction unit 107 outputs the luminance image data after the reduction process to the first LPF 108.

  The first LPF 108 is a low-pass filter that extracts a low-frequency component of the luminance image. The reduced luminance image data input to the first LPF 108 becomes luminance image data of only a low frequency component by the filtering process. The first LPF 108 outputs the filtered luminance image data to the first luminance image enlargement unit 109.

  The first luminance image enlargement unit 109 performs an enlargement process on the input luminance image data after the filter process, and generates luminance image data having the same number of pixels as the recording image data. Specifically, the first luminance image enlargement unit 109 performs the enlargement process of the luminance image data after the filter processing, using a value obtained by dividing the first reduction rate by the second reduction rate as an enlargement rate. Then, the first luminance image enlargement unit 109 outputs the obtained luminance image data after the filter processing to the first luminance correction coefficient determination unit 110.

  In the enlargement process, each pixel of the luminance image data after the filter process is not simply enlarged, but the pixel value of the pixel increased by the enlargement process is calculated by an interpolation process, an extrapolation process, or the like. Each pixel of the luminance image data to be enlarged is generated by integration of pixel values of an area composed of a plurality of pixels by the reduction process in the first luminance image reduction unit 107. For this reason, the pixel value of each pixel of the luminance image data to be enlarged corresponds to the pixel value of the pixel at the center of gravity of the area of the luminance image data after the enlargement process corresponding to the area referred to when the pixel is generated. . That is, in the luminance image data after the enlargement process, the pixel value of a pixel (pixel between the centers of gravity) sandwiched between pixels corresponding to the pixel of the luminance image data to be enlarged (hereinafter simply referred to as “corresponding pixel”) is: It is generated by interpolation processing using the pixel value of the corresponding pixel around the pixel and its distance. In addition, pixel values are calculated by extrapolation processing for pixels that are not sandwiched between corresponding pixels, that is, pixels in the outer peripheral portion of the luminance image data after enlargement processing.

  The luminance image data after the enlargement process is referred to by the first luminance correction coefficient determination unit 110 in order to determine a luminance correction coefficient used in the dodging process performed by the dodging processing unit 104 described later. That is, the enlarged luminance image data generated by the first luminance image enlargement unit 109 is reference image data for determining a luminance correction coefficient. The image data for recording and the reference image data have the same number of pixels, and the pixels at the same coordinates of each other's image data are associated one-to-one. That is, the luminance correction coefficient determined for one pixel of the reference image data is a correction coefficient of a pixel having the same coordinate as the pixel in the recording image data. For this reason, in the enlargement process in the first luminance image enlargement unit 109, the interpolation process and the extrapolation process are performed so that the positional deviation amount of the subject image is minimized between the recording image data and the reference image data.

  The first luminance correction coefficient determination unit 110 determines a luminance correction coefficient for each pixel of the input reference image data from the luminance value of the pixel. The luminance correction coefficient is a value determined for each luminance value using, for example, a gamma function. In the present embodiment, the luminance value is used as a lookup table in a non-illustrated non-volatile memory provided in the image processing unit 100. It is assumed that the brightness correction coefficient for each is stored. The first luminance correction coefficient determination unit 110 reads the lookup table from the nonvolatile memory, determines the luminance correction coefficient for each pixel of the reference image data, and then associates the luminance correction coefficient of the pixel with the coordinate information of the pixel. The brightness correction data is output to the dodging processor 104.

  Note that the image processing unit 100 according to the present embodiment uses the first luminance image reduction unit 107 to the first luminance correction coefficient determination unit 110 and the second luminance image reduction as lines for determining the luminance correction coefficient of image data for recording. Two lines of the unit 111 to the second luminance correction coefficient determination unit 114 are provided. However, the implementation of the present invention is not limited to this. The processing performed by the blocks of the second luminance image reduction unit 111 to the second luminance correction coefficient determination unit 114 is basically the same as the processing of the first luminance image reduction unit 107 to the first luminance correction coefficient determination unit 110. Therefore, description of processing performed in each block is omitted. Both lines differ only in the reduction rate of the reduction process and the enlargement rate of the enlargement process performed by the second luminance image reduction unit 111 and the second luminance image enlargement unit 113. In the present embodiment, the first luminance image reduction unit 107 performs a reduction process with a reduction ratio of 1/10 in both the vertical direction and the horizontal direction, whereas the second luminance image reduction unit 111 has 1 in both the vertical direction and the horizontal direction. It is assumed that a reduction process with a reduction rate of / 100 is performed. Note that the enlargement rate of the enlargement process in the second luminance image enlargement unit 113 is the same as the first luminance image enlargement unit 109 in the number of pixels of the output luminance image data, and therefore the enlargement rate varies depending on the reduction rate. Easy to understand.

  In the present embodiment, a configuration for outputting brightness correction data used in the dodging processing unit 104 from image data signal-processed by the signal correction unit 101 is referred to as a dodging gain generation unit 120 and will be described below. And The dodging gain generation unit 120 includes a luminance image extraction unit 105 to a second luminance correction coefficient determination unit 114.

  Based on the luminance correction data input from the first luminance correction coefficient determination unit 110 and the second luminance correction coefficient determination unit 114, the dodging processing unit 104 performs luminance correction processing (for pixels corresponding to recording image data). Dodging process). Specifically, the dodging processor 104 performs a luminance correction process using each of the two types of luminance correction data input from the first luminance correction coefficient determination unit 110 and the second luminance correction coefficient determination unit 114. The dodging processing unit 104 finally generates image data (output image data) to be recorded by subjecting the image data after luminance correction obtained as a result of the respective luminance correction processing to addition averaging processing.

  In the present embodiment, description will be made assuming that processing is realized in each block included in the image processing unit 100 as hardware. However, the embodiment of the present invention is not limited to this, and processing of each block is the same as that of each block. It may be realized by a program that performs processing.

<Dodge gain generation processing>
A specific process of the dodging gain generation process in the dodging gain generation unit 120 of the image processing unit 100 of the present embodiment having such a configuration will be described with reference to the flowchart of FIG. In the processing corresponding to the flowchart, a control unit (not shown) included in the digital camera reads the corresponding processing program stored in the ROM, for example, and controls the image processing unit 100 by developing it in the RAM and executing it. Can be realized. Note that the dodging gain generation process will be described as being started when the signal processing in the signal correction unit 101 for the input image data input to the image processing unit 100 is completed, for example.

  In the following description, as in FIG. 5, the image size of the input image data is 800 pixels × 600 pixels, the first reduction ratio is 3/10 in both the horizontal and vertical directions, and the image size of the image data for recording is 240 pixels × A case where the pixel is 180 pixels will be described as an example. For the sake of simplicity in the following description, a method for generating correction coefficient data using only the lines of the first luminance image reduction unit 107 to the first luminance correction coefficient determination unit 110 will be described. Note that the second reduction rate of the reduction processing in the first luminance image reduction unit 107 is 1/10 in both the horizontal direction and the vertical direction as described above.

  In the following description, the window processing and position change processing will be described only for the calculation method of the coordinates of the upper left corner of the region to be processed or the image data for the sake of simplicity. It will be understood that processing such as coordinate calculation is performed in the same manner.

  In step S <b> 201, the luminance image extraction unit 105 generates luminance image data from the input image data after signal processing in the input signal correction unit 101. Then, the luminance image extraction unit 105 outputs the generated luminance image data to the window area extraction unit 106.

In step S <b> 202, the window area extraction unit 106 records an image for recording in the coordinate system (x ′, y ′) having the origin at the upper left corner of the image data (development reduced image data) after the reduction processing in the development image reduction unit 102. The coordinates of the upper left corner of the area corresponding to the data are calculated. Specifically, the window area extraction unit 106 uses the first reduction rate of the reduction processing performed by the development image reduction unit 102 as information on the number of pixels of the recording image data input to the image processing unit 100 and the development processing. It is calculated from information on the number of taps of image processing applied in the unit 103. Then, the window area extraction unit 106 grasps the image size of the development reduced image data from the image size of the input image data and the first reduction rate, and records it from the number of image processing taps applied in the development processing unit 103. The coordinates of the upper left corner of the area corresponding to the image data for use are calculated. At this time, “the coordinates of the upper left corner of the area corresponding to the image data for recording in the development reduced image data” is (x ₀ ′, y ₀ ′) = (50, 10) as shown in FIG.
Suppose that

In S203, the window area extraction unit 106 calculates the coordinates of the upper left corner of the area corresponding to the image data for recording in the coordinate system (x, y) with the upper left corner of the luminance image data generated in S201 as the origin. To do. Specifically, the window area extraction unit 106 divides the “coordinate of the upper left corner of the area corresponding to the image data for recording in the development reduced image data” calculated in S202 by the first reduction ratio, thereby obtaining “brightness The coordinates of the upper left corner of the area corresponding to the image data for recording in the image data are calculated. At this time, "the coordinates of the upper left corner of the area corresponding to the image data for recording in the luminance image data" is
x ₀ = 50 ÷ (3/10) = 166.666...
y ₀ = 10 ÷ (3/10) = 33.333
It becomes.

However, as illustrated, depending on the first reduction ratio, “the coordinates of the upper left corner of the area corresponding to the image data for recording in the luminance image data” may include pixels less than one pixel that cannot be processed as image data. . For this reason, the window area extraction unit 106 converts the coordinates to an integer value by rounding off the decimal point when “the coordinates of the upper left corner of the area corresponding to the image data for recording in the luminance image data” is not an integer. To do. That is, in the above example, the coordinates are
( _X0changed , _y0changed ) = (167, 33)
Is converted to

  In S204, the window area extraction unit 106 determines the luminance of the area corresponding to the recording image data according to the information of “the upper left coordinates of the area corresponding to the recording image data in the luminance image data” obtained in S203. Extract image data. Then, the window area extraction unit 106 outputs the extracted luminance image data to the first luminance image reduction unit 107.

  In step S <b> 205, the first luminance image reduction unit 107 performs a reduction process with the second reduction ratio on the input luminance image data. Then, the first luminance image reduction unit 107 outputs the obtained luminance image data after the reduction process to the first LPF 108. Further, the first LPF 108 performs a filtering process that allows only the low frequency region component to pass through the input luminance image data after the reduction process, and sends the obtained luminance image data after the filtering process to the first luminance image enlargement unit 109. Output.

In step S <b> 206, the first luminance image enlargement unit 109 performs an enlargement process on the input luminance image data after the filtering process with an enlargement ratio r that is a value obtained by dividing the first reduction ratio by the second reduction ratio. . In the example of the present embodiment, the enlargement ratio r is r = (3/10) ÷ (1/10) = 3 times.

(Details of enlargement processing)
Here, the details of the enlargement process performed by the first luminance image enlargement unit 109 will be described. As described above, the reference image data output by performing the enlargement process in the first luminance image enlargement unit 109 has the same number of pixels as the image data for recording. Further, when the reference image data and the recording image data are overlapped so that pixels having the same coordinates overlap, the reference image data is generated so that the amount of positional deviation of the subject image between the images is minimized.

The first luminance image enlarging unit 109 first determines whether or not the region corresponding to the recording image data has been changed when the luminance image data is extracted by the window region extracting unit 106. When the area corresponding to the image data for recording has been changed, the first luminance image enlarging unit 109 performs the actual recording in the coordinate of the upper left corner of the area after the change and the coordinate system of the development reduced image data. A positional deviation amount Δf = (Δf _x , Δf _y ) with respect to the coordinates of the upper left corner of the image data area for use is calculated. In the case of the above example, since the coordinate of the upper left corner of the area after the change is 167, when converted into the coordinate system of the development reduced image data using the first reduction ratio, the coordinate is x _0changed '= 167 × (3 /10)=50.1
y _0changed '= 33 × (3/10) = 9.9
It becomes. In other words, the luminance image data extracted by the window area extraction unit 106 has the image data for recording and the coordinates of the upper left corner at that time: Δf _x = x _{0 changed} '−x ₀ ' = 50.1−50 = + 0.1
Δf _y = y _{0 changed} '−y ₀ ' = 9.9−10 = −0.1
That is, the positional deviation is caused.

Further, the first luminance image enlargement unit 109 calculates the barycentric position of the region corresponding to each pixel of the luminance image data after the filtering process in the reference image data to be generated. Specifically, the first luminance image enlargement unit 109 firstly, in the luminance image data after the enlargement process obtained by simply performing the enlargement process with the enlargement ratio r, the area corresponding to each pixel of the luminance image data after the filter process The amount of displacement of the center of gravity position of the region from the upper left end of the region is acquired. FIG. 4 shows the upper left end portion of the luminance image data after the enlargement process obtained when the enlargement process with the enlargement ratio r is simply performed. As shown in the figure, the center of gravity 401 exists in the center of the region 400 corresponding to the upper left pixel of the luminance image data after the filtering process. At this time, the positional deviation amount Δg = (Δg _x , Δg _y ) of the center of gravity 401 from the upper left end of the region 400 is expressed by Δg _x = r × (1/2) = 3 using an enlargement ratio (r = 3 times). × (1/2) = 1.5
Δg _y = r × (1/2) = 3 × (1/2) = 1.5
It can be expressed. That is, when the enlargement process of the enlargement ratio r is simply performed, the center of gravity of the area corresponding to each pixel of the luminance image data after the filter process, which is a reference for the interpolation process or the extrapolation process, is calculated from the upper left end of each area. It is in a position separated by the gravity center position deviation amount Δg.

  However, as described above, due to the structure of the image data, the movement needs to be performed in units of pixels. Therefore, the sum of the calculated center-of-gravity position shift amount Δg and the position shift amount Δf generated when extracting the luminance image data (movement reference value Δf + Δg: actual position shift amount of the center-of-gravity with respect to image data for recording) is 1. When a value less than a pixel is included, it is necessary to round the calculated movement reference value to an integer value. At this time, the first luminance image enlarging unit 109 overlaps the reference image data obtained as a result of the enlarging process including the interpolation process and the extrapolation process, and the image data for recording so that the pixels having the same coordinates coincide. The movement reference value is rounded so that the positional deviation amount of the subject image of both image data is minimized. Specifically, the first luminance image enlarging unit 109 compares the amount of change from the movement reference value when the value after the decimal point of the movement reference value is rounded up and when the value is rounded down, and the amount of change is minimized. The movement reference value is rounded by the method.

Specifically, the first luminance image enlarging unit 109 takes absolute values for the increment (A) of the movement reference value when rounded up and the decrease (B) of the movement reference value when rounded down. The first luminance image enlarging unit 109 rounds the movement reference value by rounding up when the absolute value of the result is small (close to 0) is A, and rounds down the movement reference value when the absolute value is B. The movement reference value is changed to an integer value by rounding the value. In the above numerical example, the absolute value of the increment of the movement reference value when rounded up and the decrease of the movement reference value when rounded down are respectively
X coordinate A _x = ABS (ROUNDUP (Δf _x + Δg _x ) − (Δf _x + Δg _x ))
= ABS (ROUNDUP (0.1 + 1.5)-(0.1 + 1.5))
= ABS (2-1.6) = 0.4
B _x = ABS (ROUNDDOWN (Δf _x + Δg _x ) − (Δf _x + Δg _x ))
= ABS (ROUNDDOWN (0.1 + 1.5)-(0.1 + 1.5))
= ABS (1-1.6) = 0.6
Y coordinate A _y = ABS (ROUNDUP (Δf _y + Δg _y ) − (Δf _y + Δg _y ))
= ABS (ROUNDUP (-0.1 + 1.5)-(-0.1 + 1.5))
= ABS (2-1.4) = 0.6
B _y = ABS (ROUNDDOWN (Δf _x + Δg _y ) − (Δf _x + Δg _y ))
= ABS (ROUNDDOWN (-0.1 + 1.5)-(-0.1 + 1.5))
= ABS (1-1.4) = 0.4
(However, ABS is a function that outputs an absolute value, ROUNDUP is a round-up value, and ROUNDDOWN is a function that outputs a round-down value.)
Since the x component is A <B, the movement reference value is changed to an integer value by rounding up, and since the y component is B <A, the movement reference value is changed to an integer value by rounding down. For this reason, among the reference image data generated by the enlargement process, the final position deviation of the center of gravity of the area corresponding to the pixel at the upper left end of the filtered luminance image data from the upper left end of the image data for recording The quantity ( _{Δg xfinal} , _{Δg yfinal} ) is
_{Δg xfinal} = ROUNDUP (Δf _x + Δg _x ) = 2
_{Δg yfinal} = ROUNDDOWN (Δf _x + Δg _y ) = 1
It becomes.

  That is, the first luminance image enlargement unit 109 has the center of gravity of the region corresponding to the upper left pixel of the luminance image data after the filtering process at a position shifted by the final positional deviation amount from the upper left end of the reference image data to be generated. Adjust to move. That is, the position of the center of gravity of the region corresponding to the pixels other than the upper left corner of the luminance image data after the filter processing is changed according to the center of gravity of the region corresponding to the upper left upper pixel.

  The first luminance image enlarging unit 109 determines the position in the reference image data of the center of gravity of the region corresponding to each pixel of the luminance image data after the filter processing in this way, and then performs interpolation processing and extrapolation processing, Reference image data having the same number of pixels as the image data for recording is generated. Then, the first luminance image enlargement unit 109 outputs the generated reference image data to the first luminance correction coefficient determination unit 110.

  In S207, the first luminance correction coefficient determination unit 110 determines a luminance correction coefficient based on the luminance value of each pixel of the input reference image data, and outputs the luminance correction data in association with the pixel coordinate information. Specifically, the first luminance correction coefficient determination unit 110 reads a look-up table for determining the luminance correction coefficient from a non-illustrated nonvolatile memory included in the image processing unit 100, and corrects the threshold according to the luminance value of the target pixel. Get coefficient information. Then, the first luminance correction coefficient determination unit 110 associates the luminance correction coefficient of the pixel with the coordinate information of the target pixel, and outputs the luminance correction data to the dodging processing unit 104 to complete the dodging gain generation process.

  In this way, when generating the reference image data for determining the luminance correction coefficient, the subject between the recording image data and the reference image data is taken into account, taking into account the positional deviation that occurred during the extraction of the luminance image. The positional deviation of the image can be minimized.

  As described above, the image processing apparatus according to the present embodiment can reduce the positional deviation between the recording image and the reference image for determining the luminance correction coefficient, and perform a suitable luminance correction process. Specifically, the image processing apparatus generates a recording image having a preset number of pixels from an image obtained by performing reduction processing on the acquired processing target image. In addition, a luminance image in an area corresponding to the image to be recorded is extracted from the processing target image and subjected to a reduction process to generate a luminance image having a smaller number of pixels than a preset number of pixels. If the area corresponding to the recording image includes pixels less than one pixel, the corresponding area is changed to an area not including pixels less than one pixel, and a luminance image is extracted. Furthermore, the image processing apparatus performs an enlargement process on the luminance image after the reduction process, and generates a reference image for determining the luminance correction coefficient of the image for recording having a preset number of pixels. At this time, when the area for extracting the luminance image is changed, the center of gravity of the area corresponding to each pixel of the luminance image after the reduction process is referred to based on the information on the change of the corresponding area and the enlargement ratio. The reference image is generated by changing the position of the subject image between the image and the recording image to a position where the amount of positional deviation is minimized.

[Modification]
In the above-described embodiment, since image processing is performed for each unit pixel, it has been described that the region for extracting the luminance image is changed in units of pixels. In this modification, processing performed in each block of the image processing unit 100 when a circuit related to image processing simultaneously processes a plurality (a predetermined number) of pixels will be described.

  In this modification, the signal correction unit 101, the developed image reduction unit 102, the luminance image extraction unit 105, and the window area extraction unit 106 of the image processing unit 100 are configured to process four pixels simultaneously. For the other blocks, it is assumed that four pixels are not simultaneously processed and one pixel is processed at a time. A circuit that outputs pixels one by one after processing a plurality of pixels in a block that can be processed simultaneously may use a known technique. For example, the luminance image extraction unit 105 may reduce the number of output pixels per cycle by performing weighted addition on a plurality of input pixels.

In such a configuration, the processing performed by the window region extraction unit 106 when extracting the region corresponding to the recording image data from the luminance image data is different from that of the above-described embodiment. That is, the setting of the input window in the window processing needs to be a unit of 4 pixels, that is, the number of pixels in the vertical and horizontal directions is a multiple of 4. Therefore, as in the above-described embodiment, the coordinates of the upper left corner of the area corresponding to the image data for recording is x ₀ = 50 ÷ (3/10) = 166.666.
y ₀ = 10 ÷ (3/10) = 33.333
In the case of, the corresponding region is changed so that the upper left coordinate after the change is a multiple of four. In this case, the coordinates of the upper left corner after the change are (x _0changed , y _0changed ) = (168, 32)
It becomes.

  In this modification, when extracting the region corresponding to the recording image data from the luminance image data in this way, the region is changed by a method different from the above-described embodiment. However, since the positional deviation information generated by the change is added and corrected in the first luminance image enlargement unit 109, the recording image data, the reference image data, and the like, as in the above-described embodiment. The positional deviation of the subject image during the interval can be minimized.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (7)

An acquisition means for acquiring an image to be processed;
Generating means for generating a recording image having a preset number of pixels from an image obtained by performing reduction processing at a first reduction ratio on the processing target image acquired by the acquiring means; ,
Extracting means for extracting a luminance image of an area corresponding to the recording image generated by the generating means from the processing target image acquired by the acquiring means;
The luminance image of the corresponding area extracted by the extraction unit is subjected to a reduction process with a second reduction ratio different from the first reduction ratio, and the number of pixels smaller than the preset number of pixels is set. Reduction means for generating a luminance image having;
The luminance image after the reduction processing generated by the reduction means is subjected to enlargement processing at an enlargement rate obtained by dividing the first reduction rate by the second reduction rate, and the preset pixel A magnifying means for generating a reference image for determining a luminance correction coefficient of the image for recording having a number;
Correction means for performing luminance correction on each pixel of the image for recording with a luminance correction coefficient determined from the luminance value of the pixel at the same coordinate of the reference image;
When the corresponding area includes pixels less than one pixel, the extraction unit changes the corresponding area to an area not including pixels less than one pixel, and extracts a luminance image of the corresponding area.
The enlargement unit, when the corresponding region is changed by the extraction unit, displays the center of gravity of the region in the reference image corresponding to each pixel of the luminance image after the reduction process, by the corresponding extraction unit. Based on the area change information and the enlargement ratio, the reference image is generated by changing to a position where the amount of positional deviation of the subject image between the reference image and the recording image is minimized. Image processing device. When the generation unit performs a process of generating the recording image in a predetermined number of pixels,
When the extraction unit determines that processing is performed on pixels that are less than the predetermined number when the corresponding region is processed in units of the predetermined number of pixels, the extraction unit satisfies the predetermined number of the corresponding regions. The image processing apparatus according to claim 1, wherein a luminance image of the corresponding area is extracted by changing to an area having a number of pixels in which processing is not performed for a non-existing pixel.   The enlargement means, when the corresponding area is changed by the extraction means, in the image obtained by performing the reduction process of the first reduction ratio and the corresponding area changed Movement standard obtained by adding the positional deviation amount between the center of gravity of the region in the reference image corresponding to each pixel of the luminance image after the reduction processing and the upper left end of the region to the positional deviation amount from the image for recording A value obtained by subtracting the movement reference value from a value obtained by rounding up the decimal point of the movement reference value, and a value obtained by subtracting the movement reference value from a value obtained by rounding down the decimal point of the movement reference value. The center of gravity of a region in the reference image corresponding to each pixel of the luminance image after the reduction processing is changed to a position moved by a value obtained by adding the movement reference value to a value close to 0. Item 1 or 2 The image processing apparatus.   The enlargement means uses the pixel value of each pixel of the luminance image after the reduction processing as a pixel value of the center of gravity of the region in the reference image corresponding to the pixel, and the pixels between the centers of gravity of the reference image 4. The image processing according to claim 1, wherein the pixel value is calculated by interpolation processing from a pixel value of the center of gravity, and the pixel values of other pixels of the reference image are calculated by extrapolation processing. 5. apparatus.   The change in the position of the center of gravity of the region in the reference image corresponding to each pixel of the luminance image after the reduction processing by the enlargement processing is performed in the vertical direction and the horizontal direction. The image processing apparatus according to item 1. An acquisition step in which an acquisition unit of the image processing apparatus acquires an image to be processed;
A recording having a preset number of pixels from an image obtained by performing generation processing of the first reduction ratio on the processing target image acquired in the acquisition step by the generation unit of the image processing apparatus. A generation process for generating an image for
An extraction step in which a generation unit of the image processing apparatus extracts a luminance image of an area corresponding to the recording image generated in the generation step from the processing target image acquired in the acquisition step;
The generation unit of the image processing device performs a reduction process of a second reduction rate different from the first reduction rate on the luminance image of the corresponding region extracted in the extraction step, and the preset A reduction process for generating a luminance image having a smaller number of pixels than the number of pixels,
The generation unit of the image processing apparatus performs an enlargement process on the luminance image after the reduction process generated in the reduction process with an enlargement ratio of a value obtained by dividing the first reduction ratio by the second reduction ratio. Performing an enlargement step of generating a reference image for determining a luminance correction coefficient of the image for recording having the preset number of pixels;
A correction step in which the generation unit of the image processing device performs luminance correction on each pixel of the image for recording with a luminance correction coefficient determined from a luminance value of a pixel at the same coordinate of the reference image,
In the extraction step, when the corresponding area includes pixels less than one pixel, the extraction unit changes the corresponding area to an area not including pixels less than one pixel, and extracts a luminance image of the corresponding area And
In the enlargement step, the enlargement unit extracts the center of gravity of the region in the reference image corresponding to each pixel of the luminance image after the reduction process when the corresponding region is changed in the extraction step. Based on the information on the change of the corresponding area in the process and the enlargement ratio, the reference image is generated by changing the position of the subject image between the reference image and the recording image to a position where the amount of positional deviation is minimized. And a control method for the image processing apparatus.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 5.

Images