JP2011009937A - Image processing apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that brightness and white balance are largely different when aligning and displaying the images of crop regions which are adjacently positioned in the same image and are different in photographing conditions in a conventional manner.SOLUTION: An image processing apparatus includes an imaging unit for imaging a subject image, a crop region designation unit for designating a plurality of crop regions in the subject image to be imaged by the imaging unit, a distance calculation unit for obtaining a distance between a plurality of crop regions designated by the crop region designation unit, a correction coefficient calculation unit for obtaining image correction coefficients by which the correction is made to obtain the optimum images with respect to the respective whole images in units of crop regions, and a recording unit for associating image data of the crop regions designated by the crop region designation unit, distance information between the crop regions calculated by the distance calculation unit, and the image correction coefficients obtained by the correction coefficient calculation unit and storing the associated data, information, and coefficients in a storage medium.

Description

  The present invention relates to an image processing apparatus and an image processing program.

  2. Description of the Related Art In recent years, a technique for partially cropping (cutting out) an image capturing range and storing it in an image capturing apparatus such as an electronic camera with an increasing number of pixels. The method of partially cropping and storing the shooting range not only saves the memory capacity, but can be developed for various uses. For example, functions such as performing image processing on each of a plurality of crop areas and storing them, and moving one or a plurality of crop areas in accordance with the movement of a target subject have been studied (for example, Patent Document 1).

JP 2005-175683 A

  In general, automatic exposure adjustment and auto white balance adjustment are performed based on image data (RGB data) for the entire captured image. As a result, although the brightness and white balance for the entire captured image are optimally adjusted, there is a problem that the brightness and white balance for the crop area obtained by cutting out a part of the captured image is not necessarily optimally adjusted. . For example, the exposure adjustment is performed so that the entire image is not too bright for a photographed image with many bright parts (such as the sun), but if the dark part (such as the shade) is extracted as a crop area, it is extracted. The crop area will be a dark image with a slight bluish tint. In order to prevent such a problem, a method of performing an adjustment suitable for each crop area has been considered. For example, when a plurality of crop areas are close to each other at the boundary between the sun and the shade, there is a slight difference in position. The brightness and white balance adjustments of the two crop areas are greatly different. In particular, when the images adjusted for each crop area are viewed side by side, there is a problem that it is unnatural that the brightness and white balance differ greatly even though the images are almost in the same position.

  An object of the present invention is to provide an image processing apparatus and an image processing program capable of performing optimal image processing on an image in a crop area cut out from the same image.

  An image processing apparatus according to the present invention includes: an imaging unit that captures a subject image; a crop area designation unit that designates a plurality of crop areas in the subject image captured by the imaging unit; and the crop area designation unit that is designated by the crop area designation unit A distance calculation unit for obtaining a distance between a plurality of crop areas, a correction coefficient calculation unit for obtaining an image correction coefficient for correcting each of the plurality of crop areas and an entire image, and the crop area designation The image data of the plurality of crop areas designated by the section, the distance information between the plurality of crop areas calculated by the distance calculation section, and the image correction coefficient obtained by the correction coefficient calculation section are associated with each other in a storage medium. And a recording unit for storing.

  The image processing apparatus according to the present invention corrects image data of a plurality of crop areas cut out from the same image, distance information between the plurality of crop areas, and an optimum image for each of the plurality of crop areas. An input unit for inputting an image correction coefficient to be performed; a distance correction coefficient calculation unit for obtaining a distance correction coefficient for adjusting the image correction coefficient for each of the plurality of crop areas based on distance information between the plurality of crop areas; and A correction processing unit that corrects image data for each crop region using an image correction coefficient and the distance correction coefficient; and a recording unit that stores the image data for each crop region processed by the correction processing unit in a storage medium; Is provided.

  The image processing apparatus according to the present invention includes: an imaging unit that captures a subject image; a crop area designation unit that designates a plurality of crop areas in the subject image captured by the imaging unit; and the crop area designation unit A distance calculation unit for obtaining a distance between the plurality of crop areas, a correction coefficient calculation unit for obtaining an image correction coefficient for correcting each of the plurality of crop areas and an entire image, and the plurality A distance correction coefficient calculation unit for obtaining a distance correction coefficient for adjusting an image correction coefficient for each crop area based on distance information between the plurality of crop areas, and the crop correction using the image correction coefficient and the distance correction coefficient. A correction processing unit that corrects image data for each region and a recording unit that stores the image data for each crop region processed by the correction processing unit in a storage medium are provided. Characterized in that was.

  More preferably, the distance correction coefficient calculation unit obtains the distance correction coefficient based on an image correction coefficient of the entire image or any one of the plurality of crop areas.

  More preferably, the distance between the plurality of crop areas determined by the distance calculation unit is a center of the plurality of crop areas on an imaging surface configured by a plurality of pixels arranged in a two-dimensional shape of the imaging unit. It is a pixel distance between points.

  More preferably, the image correction coefficient is at least one of a gamma correction gain, a white balance gain, a color correction coefficient, and an exposure correction value.

  More preferably, when the image correction coefficient corresponds to any two or more of a gamma correction gain, a white balance gain, a color correction coefficient, and an exposure correction value, the distance correction coefficient calculation unit includes the plurality of image correction coefficients. A different distance correction coefficient is obtained for each.

  More preferably, when cutting out a plurality of crop areas designated for each image of a plurality of images captured continuously in time, the correction coefficient calculation unit is configured to perform a correction between the images that are temporally continuous. When the distance between the specified plurality of crop areas approaches, the followability of the image correction coefficient for each of the plurality of crop areas is increased between the temporally continuous images.

  An image processing program according to the present invention is an image processing program executed by a computer, and includes image data of a plurality of crop areas cut out from the same image, distance information between the plurality of crop areas, An input procedure for inputting an image correction coefficient to be corrected to an optimum image for each of the plurality of crop areas and the entire image, and a distance between the plurality of crop areas for the image correction coefficient for each of the plurality of crop areas A distance correction coefficient calculation procedure for obtaining a distance correction coefficient to be adjusted based on the information; a correction processing procedure for correcting image data for each crop region using the image correction coefficient and the distance correction coefficient; and the correction processing unit. And a recording procedure for storing the image data for each crop area processed in the storage medium.

  The image processing apparatus and the image processing program according to the present invention can perform image processing optimum for an image in a crop area cut out from the same image.

It is a block diagram which shows the structure of the electronic camera 101 which concerns on this embodiment. It is an auxiliary | assistant figure for demonstrating the setting of a crop area | region and the pixel distance (L) between crop areas. It is an auxiliary | assistant figure which shows the example of a characteristic showing the relationship between the pixel distance (L) between crop areas and the distance correction gain Gain (L). It is a flowchart which shows the process of "crop photography mode".

  Embodiments relating to an image processing apparatus and an image processing program according to the present invention will be described below. In the present embodiment, an example in which the image processing apparatus and the image processing program according to the present invention are applied to the electronic camera 101 will be described. However, the present invention is not limited to a camera, and an apparatus that captures an image such as a scanner or a captured image. The present invention can also be applied to a computer that executes software for processing data, a dedicated image processing apparatus, and the like. In addition, the electronic camera 101 according to the present embodiment cuts out a plurality of designated crop areas from the entire photographed image, performs an optimum image processing for each crop area, and records it (hereinafter referred to as “crop photography mode”). An electronic camera. Hereinafter, the electronic camera 101 will be described in detail.

[Overall Configuration of Electronic Camera 101]
FIG. 1 is a block diagram illustrating a configuration of the electronic camera 101. The electronic camera 101 includes an imaging unit 102, an A / D conversion unit 103, a processing buffer 104, an image processing unit 105, a system bus 106, a memory card I / F (interface) 107, a display unit 108, The control unit 109, the operation unit 110, the photometric sensor 111, and the memory 112 are configured. Here, the system bus 106 is a data bus that connects blocks, and inputs and outputs control data, image data, and the like between the blocks.

  The imaging unit 102 includes an imaging optical system, a diaphragm and shutter mechanism, an imaging device, and the like, and is controlled by the control unit 109 via the system bus 106. For example, the control unit 109 performs exposure control such as position adjustment of the focus lens of the photographing optical system, aperture value adjustment, shutter speed adjustment, electronic shutter speed adjustment of the image pickup device, control of the reading area, and the like for the imaging unit 102. Do. This embodiment includes a case where an image sensor that can control the exposure amount for each crop area by adjusting the electronic shutter speed for each crop area in the image sensor.

  The A / D conversion unit 103 converts the analog image signal output from the imaging unit 102 into digital image data (RGB RAW data) and temporarily stores the digital image data in the processing buffer 104. Note that the image data stored in the processing buffer 104 is used as a buffer when the image processing unit 105 performs image processing, and the control unit 109 stores the image data stored in the processing buffer 104 via the system bus 106. Data is read and exposure control is performed, or a preview image is displayed on the display unit 108. Further, the control unit 109 records the image data processed by the image processing unit 105 from the processing buffer 104 to the memory card 107b attached to the memory card I / F 107.

  The image processing unit 105 includes a white balance adjustment unit 151, a color / gradation processing unit 152, and an outline enhancement unit 153. The image processing unit 105 cuts out the crop area designated by the control unit 109, and performs white balance adjustment processing, color / gradation processing, and edge enhancement processing described below for each cropped crop area.

  The white balance (WB) adjustment unit 151 performs the R / G gain and B calculated by the control unit 109 on the RGB image data taken into the processing buffer 104 from the imaging unit 102 via the A / D conversion unit 103. Multiply / G gain to adjust white balance.

  Here, the white balance adjustment will be described. White balance adjustment is a process of adjusting so that an achromatic subject is photographed as an achromatic image regardless of the characteristics of ambient light. In general, when the characteristics (such as color temperature) of ambient light (such as sunlight, incandescent light, and fluorescent light) change for an achromatic subject, the image data captured by the camera depends on the characteristics of ambient light. Change. Therefore, even when the ambient light changes, the RGB ratio of the image data must be adjusted so that the achromatic subject becomes an achromatic image. This is white balance adjustment.

  This white balance adjustment is realized, for example, by multiplying image data by gains (gainR, gainB) obtained according to the R and B channels. For example, an average value of image data obtained by shooting an achromatic subject is calculated for each of the R, G, and B channels. The calculated average values of the colors are Ra, Ga, and Ba, respectively. In this case, the white balance gains GainR and GainB can be obtained as in (Expression 1) and (Expression 2).

GainR = Ra / Ga (Formula 1)
GainB = Ba / Ga (Formula 2)
The white balance gains GainR and GainB obtained in (Equation 1) and (Equation 2) are respectively multiplied by the R data and B data of the captured image captured in the processing buffer 104 in the white balance adjustment unit 151 to obtain the white balance. RGB image data after adjustment is obtained. In particular, in the “crop shooting mode” of the electronic camera 101 according to the present embodiment, white balance adjustment is performed for each crop region designated by the control unit 109. Note that image data processed by the white balance adjustment unit 151 is also held in the processing buffer 104. In addition, when “Continuous Shooting Mode” or “Movie Shooting Mode” is selected, the white balance between consecutive images is prevented so that the white balance of images captured continuously in time does not change abruptly. Control is performed by the control unit 109 so that the followability of the balance gain becomes slow.

  The color / gradation processing unit 152 uses the color correction coefficient and the gamma correction gain given from the control unit 109 for the image data after white balance adjustment held in the processing buffer 104 to perform color correction processing and gradation processing. Perform key processing. For example, in the color correction process, a matrix operation that multiplies RGB image data by a color correction coefficient (for example, a color conversion matrix for changing the RGB component ratio) to adjust the hue and saturation, and in the gradation process, A gamma characteristic representing the relationship between image data and output luminance is adjusted. In particular, in the “crop shooting mode” of the electronic camera 101 according to the present embodiment, color correction processing, gradation processing, and the like are performed for each crop region designated by the control unit 109 in the same manner as white balance processing. Note that image data processed by the color / gradation processing unit 152 is also stored in the processing buffer 104. Similarly to white balance gain, the brightness and color of images shot continuously in time will not change abruptly when “continuous shooting mode” or “movie mode” is selected. In addition, the control unit 109 controls the color correction processing and the gradation processing between successive images so that the follow-up performance is delayed.

  The contour emphasizing unit 153 uses the edge emphasis parameter (edge emphasizing process) output from the control unit 109 for the image data processed by the color / gradation processing unit 152 held in the processing buffer 104. )I do. In particular, in the “crop shooting mode” of the electronic camera 101 according to the present embodiment, contour enhancement processing is performed for each crop region designated by the control unit 109. Note that image data processed by the contour emphasizing unit 153 is also held in the processing buffer 104.

  A memory card I / F (interface) 107 is an interface for connecting the memory card 107b to the electronic camera 101. The control unit 109 writes image data, setting data, and the like to the memory card 107b via the memory card IF 107, Image data and setting data stored in the memory card 107b are read out.

  The display unit 108 is configured with a liquid crystal monitor or the like, and includes image data stored in the memory card 107b, image data temporarily stored in the processing buffer 104, an operation menu screen output by the control unit 109, and the like. Is displayed.

  The control unit 109 operates according to a program stored in advance inside and controls each unit of the electronic camera 101. The configuration of the control unit 109 will be described in detail later.

  The operation unit 110 selects a power button, a release button, a cursor button, an enter button, a shooting mode selection dial (“single shooting mode”, “continuous shooting mode”, “movie shooting mode”, “crop shooting mode”, etc. Or a combination mode (a dial for selecting crop shooting during continuous shooting) or the like, and the user operates the electronic camera 101 using these operation buttons. Operation information of the operation unit 110 (button on / off, dial setting, etc.) is output to the control unit 109. The control unit 109 controls the entire operation of the electronic camera 101 in accordance with operation information input from the operation unit 110.

  The photometric sensor 111 is a sensor that measures the brightness of the subject imaged by the imaging unit 102, and photometric data is output to the control unit 109 via the system bus 106 and exposure controlled. In addition, without providing the photometric sensor 111, the brightness of the subject is detected from the image data when the preview image for confirming the composition at the time of shooting is taken into the processing buffer 104 from the imaging unit 102, and exposure control is performed. It doesn't matter. In the “crop shooting mode” of the electronic camera 101 according to the present embodiment, the photometric sensor 111 outputs a photometric value for each crop area to be described later to the control unit 109 in response to a command from the control unit 109. In particular, as described above, when the exposure amount for each designated area can be controlled by adjusting the electronic shutter speed for each crop area in the image sensor that constitutes the imaging unit 102, the exposure is performed for each designated crop area. By performing control, it is possible to photograph with an appropriate exposure amount for each crop region.

The memory 112 is a non-volatile memory in which the control unit 109 stores parameters necessary for setting and processing of the electronic camera 101. Note that a captured image or the like may be stored in the memory 112 instead of the memory card 107b.
[Configuration of Control Unit 109]
Next, the configuration of the control unit 109 in FIG. 1 will be described in detail. The control unit 109 includes an area designation unit 161, a distance calculation unit 162, an image correction coefficient calculation unit 163, and a distance correction unit 164. These processing blocks operate when the “crop shooting mode” is selected. To do.

  For example, as shown in FIG. 2A, FIG. 2B, and FIG. 2C, the area specifying unit 161 includes a crop area 201, a crop area 202, and the like out of the entire captured image captured by the imaging unit 102. Specify multiple crop areas. For example, as shown in the figure, the crop area is specified by displaying a dotted frame superimposed on the subject image displayed on the screen of the display unit 108, and operating the cursor button or the like of the operation unit 110 to display the dotted frame. The cropping area is determined with the enter button of the operation unit 110 according to the target subject by moving up / down / left / right or changing the width. Alternatively, the crop area may be automatically specified using a face recognition function that has already been put to practical use, or the crop area may automatically follow the movement of the subject using the subject tracking function. Absent.

  In this way, the area designating unit 161 designates a plurality of crop areas from the entire image captured by the imaging unit 102. Note that the crop area designated here is held in the memory 112 until a series of photographing is completed.

  The distance calculation unit 162 obtains the distance (L) between the plurality of crop areas designated by the area designation unit 161. For example, in the case of FIG. 2A, the pixel distance L1 of the image sensor of the imaging unit 102 between the center point of the crop area 201 and the center point of the crop area 202 is obtained. Here, the pixel distance is, for example, when the imaging device of the imaging unit 102 is configured with vertical: 1000 pixels and horizontal: 2000 pixels, the coordinates of the pixels on the imaging surface of the imaging device (vertical pixel number, horizontal pixel number). The imaging surface can be expressed from coordinates (0, 0) to coordinates (1000, 2000). At this time, for example, when the center point of the crop area 201 in FIG. 2A is the coordinate (600, 200) and the center point of the crop area 202 is the coordinate (400, 1800), the center point of the crop area 201 and the crop area The pixel distance L1 between the center point 202 and the center point 202 can be obtained by (Expression 3).

L1 = √ {(600−400) ^ 2 + (1800−200) ^ 2} (Expression 3)
Note that the maximum distance Lmax in the above case is a pixel distance between the coordinates (0, 0) and the coordinates (1000, 2000) of the diagonal position of the imaging surface of the image sensor, and thus the maximum distance Lmax is about 2250. Become.

  In this way, the distance calculation unit 162 obtains the distance (L) between the plurality of crop areas designated by the area designation unit 161. In the above description, the pixel distance is used, but if the parameter corresponds to the distance between a plurality of crop areas, the distance need not be based on the number of pixels.

  The image correction coefficient calculation unit 163 calculates an optimal image correction coefficient for each of a plurality of crop areas specified by the area specifying unit 161 and an optimal image correction coefficient for the entire captured image. Here, as described above, the image correction coefficient is a coefficient for correcting an image such as a gamma correction gain, a white balance gain, a color correction coefficient, or an exposure correction value (in the case of electronic shutter control). It is. In particular, in the present embodiment, the image correction coefficient calculation unit 163 obtains an optimal image correction coefficient for each of a plurality of crop areas. Here, the gamma correction gain, the white balance gain, the color correction coefficient, the exposure correction value, and the like including the image correction coefficient Para _ *** are optimal for the crop area 201 in the case of FIG. The correct image correction coefficient can be expressed as Para_Each (201), the optimal image correction coefficient for the crop area 202 can be expressed as Para_Each (202), and the optimal image correction coefficient for the entire image can be expressed as Para_All.

  In the above case, the image correction coefficient calculation unit 163 obtains an image correction coefficient Para_Each (201), an image correction coefficient Para_Each (202), an image correction coefficient Para_All, and the like. For example, when the crop area 201 and the crop area 202 are respectively cut out from the captured images of FIGS. 2A, 2B, and 2C, the crop area 201 is located in the shaded part, so that the crop area 201 is An image correction coefficient Para_Each (201) that makes the image brighter is obtained, and an image correction coefficient Para_Each (202) that makes the crop region 202 darker is obtained because the crop region 202 is located in the sunlit portion. In each figure of FIG. 2, the part with the oblique line in the whole image shows the shade, and the part without the oblique line shows the sun.

  Furthermore, an image correction coefficient Para_All optimal for the entire image is also obtained. The image correction coefficient Para_All that is optimal for the entire image is normally considered to be an average image correction coefficient of the image correction coefficient Para_Each (201) and the image correction coefficient Para_Each (202). Therefore, when the image correction coefficient Para_All is applied, The crop area 201 in the shaded area is an image of a slightly bluish color with an undertone, and the crop area 202 in the shaded area is an overexposed image. As will be described later, it is not always necessary to obtain the image correction coefficient Para_All that is optimal for the entire image unless the image correction coefficient Para_All is designated as a target “alignment destination area” for distance correction.

  In this way, the image correction coefficient calculation unit 163 calculates the optimal image correction coefficient for each of the plurality of crop areas designated by the area designation unit 161 and for the entire image.

  The distance correction unit 164 adjusts the image correction coefficient obtained by the image correction coefficient calculation unit 163 according to the distance (L) between the plurality of crop areas obtained by the distance calculation unit 162. For example, in the case of FIG. 2A, the pixel distance between the center point of the crop area 201 and the center point of the crop area 202 is L1, but in the case of FIG. 2B, the center point of the crop area 201 is The pixel distance between the center point of the crop area 202 is L2 smaller than L1. Further, in the case of FIG. 2C, the pixel distance between the center point of the crop area 201 and the center point of the crop area 202 is L3 which is smaller than L2.

  Here, if an optimal image correction coefficient is applied to each crop area obtained by the image correction coefficient calculation unit 163, an optimal image can be obtained in each crop area. However, as shown in FIG. A problem occurs when the area 201 and the crop area 202 are close enough to overlap. For example, since the crop area 201 in FIG. 2C has a lot of shaded areas and the crop area 202 has a lot of sun, when image correction is performed by applying an optimal image correction coefficient to each crop area, The two cropped cropped areas have different brightness and color. In particular, when two cut-out images are viewed side by side, there is a problem that the brightness and color of a portion (such as a ball) that overlaps both images are greatly different, giving the viewer a sense of incongruity.

  Therefore, the distance correction unit 164 adjusts the image correction coefficient obtained by the image correction coefficient calculation unit 163 according to the distance (L) between the plurality of crop areas obtained by the distance calculation unit 162.

  Here, as a method of adjusting the image correction coefficient, a process of adjusting the image correction coefficient continuously as appropriate according to the distance and a process of adjusting the image correction coefficient only when the distance is smaller than a threshold value can be considered. For example, in the latter case, as shown in FIG. 2A, when the pixel distance L1 between the center point of the crop area 201 and the center point of the crop area 202 is larger than a preset threshold, Apply the optimal image correction factor. That is, the image correction coefficient obtained by the image correction coefficient calculation unit 163 is not adjusted. On the contrary, as shown in FIG. 2C, when the pixel distance L3 between the center point of the crop area 201 and the center point of the crop area 202 is smaller than a preset threshold value, the image correction optimum for each crop area is performed. The distance correction unit 164 adjusts the image correction coefficient obtained by the image correction coefficient calculation unit 163 without applying the coefficient as it is. The threshold value may be a value set as a default value when the electronic camera 101 is manufactured, or the user may freely set the electronic camera 101 in the setting menu after purchase.

  Next, the process of adjusting the image correction coefficient appropriately and continuously according to the distance will be described in detail. For example, as shown in FIG. 3, the distance correction unit 164 is a distance correction for adjusting the image correction coefficient obtained by the image correction coefficient calculation unit 163 according to the distance between crop areas obtained by the distance calculation unit 162: L. Gain: Find Gain. Here, since the distance correction gain Gain is a function of L, it is expressed as a distance correction gain Gain (L). For example, since the pixel distance between the center point of the crop area 201 and the center point of the crop area 202 in FIG. 2A is L1, the distance correction gain at this time is Gain (L1). Similarly, the distance correction gain in the case of FIG. 2B is Gain (L2), and the distance correction gain in the case of FIG. 2C is Gain (L3). In FIG. 3, the conversion characteristic 401 indicating the relationship between the pixel distance L between the two crop areas and the distance correction gain Gain does not have to be a cubic curve, and may be, for example, a straight line 404 or a polygonal line shape. The approximate characteristic 405 may be used.

  In the conversion characteristic 401 of FIG. 3, the dotted circle 402 indicates a portion to which the distance correction gain Gain = 1.0 is applied, and the image correction coefficient calculation unit 163 obtains the distance correction gain Gain = 1.0. This means that the image correction coefficient is not adjusted. That is, the optimal image correction coefficient is applied to the crop area obtained by the image correction coefficient calculation unit 163 as it is. On the other hand, the dotted circle 403 indicates a portion to which the distance correction gain Gain = 0.0 is applied, and the distance correction gain Gain = 0.0 indicates the image correction coefficient obtained by the image correction coefficient calculation unit 163 in the crop area. Means not applicable. In this case, an image correction coefficient that is optimal for the reference image region (“alignment destination region”) is applied. For example, the image correction coefficient that is optimal for the entire image is applied with the entire image as the “alignment destination area”. In the case of FIG. 2C, an image correction coefficient that is optimal for the entire image is applied to both the crop area 201 and the crop area 202. As a result, even when the crop area 201 and the crop area 202 are viewed side by side, the brightness and color of the portions shown in both are the same, so there is no sense of incongruity. In the above description, the entire image is referred to as the “alignment destination area”, but the image correction coefficient that is optimal in either the crop area 201 or the crop area 202 is applied to one of the crop areas. It doesn't matter. Alternatively, the average value of the crop area 201 and the crop area 202 may be used as the image correction coefficient of the “alignment destination area”. In any case, since the same image correction coefficient is applied to both the crop area 201 and the crop area 202, there is no sense of incongruity in brightness and color when viewing the crop area 201 and the crop area 202 side by side.

  Here, a case where a plurality (N: N is a natural number) of crop areas exist in the entire image will be described. The first crop area is represented as a crop area (1), and the nth crop area is represented as a crop area (n) (n is a natural number from 1 to N). In the description here, the (optimal) image correction coefficient described above is expressed in other words as the (optimal) processing amount. This is because the image correction coefficient includes not only a value expressed by an absolute value such as a white balance gain but also a value expressed by a difference value such as an exposure value. Express.

  For example, the optimal processing amount of the crop area (n) for correction according to the distance is Para_Each (n), the optimal processing amount of the “matching destination area” is Para_target, and the processing amount of the crop area (n) after the distance correction is Para_adj (n). The optimum processing amount may be a difference value or an absolute value. For example, when the optimum processing amount is a difference value, the optimum exposure amount is the optimum exposure of the crop area 201 = 0.3 eV, the optimum exposure of the crop area 202 = −0.3 eV, etc. Become. Alternatively, as an example of the case where the optimum processing amount is an absolute value, in the case of the white balance gain shown in (Expression 1) or (Expression 2), the optimal white balance gain (R / G) of the entire image = 2. 2, the optimum white balance gain (R / G) = 2.1 in the crop area 201, the optimum white balance gain (R / G) = 1.8 in the crop area 202, and the like are optimum processing amounts.

The target optimum processing amount Para_target of the “matching destination area” may be based on any of the following optimum processing amounts.
(1) Optimum processing amount of entire image (2) Optimal processing amount of designated crop area in plural crop areas (3) Average value of optimum processing amount of plural crop areas (4) Separate from captured image Optimum processing amount set in advance Note that the optimum processing amount used here corresponds to the image correction coefficient obtained by the image correction coefficient calculation unit 163 as described above, and the image correction coefficient (which is obtained by the image correction coefficient calculation unit 163 ( The white balance gain, gamma correction gain, color correction coefficient, exposure correction value (in the case of electronic shutter control) and the like may be not only absolute values but also differential values.

For example, in FIG. 3, when the optimal processing amount of the entire image is the optimal processing amount Para_target of the “fit destination region” and the optimal processing amount Para_Each (n) of each of the N crop regions (n), The processing amount Para_Adj (n) of each of the N crop areas (n) can be obtained as shown in (Expression 4).
Para_Adj (n) = Gain (L) x Para_Each (n) + (1-Gain (L)) x Para_target ... (Formula 4)
As described in FIG. 2, in the case of two crop areas, the distance between the two crop areas is set to L, but when there are three or more crop areas, one reference crop area is selected. Then, the distance from the crop area may be obtained. For example, when n = 1 crop region (1) is used as the optimum processing amount Para_target of the “matching destination region”, the processing amount Para_Adj (n) of each of the N crop regions (n) after the distance correction is expressed by (Expression 5). ).
Para_Adj (n) = Gain (L) x Para_Each (n) + (1-Gain (L)) x Para_Each (1) ... (Formula 5)
At this time, as shown in (Equation 6), the crop area (1) itself of the “alignment destination area” is not corrected by the distance, and the optimum processing amount of the own area is applied as it is.
Para_Each (1) = Gain (L) × Para_Each (1) + (1-Gain (L)) × Para_Each (1)
= Para_Each (1) (Formula 6)
Here, the distance correction gain Gain (L) decreases as the distance L shown in FIG. 3 becomes smaller (the crop area becomes closer). Therefore, in “Expression 4” and “Expression 5”, The influence of the optimum processing amount becomes large, and in the case of the same position (L = 0), the optimum processing amount of the “alignment destination area” is applied as it is. On the other hand, the distance correction gain Gain (L) increases as the distance L shown in FIG. 3 increases (the crop area becomes far), and therefore, in (Expression 4) and (Expression 5), The influence of the optimum processing amount is reduced, and in the case of a position close to the diagonal distance of the screen (L = 3000), the distance correction gain Gain (L) = 1, so that the optimum processing amount of the own area is applied as it is.

  As described above, when the distance L between the two crop areas is long, the distance correction gain Gain (L) is increased and optimal correction is performed for the own area, but as the distance L between the two crop areas decreases. Since the distance correction gain Gain (L) is reduced and correction close to the “alignment destination area” is performed, the brightness and color of the image between the crop areas are corrected so that the images in the two crop areas are Discomfort when viewing side by side is eliminated.

  In the above description, the distance correction unit 164 adjusts the image correction coefficient obtained by the image correction coefficient calculation unit 163 according to the distance L between the crop areas obtained by the distance calculation unit 162. When there are a plurality of gains and coefficients such as white balance gain, gamma correction gain, color correction coefficient, and exposure correction value, different distance correction may be performed for each gain and coefficient. For example, in FIG. 3, the distance correction of the white balance gain is performed using the characteristic 401, the distance correction of the gamma correction gain is performed using the approximate characteristic 405, and the distance correction of the color correction coefficient is performed using the straight line 404. It does not matter if only one of them is correct.

  The distance correction unit 164 described above is a distance correction gain for adjusting the image correction coefficient obtained by the image correction coefficient calculation unit 163 according to the distance (L) between the crop areas obtained by the distance calculation unit 162. Gain (L) is obtained, and an image correction coefficient adjusted with the obtained distance correction gain Gain (L) is given to the image processing unit 105. Alternatively, when the image correction coefficient is an exposure correction value, the electronic shutter of the imaging unit 102 is controlled to perform exposure correction for each crop area.

In this manner, the image processing unit 105 can perform white balance processing, gamma correction processing, color correction processing, exposure control, and the like with the image correction coefficient corrected according to the distance between the crop areas.
[Operation of “Crop shooting mode”]
Next, the processing of the control unit 109 when “Crop shooting mode” is selected with the shooting mode selection dial of the operation unit 110 will be described in detail with reference to the flowchart of FIG. It is assumed that the electronic camera 101 is turned on.

  (Step S <b> 101) The control unit 109 starts the “crop shooting mode” process when the user selects “crop shooting mode” with the shooting mode selection dial of the operation unit 110. In this state, a preview image for determining the shooting composition is displayed on the display unit 108.

  (Step S102) The area designating unit 161 of the control unit 109 sets a crop area from the subject images displayed on the display unit 108. Here, the setting of the crop area is performed by the area designating unit 161 of the control unit 109 as described above. The user changes the position and size of the frame indicating the crop area displayed on the display unit 108 using the cursor button of the operation unit 110, for example, the crop area 201 and the crop area 202 shown in FIG. Set the crop area as follows. The area designating unit 161 stores the set crop area in the memory 112 as the pixel coordinates of the image sensor of the imaging unit 102. For example, in the case of the crop area 201 in FIG. 2A, the coordinates of the diagonal positions of the lower left coordinates (500, 20) and the upper right coordinates (980, 400) of the area are stored as the coordinates of the crop area.

  (Step S103) The control unit 109 determines whether or not the release button of the operation unit 110 has been pressed, and waits until it is pressed. If the release button has been pressed, the process proceeds to step S104.

  For each crop area set in step S102, fixed (does not move position), movable (automatic tracking corresponding to continuous shooting, video shooting, etc., changes the position and size of the crop area), etc. It may be possible to set the following. For example, when the subject moves before the release button is pressed or the position of the electronic camera 101 changes, a method of automatically tracking the subject using the face recognition function of the electronic camera 101 can be considered. . For example, control is performed so that a region expanded vertically and horizontally from the face region recognized by the face recognition function is set as a crop region. In this case, the size of the crop area is changed in accordance with the size of the recognized face area, and the vertical width of the face area is 0.5 times the vertical width of the face area in the upward direction and the vertical width of the face area in the downward direction. Processing is performed so that the size of the crop area is automatically changed, such as 3.0 times and 1.0 times the left-right width of the face area in the left-right direction. As a result, the size of the crop area decreases when the face of the person set for face recognition decreases, and the size of the crop area increases when the face of the person increases, so the face recognition function automatically tracks the subject while tracking the subject. It is possible to automatically change the position and size of the crop area according to the size. Furthermore, if the direction of the head is determined from the positions of the eyes, nose, mouth, etc. of the recognized face area, the position and size of the crop area can be automatically and appropriately even when a person is shown obliquely or sideways. Can be varied.

  (Step S104) The distance calculation unit 162 of the control unit 109 measures the distance (L) between the crop regions when there are a plurality of crop regions set in Step S102.

  As described with reference to FIG. 2, in the case of two crop areas, the distance between the two crop areas is set to L. However, when there are three or more crop areas, the reference is as described above. The crop area may be selected as the “matching area” and the distance from the crop area may be obtained.

  Alternatively, when there are three or more crop areas, the two crop areas having the closest distance may be selected and set as the distance (L) between the crop areas. In this case, for example, when three crop areas A, B, and C are set, the distance between the crop area A and the crop area B is a distance (Lab), and the distance between the crop area B and the crop area C is a distance (Lbc). , The distance between the crop area A and the crop area C is a distance (Lac). At this time, it is assumed that the relationship between the distance (Lab), the distance (Lbc), and the distance (Lac) is distance (Lab) <distance (Lbc) <distance (Lac). In this case, the distance (Lab) is applied to the crop area A and the crop area B, and the distance (Lbc) is applied to the crop area C.

  (Step S105) The image correction coefficient calculation unit 163 of the control unit 109 calculates an optimal image correction coefficient for each crop region set in step S102. When the entire image is selected as the “matching area”, an image correction coefficient optimum for the entire image is also obtained. The image correction coefficient calculated in this processing step is a gamma correction gain, a white balance gain, a color correction coefficient, an exposure correction value, or the like as described above.

  When the image correction coefficient is an exposure correction value, the electronic shutter speed is adjusted for each crop area in the image sensor that constitutes the image capturing unit 102 to control the exposure amount for each designated area, and is appropriate for each crop area. Shoot with a good exposure. In this case, the release button is pressed in step S103, but the release button is pressed after step S105. At this time, the processes in steps S104 and S105 are performed during preview display. In addition, when an extremely large brightness adjustment is performed on image data (RAW data) captured in the processing buffer 104 after shooting, whiteout or blackout occurs due to lack of gradation information in the RAW data. Since there is a fear, as described above, it is preferable to control the exposure amount for each crop area by controlling the electronic shutter speed for each crop area during shooting.

  (Step S106) The distance correction unit 164 of the control unit 109 performs correction according to the distance (L) between the crop areas obtained in step S104, and corrects the optimum image correction coefficient for each crop area obtained in step S105. An image correction coefficient after correction is obtained.

  Then, the image processing unit 105 performs image correction of each crop region using the image correction coefficient after distance correction. Alternatively, when the image correction coefficient is an exposure correction value, the electronic shutter of the imaging unit 102 is controlled to perform exposure correction for each crop area.

  (Step S107) The control unit 109 stores the image data for each crop area after the image processing in step S106 in the memory card 107b via the memory card I / F 107.

  (Step S <b> 108) The control unit 109 ends a series of “crop shooting modes”.

  In the above flowchart, the image data after the image processing using the image correction coefficient after the distance correction in step S106 is stored in the memory card 107b, but the image correction after the distance correction in step S106. In step S107, image processing using coefficients is not performed, and image processing is performed using image data before image processing (RAW data) or image correction coefficients that are not subjected to distance correction (image correction coefficients that are optimal for each crop region). You may make it preserve | save the image data which performed it to the memory card 107b. In this case, the image data of each crop area, the distance information obtained in step S104, and the optimum image correction coefficient for each crop area are stored in the memory card 107b in association with each other. For example, distance information and image correction coefficients may be stored as header information of image data of each crop area.

  In the above description, the case of single shooting (single shooting) has been described. However, as described above with reference to FIG. 2, when continuous shooting is performed in a sports scene or the like, the crop area is set as described above. If the process from step S104 to step S107 in the flowchart of FIG. 4 is repeated as many times as the number of continuous shots so as to automatically track according to the movement of the subject, the distance between the crop areas is the same as in the case of single-shot shooting. It is possible to take an image that has been appropriately corrected according to the above.

  In general, when “continuous shooting mode” or “movie shooting mode” is selected in white balance processing, color correction processing, gradation processing, and exposure control, Parameters for white balance processing, color correction processing, gradation processing, and exposure control between consecutive images (white balance gain, color correction coefficient, gamma correction gain, exposure correction value, etc.) so that brightness and color do not change abruptly In the electronic camera 101 according to the present embodiment, the followability of the image correction coefficient is set between crop regions. You may make it vary according to distance. For example, if the distance between crop areas is farther than a preset threshold value, the followability of white balance processing, color correction processing, and gradation processing parameters between successive images is slowed, and the distance between crop areas is preset. If it is closer to the threshold value, the control unit 109 may control the image processing unit 105 so as to speed up the follow-up of parameters of white balance processing, color correction processing, and gradation processing between successive images. Alternatively, the followability of the image correction coefficient may be varied continuously in succession according to the distance between the crop areas without providing a threshold value. By performing such control, when the two crop areas are close, it can be quickly converged without being affected by the history of the parameters of the previous captured image, so that it has been described in the previous embodiment. In addition to the effect, differences in image brightness, white balance, etc. between crop areas can be further reduced.

  In this way, the electronic camera 101 according to the present embodiment can perform optimal image processing for each crop area in accordance with the distance between the plurality of crop areas. Even when the crop areas are close to each other, the brightness and white balance of the two crop areas are not extremely different, and an unnatural impression is not given when viewing images of these crop areas side by side.

  In this embodiment, the case where the image processing apparatus and the image processing program according to the present invention are applied to the electronic camera 101 has been described. However, when image data that has already been taken is processed by a dedicated image processing apparatus or a personal computer. The processing performed by the control unit 109 of FIG. 1 and the processing performed by the flowchart of FIG. 4 may be executed. In particular, the image data of a plurality of crop areas cut out from the same image photographed by the electronic camera 101 described in the present embodiment, distance information between the plurality of crop areas, and an optimal image for each of the plurality of crop areas. When displaying an image of crop areas at adjacent positions at a distance by providing an input unit (or an input procedure in the case of an image processing program) for inputting a memory card 107b in which image correction coefficients to be corrected are stored However, by performing the distance correction process performed in step S106 of the flowchart of FIG.

  As described above, the image processing apparatus and the image processing program according to the present invention have been described by way of examples in the embodiment, but can be implemented in various other forms without departing from the spirit or main features thereof. . Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 101 ... Electronic camera 102 ... Imaging part 103 ... A / D conversion part 104 ... Processing buffer 105 ... Image processing part 106 ... System bus 107 ... Memory card I / F 108 Display unit 109 Control unit 110 Operation unit 111 Photometric sensor 112 Memory 151 White balance (WB) adjustment unit 152 Color / tone processing unit 153 ... Outline emphasis part 161 ... Area designation part 162 ... Distance calculation part 163 ... Image correction coefficient calculation part 164 ... Distance correction part

Claims (9)

An imaging unit for imaging a subject image;
A crop area designating unit for designating a plurality of crop areas in a subject image captured by the imaging unit;
A distance calculation unit for obtaining a distance between the plurality of crop regions designated by the crop region designation unit;
A correction coefficient calculation unit for obtaining an image correction coefficient for correcting each of the plurality of crop regions and the entire image to an optimum image;
The image data of the plurality of crop areas designated by the crop area designation unit, the distance information between the plurality of crop areas calculated by the distance calculation unit, and the image correction coefficient obtained by the correction coefficient calculation unit are associated with each other. And a recording unit for storing in a storage medium. An input unit for inputting image data of a plurality of crop areas cut out from the same image, distance information between the plurality of crop areas, and an image correction coefficient for correcting the image to an optimum image for each of the plurality of crop areas; ,
A distance correction coefficient calculation unit for obtaining a distance correction coefficient for adjusting the image correction coefficient for each of the plurality of crop areas based on distance information between the plurality of crop areas;
A correction processing unit that corrects the image data for each crop region using the image correction coefficient and the distance correction coefficient;
An image processing apparatus comprising: a recording unit that stores, in a storage medium, image data for each of the crop areas processed by the correction processing unit. An imaging unit for imaging a subject image;
A crop area designating unit for designating a plurality of crop areas in a subject image captured by the imaging unit;
A distance calculation unit for obtaining a distance between the plurality of crop regions designated by the crop region designation unit;
A correction coefficient calculation unit for obtaining an image correction coefficient for correcting each of the plurality of crop regions and the entire image to an optimum image;
A distance correction coefficient calculating unit for obtaining a distance correction coefficient for adjusting an image correction coefficient for each of the plurality of crop areas based on distance information between the plurality of crop areas;
A correction processing unit that corrects the image data for each crop region using the image correction coefficient and the distance correction coefficient;
An image processing apparatus comprising: a recording unit that stores image data for each crop area processed by the correction processing unit in a storage medium. The image processing apparatus according to claim 2 or 3,
The distance correction coefficient calculation unit obtains the distance correction coefficient based on an image correction coefficient of the entire image or any one of the plurality of crop areas. The image processing apparatus according to any one of claims 1 to 4,
The distance between the plurality of crop areas determined by the distance calculation unit is a pixel distance between the center points of the plurality of crop areas on an imaging surface configured by a plurality of pixels arranged in a two-dimensional shape of the imaging unit. An image processing apparatus comprising: In the image processing device according to any one of claims 1 to 5,
The image processing apparatus is characterized in that the image correction coefficient is at least one of a gamma correction gain, a white balance gain, a color correction coefficient, and an exposure correction value. In the image processing device according to any one of claims 1 to 5,
When the image correction coefficient corresponds to any two or more of a gamma correction gain, a white balance gain, a color correction coefficient, and an exposure correction value, the distance correction coefficient calculator is different for each of the plurality of image correction coefficients. An image processing apparatus characterized in that In the image processing device according to any one of claims 1 to 7,
When cutting out a plurality of crop areas designated for each image of a plurality of images taken continuously in time,
The correction coefficient calculating unit corrects the image for each of the plurality of crop regions between the temporally continuous images when the distance between the specified crop regions is close between the temporally continuous images. An image processing apparatus characterized in that coefficient followability is increased. An image processing program executed on a computer,
Image data for a plurality of crop areas cut out from the same image, distance information between the plurality of crop areas, and an image correction coefficient for correcting each of the plurality of crop areas and the entire image to an optimum image Input procedure to input and
A distance correction coefficient calculation procedure for obtaining a distance correction coefficient for adjusting an image correction coefficient for each of the plurality of crop areas based on distance information between the plurality of crop areas;
A correction processing procedure for correcting the image data for each crop region using the image correction coefficient and the distance correction coefficient;
An image processing program comprising: a recording procedure for storing, in a storage medium, image data for each of the crop areas processed by the correction processing unit.

Images