JP2009239772A - Imaging device, image processing device, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable executing a process such as a color correction corresponding to a scene by performing an exact scene determination. <P>SOLUTION: An imaging device carries out processes to calculate a pixel value ratio of each color in an unit of a block as a sectional area of a RAW image, to produce a histogram of frequency distribution information of the calculated pixel value ratio, and to practice the determination processing of the shot scene to make a pixel processing correspondent to a determination result by a comparison processing between the histogram and a scene determination dictionary. The histogram and a parameter of rectangle feature characteristic determination data are compared to make a scene determination. The composition can exactly determine a various scenes (outdoor, evening scene, night scene, indoor, and in-water, etc.), and the processing etc. of the optimum color correction specialized to scenes becomes possible. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device, an image processing device, an image processing method, and a program. More specifically, the present invention relates to an imaging apparatus, an image processing apparatus, an image processing method, and a program that can perform scene determination based on image characteristics and perform correction processing according to the determination result.

  When shooting images with a camera, there are various shooting situations such as outdoors, evening scenes, night scenes, indoors, or underwater, that is, shooting scenes. These shot images are of high quality by performing corrections according to each shooting scene. It is possible to output a simple image. For example, in white balance adjustment processing, color reproduction processing for adjusting saturation and hue, and the like, it is possible to improve the quality of an output image by performing correction using different parameters according to the shooting scene. Specifically, for example, an underwater photograph has an overall bluish image, and by performing image correction such as reducing bluishness, a high-quality image can be output. As described above, there are cases where it is required to perform image processing such as color correction according to the shooting scene.

As a conventional technique that discloses image processing according to a shooting scene, for example, there is the following technique.
(1) There is a method of analyzing the detection data from the external light at the time of photographing, and using this analysis result for scene discrimination to determine the optimum white balance at the time of strobe light emission and non-issuance to perform color correction. Such a technique is described in, for example, Japanese Patent Application Laid-Open No. 2001-358986.
Also,
(2) As a technique for discriminating a scene using the color of a captured image, a color feature amount is calculated from the image, an average value of colors of all pixels is calculated, and using this calculation result and a preset threshold value, There is a method for determining what scene was shot.

However, the method (1) is configured to determine the scene based on the brightness at the time of shooting, and does not make a determination regarding the color and cannot be applied to a process of changing the color correction mode.
The method (2) is a configuration in which an average value of color feature amounts in the entire image is calculated, and a correction mode is determined based on an average feature of the entire image. In such a method, the colors of various objects existing in the image affect the average value of the color feature amount of the entire image. Therefore, for example, depending on the combination of colors in the video even in different scenes, similar feature amounts may be output, which increases the possibility of erroneous scene discrimination.
JP 2001-358986 A

  The present invention has been made in view of the above problems, for example, and does not use an average value of colors of an image but analyzes a distribution state of colors included in the image, thereby enabling more accurate scenes. An object of the present invention is to provide an imaging apparatus, an image processing apparatus, an image processing method, and a program that enable discrimination and realize appropriate correction according to a discrimination scene.

The first aspect of the present invention is:
An imaging device that executes imaging processing and outputs a RAW image;
A RAW image acquired by the image sensor or a reduced RAW image obtained by reducing the RAW image is input, a pixel value ratio of each color is calculated in units of blocks that are segmented regions of the input RAW image, and the calculated pixel value ratio A feature amount calculation unit that generates a histogram which is frequency distribution information;
An image discriminating unit that performs a comparison between a histogram calculated by the feature amount calculating unit and a scene discriminating dictionary stored in a storage unit, and executes a shooting scene discriminating process of the RAW image;
An image processing unit that executes image processing according to a scene determination result of the image determination unit;
In an imaging apparatus having

  Furthermore, in an embodiment of the imaging apparatus of the present invention, the feature amount calculation unit calculates a ratio R / G, B / G of each RGB pixel value in units of blocks constituting the input RAW image, and the input RAW This is a configuration for generating a histogram having a frequency distribution of appearance frequencies of a plurality of R / G and B / G values corresponding to all the constituent blocks of an image.

  Furthermore, in an embodiment of the imaging apparatus of the present invention, the imaging apparatus includes an image reduction unit that generates a reduced RAW image by executing a reduction process of the RAW image output from the imaging element, and the feature amount calculation The unit is configured to generate the histogram by inputting the reduced RAW image generated by the image reduction unit.

  Furthermore, in an embodiment of the imaging apparatus of the present invention, the storage unit sets parameters defining rectangular shape feature determination data (Rectangle Feature) for determining the frequency distribution tendency of the histogram to scenes corresponding to a plurality of different scenes. Stored as a discrimination dictionary, and the image discrimination unit executes a comparison process between the histogram calculated by the feature amount calculation unit and the parameter stored in the storage unit, and is similar to the histogram at a predetermined similarity or higher. A parameter-corresponding scene having a degree is determined to be a scene of the RAW image.

  Furthermore, in one embodiment of the imaging apparatus of the present invention, the storage unit stores a result of calculating a parameter defining the rectangular shape feature determination data (Rectangle Feature) using learning data based on an image corresponding to a specific scene. This is the configuration.

  Furthermore, in an embodiment of the imaging apparatus of the present invention, the image processing unit is configured to execute color correction processing specialized for a scene discrimination result of the image discrimination unit.

Furthermore, the second aspect of the present invention provides
A RAW image acquired by the image sensor or a reduced RAW image obtained by reducing the RAW image is input, a pixel value ratio of each color is calculated for each block which is a segmented area of the input RAW image, and the frequency of the calculated pixel value ratio A feature amount calculation unit for generating a histogram as distribution information;
An image discriminating unit that performs a comparison between a histogram calculated by the feature amount calculating unit and a scene discriminating dictionary stored in a storage unit, and executes a shooting scene discriminating process of the RAW image;
An image processing unit that executes image processing according to a scene determination result of the image determination unit;
In an image processing apparatus.

Furthermore, the third aspect of the present invention provides
A feature amount calculation unit inputs a RAW image acquired by an image sensor or a reduced RAW image obtained by reducing the RAW image, calculates a pixel value ratio of each color in units of blocks that are divided regions of the input RAW image, and calculates A feature amount calculating step for generating a histogram which is frequency distribution information of the pixel value ratio,
An image discriminating step, wherein the image discriminating unit performs comparison between the histogram calculated in the feature amount calculating step and the scene discriminating dictionary stored in the storage unit, and executes the shooting scene discriminating process of the RAW image;
An image processing unit that performs image processing according to a scene determination result in the image determination step;
An image processing method having

Furthermore, the fourth aspect of the present invention provides
A program for causing an image processing apparatus to execute image processing,
A RAW image acquired by the image sensor or a reduced RAW image obtained by reducing the RAW image is input to the feature amount calculation unit, and a pixel value ratio of each color is calculated for each block that is a divided region of the input RAW image. A feature amount calculating step for generating a histogram which is frequency distribution information of the pixel value ratio,
An image discriminating step for causing the image discriminating unit to perform a comparison between the histogram calculated in the feature amount calculating step and the scene discriminating dictionary stored in the storage unit, and to execute a shooting scene discriminating process of the RAW image;
An image processing step for causing the image processing unit to execute image processing according to a scene determination result in the image determination step;
Is in a program with

  The program of the present invention is, for example, a program that can be provided by a storage medium or a communication medium provided in a computer-readable format to a general-purpose system capable of executing various program codes. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of one embodiment of the present invention, the pixel value ratio of each color is calculated and calculated for each block, which is a divided area of a RAW image acquired by an image sensor or a reduced RAW image obtained by reducing the RAW image. A histogram which is frequency distribution information of the pixel value ratio is generated, a comparison between the generated histogram and a dictionary for scene determination stored in advance in a storage unit is performed, and a shooting scene determination process for a RAW image is performed. Then, image processing according to the determination result is executed. Specifically, since the scene is discriminated by comparing the histogram and the parameters that define the rectangular feature determination data (Rectangle Feature) prepared for various scenes, brightness and color Compared to analysis using average values, various scenes (outdoor, sunset, night view, indoor, underwater, etc.) can be determined more accurately, and optimal color correction specialized for each scene, etc. Image correction processing can be executed.

  Hereinafter, an imaging device, an image processing device, an image processing method, and a program according to an embodiment of the present invention will be described in detail with reference to the drawings.

  An embodiment of an image processing apparatus and an imaging apparatus according to the present invention will be described with reference to FIG. The configuration illustrated in FIG. 1 illustrates a configuration example of the imaging apparatus 100. When the image capturing apparatus 100 captures an image, the subject image is captured by the lens 101. Image light taken in from the lens 101 is converted into an electric signal (analog signal) by photoelectric conversion in the image sensor 102. Next, the electrical signal (analog signal) is converted into a digital signal by the digital signal converting unit 103.

  The digital signal generated by the digital signal converting unit 103 is a signal according to the arrangement of the imaging elements 101, and is a signal (mosaic signal) in which specific color signals are arranged on a mosaic. Specifically, for example, a RAW image having R, G, and B data arrays as shown in FIG. In the example shown in FIG. 2, the G signal is composed of two types of Gr and Gb. This indicates that Gr and Gb have slightly different color signals of G + b and G + b. However, this arrangement is a configuration corresponding to the arrangement of the image sensors 101. When three simple RGB image sensor configurations are applied, three color signal arrays of RGB are obtained.

  The RAW image signal shown in FIG. 2 is an example, and the color signal array applied in the present invention is not limited to this array, and other configurations such as a simple RGB color signal array may be used.

  In the RAW image shown in FIG. 2, each pixel has signal values of an R signal, a G (Gr or Gb) signal, and a B signal. The pixel value of each pixel is held at, for example, 16 bits. When the final output image is generated, the output values of R, G, and B of each pixel are determined by demosaic processing based on the values of these individual color signals.

  The RAW image as shown in FIG. 2 is generated by the digital signal conversion unit 103 of the imaging apparatus 100 shown in FIG. 1 and input to the scene determination unit 110 of the imaging apparatus 100 shown in FIG. The scene discrimination unit 110 inputs this RAW image data and performs scene discrimination. As shown in FIG. 1, the scene determination unit 110 includes an image reduction unit 111, a feature amount calculation unit 112, and an image determination unit 113.

  The image reduction unit 111 executes a reduction process for the image generated by the digital signal conversion unit 103. This is a process for reducing the processing load in the feature quantity calculation unit 112 in the subsequent stage. For example, the RAW image having a size of 247 × 1124 pixels generated by the digital signal converting unit 103 is reduced to 60 × 80 pixels. Note that if the processing unit subsequent to the image reducing unit 111, that is, the feature amount calculating unit 112 or the like can process large image data, the image reducing unit 111 is omitted and the digital signal converting unit 103 is omitted. The generated RAW image may be directly input to the feature amount calculation unit 112 to execute processing.

A specific example of processing executed by the image reduction unit 111 will be described with reference to FIG. In FIG.
(A) RAW image input from digital signal conversion unit 103 (b) Reduced image generated by image reduction unit 104 (c) Configuration of blocks constituting reduced image These are shown.

  (A) The RAW image is a RAW image having a size of 247 × 1124 pixels generated by the digital signal conversion unit 103, and the image reduction unit 104 converts the size 247 × 1124 pixels into a (b) reduced image having a size of 60 × 80 pixels. Reduce to. If a fraction is given at this time, it will be rounded off from the right end and the lower end. (B) As shown in the block (c), one block of the reduced image is composed of four pixels, one for each of R and B, and two for G. These four pixel values are calculated according to the following calculation formula by inputting the pixel values of the constituent pixels of the (a) RAW image to be reduced as a block.

  Specifically, the average values of the R, G, and B values in the block are calculated according to the following equations. However, Gr and Gb are calculated together and the average is G. The calculated value is rounded off to the first decimal place to obtain an integer of 16 bits. That is, the pixel values of the constituent pixels R, G, and B of the block are calculated according to the following formula. The pixel values of the two G pixels are set to the same value.

  The values of R, G, and B calculated by the above formula are set to the pixel values of the four pixels of the block, and an image of 30 × 40 = 1200 blocks with the four pixels as one block is defined as (b) a reduced image. .

The feature amount calculation unit 104 receives the reduced RAW image of 60 × 80 pixels created by the image reduction unit 103, and calculates the color feature amount of the reduced RAW image. The feature amount calculation unit 104 converts the reduced RAW image of 60 × 80 pixels into an image of 30 × 40 blocks in units of blocks configured by the four pixels illustrated in FIG. , G (two G are common), the following two color ratios are calculated as image feature amounts. That is,
R / G,
B / G,
It is.

R / G is a ratio of the R pixel value and the G pixel value of one block.
B / G is a ratio of the B pixel value and the G pixel value of one block.
Although there are two G pixel values in one block, as described above, the pixel values of two G pixels in one block are calculated as the same value, so which value is used. Is the same. The feature amount calculation unit 104 calculates these pixel value ratios as feature amount data corresponding to each block.

  The feature amount calculation unit 104 calculates the image feature amounts, R / G, and B / G, which are the above color ratios, for all 30 × 40 = 1200 blocks constituting the reduced RAW image. Image feature amounts [R / G] and [B / G] as color ratios corresponding to 1200 blocks are calculated from one frame of the reduced RAW image.

  Further, the feature quantity calculation unit 104 uses the 1200 sets of data corresponding to these blocks, and a histogram on a two-dimensional plane constituted by R / G and B / G axes as shown in FIG. Create The vertical axis corresponds to the number of blocks having the same feature value set ([R / G], [B / G]). This histogram is an image feature histogram indicating feature amounts based on the color of the image.

  Details of the image feature histogram generation processing shown in FIG. 4 will be described with reference to FIGS. In FIG. 5, (1) (2) block is acquired from the reduced RAW image, and (2) image feature amounts, R / G, B / G, which are color ratios, are calculated from the RGB pixel values in the block. The (3) image feature histogram generated by the calculation result is shown.

  The feature amount calculation unit 104 calculates the value of (R / G, B / G) in (1) all 1200 blocks included in the reduced RAW image, and based on 1200 pieces of (R / G, B / G) data. (C) An image feature histogram is generated with (R / G, B / G) as the two-dimensional plane and the vertical axis as the appearance frequency.

  A specific histogram generation sequence will be described with reference to the flowchart shown in FIG. This process is executed as the process of the feature quantity calculation unit 105 shown in FIG. First, in step S101, the reduced RAW image generated by the image reduction unit is input. In step S102, the input reduced RAW image is divided into 1200 blocks in which 4 pixels are included in one block.

  The reduced RAW image input in step S101 is, for example, RAW image data including 1200 blocks including 4 pixels in one block illustrated in FIGS. 4 and 5.

  Next, in step S103, one block data is read, and in steps S104 to S106, average values of pixel values R, G, and B of pixels included in the read block are calculated. In this embodiment, as described with reference to FIG. 4 and FIG. 5, one block is composed of two pixels each having R and B and G having the same pixel value. Therefore, the average value calculation can be omitted only by acquiring the RGB pixel values in the block.

  Various setting modes are possible for the setting mode of one block. For example, when there are a plurality of R pixels, G pixels, and B pixels having different pixel values in one block, the average value in steps S104 to S106. It is necessary to perform calculation.

  Next, in steps S107 and S108, color ratios R / G and B / G based on the RGB pixel values in the block indicating the image feature amount are calculated. Next, in step S109, the count value (appearance frequency) of the calculated (R / G, B / G) value set is set to +1. Next, in step S110, it is determined whether or not all the processes of 1200 blocks included in the reduced RAW image have been completed. If not, the process returns to step S103, and unprocessed blocks are read to perform step S104. The following process is repeated.

  If it is determined in step S110 that all the processing of 1200 blocks included in the reduced RAW image has been completed, the process proceeds to step S111, and the count values (appearance) of the (R / G, B / G) values of all 1200 blocks are displayed. Histogram made up of color ratio (R / G, B / G) plane and frequency information, that is, (R / G, B / G) described with reference to FIG. 4 and FIG. As a result, an image feature (color ratio) histogram (Ratio Histgram) is generated with the frequency as the vertical axis.

  The feature quantity calculation unit 105 shown in FIG. 1 generates an image feature (color ratio) histogram (Ratio Histgram) through such processing.

  Next, the image determination unit 105 illustrated in FIG. 1 performs an image scene determination process using the histogram generated by the feature amount calculation unit 104. The image discriminating unit 105 performs image scene discrimination processing using dictionary data obtained from learning data based on histograms obtained by photographing scenes such as underwater, sky, evening scene, night scene, trees, snow, and autumn leaves. I do. The image discriminating unit 105 uses dictionary data corresponding to various scenes (underwater, sky, evening scene, night scene, tree, snow, autumn leaves, etc.) held in the storage unit to determine which type of histogram of the scene discrimination target image. It is determined whether the image corresponds to the scene.

The dictionary data stored in the storage unit by the image determination unit 105 is, for example,
* Dictionary data for judging the tendency of the distribution of histograms generated based on images taken underwater,
* Dictionary data for judging the tendency of the distribution of histograms generated based on images of sunset scenes.
Dictionary data corresponding to each scene such as underwater, sky, evening scene, night scene, tree, snow, autumn leaves, etc. The image discriminating unit 105 compares the histogram generated by the feature amount calculating unit 104 with each dictionary data. The most similar dictionary data is selected and discriminated as the scene of the judgment target image.

  A specific example of dictionary data will be described with reference to FIG. FIG. 7A shows a histogram generated by the feature amount calculation unit 104, and examples of rectangular feature determination data (Rectangle Feature) 301 and 302 which are dictionary data used by the image determination unit 105 in scene determination. Show.

  As shown in FIG. 7B, the rectangular feature determination data (Rectangle Feature) is a line parallel to the [R / G] axis and the [B / G] axis, which are two-dimensional planes in which a histogram is generated. As shown in FIG. 7B, the rectangular shape is composed of two divided areas A and B. The areas of region A and region B are set to be equal.

  The size (defined by h and w) and the set position (defined by one vertex position (x, y)) of the rectangular feature determination data (Rectangle Feature) are underwater, sky, sunset, and night view. It may be different for each dictionary data corresponding to each scene such as trees, snow and autumn leaves. The number of rectangular feature determination data (Rectangle Feature) to be used may be different for each dictionary corresponding to each scene.

  On the histogram generated by the feature quantity calculation unit 104, rectangular feature determination data (Rectangle Feature) is set, and a difference value of the total number of frequencies (integrated values) in A and B is calculated. The set position of the rectangular feature determination data (Rectangle Feature) is a position defined in a scene-corresponding dictionary to be compared. In the dictionary corresponding to each scene, information such as the setting position, size, and type (whether the arrangement of AB is a parallel type or a column type) of rectangular shape feature determination data (Rectangle Feature) is defined.

  FIG. 8 shows a processing example in which rectangular shape feature determination data (Rectangle Feature) is set and the difference value of the total number of frequencies (integrated values) in A and B is calculated. FIG. 8 (1) shows a case where the arrangement of AB is a parallel type. The total number of frequencies (integrated value) in region B is subtracted from the total number of frequencies (integrated value) in region A to obtain a difference value (St ) Is calculated. Note that t is an identifier corresponding to the number of rectangular feature determination data (Rectangle Feature) registered in a dictionary corresponding to a scene. For example, when K pieces of rectangular feature determination data (Rectangle Feature) are registered in one scene-corresponding dictionary, the difference value St is calculated as S1 to Sk and K pieces in total.

  FIG. 8 (2) shows a case where the arrangement of AB is a tandem type. The total number of frequencies (integrated value) in region B is subtracted from the total number of frequencies (integrated value) in region A to obtain a difference value (St ) Is calculated.

  FIG. 9 shows a configuration example of a dictionary applied to determination of a certain scene (for example, an underwater scene). The dictionary data is composed of parameters (x, y, w, h, type, θ, α, p) corresponding to a plurality of rectangular feature determination data (Rectangle Feature). The example illustrated in FIG. 9 illustrates parameters corresponding to K rectangular shape feature determination data (Rectangle Feature).

Each parameter has the following meaning.
x: R / G axis coordinates of one vertex of the rectangular area y: B / G axis coordinates of one vertex of the rectangular area w: width of the rectangular area h: height of the rectangular area type: AB of the AB to be set in the rectangular area Whether the arrangement is a parallel type or a column type θ: a threshold value for discriminating a difference value (St) α: a weight for discriminating a scene p: a flag applied when scene discrimination data is calculated from a calculation result of a difference value (St), +1 or The value is set to -1.

  With reference to the flowchart shown in FIG. 10, a sequence of scene determination processing to which dictionary data is applied and image processing mode determination processing based on the determination result will be described. This process is executed as a process of the image discrimination unit 113 of the imaging apparatus 100 shown in FIG.

  First, in step S201, the color ratio histogram generated by the feature amount calculation unit 112 is input. This is a histogram having R / G and B / G planes and frequency information (height) described with reference to FIGS. Further, as initial settings, the initial setting of t = 0 and f = 0 is set as the initial setting of the identifier [t] of the rectangular feature determination data (Rectangle Feature) included in the dictionary data to be applied and the scene determination data [f]. Set the value.

  In step S202, a dictionary corresponding to a scene to be compared is acquired from the storage unit and corresponds to the t-th rectangular shape feature determination data (Rectangle Feature) from a plurality of rectangular shape feature determination data (Rectangle Feature) included in the dictionary. Get the parameters to perform. As described with reference to FIG. 9, the dictionary data is based on parameters (x, y, w, h, type, θ, α, p) corresponding to a plurality of rectangular feature determination data (Rectangle Feature). Composed.

  For example, when it is determined whether or not a captured image that has generated a histogram is an underwater scene, the processing is performed using dictionary data generated from learning data of the underwater scene. When determining whether or not the scene is an evening scene, processing is performed using dictionary data generated from learning data of the evening scene.

  In steps S203 to S204, t-th rectangular shape feature determination data (Rectangle Feature) of the scene correspondence dictionary to be determined is set in the histogram generated based on the photographed image, and the integration of the region A and the region B. That is, the total number of frequencies is calculated. Next, in step S205, the difference [St] between the integral values of region A and region B is calculated. This process corresponds to the process described with reference to FIG.

  Next, in step S206, the difference [St] calculated in step S205 is compared with the threshold [θt] registered in the dictionary. As described above, θ is a difference value (St) determination threshold.

  In the determination process of step S206, when St> θt is satisfied, the process proceeds to step S207, and when St> θt is not satisfied, the process proceeds to step S208.

In the determination process of step S206, when St> θt is established, the process proceeds to step S207, and in step S207,
ft = p × αt
By applying the above formula, scene discrimination data [ft] corresponding to one t-th rectangular feature determination data (Rectangle Feature) is calculated. In the above formula,
p is a flag applied when scene discrimination data is calculated from the difference value (St) calculation result, and is a value of (+1) or (−1).
αt is a scene discrimination weight corresponding to one rectangular feature determination data (Rectangle Feature).

On the other hand, if St> θt is not satisfied in the determination process in step S206, in step S208,
ft = −p × αt
By applying the above formula, scene discrimination data [ft] corresponding to one t-th rectangular feature determination data (Rectangle Feature) is calculated.

That is, as shown in FIG.
If St> θt holds, ft = p × αt,
If St> θt does not hold, ft = −p × αt,
In this way, scene discrimination data [ft] corresponding to one t-th rectangular feature determination data (Rectangle Feature) is calculated.

Next, in step S209,
The scene discrimination data [ft] corresponding to one t-th rectangular feature determination data (Rectangle Feature) calculated in step S207 or step S208 is added to the already calculated scene discrimination data [Σf].

  Next, in step S210, the identifier [t] of the rectangular shape feature determination data (Rectangle Feature) included in the dictionary applied to the scene determination is the total number of rectangular shape feature determination data (Rectangle Feature) included in the selected dictionary. It is judged whether it is less than. That is, it is determined whether there is unprocessed rectangular feature determination data (Rectangle Feature) in the dictionary being used.

  When unprocessed rectangular shape feature determination data (Rectangle Feature) exists in the dictionary used, the process returns to step S202, and the same processing is performed using unprocessed rectangular shape feature determination data (Rectangle Feature). Repeat the process. In this way, scene discrimination data [ft] corresponding to all rectangular shape feature determination data (Rectangle Feature) is calculated, and a total sum [Σf] of the total addition value f is calculated.

Next, in step S211, it is determined whether or not the total sum [Σf] of f is greater than zero.
When the total sum [Σf] of f is larger than 0, it is determined that the image in which the histogram is generated has a color distribution similar to the dictionary data applied to the scene determination. In this case, in step S212, it is determined that the image is a scene corresponding to the applied dictionary, and image processing specialized for the scene is executed. Image processing based on the determination result is executed by the image processing unit 105 of the imaging apparatus shown in FIG.

  On the other hand, if the total sum [Σf] of f is not larger than 0, it is determined that the image for which the histogram is generated does not have a color distribution similar to the dictionary data applied to the scene determination. In this case, in step S213, it is determined that the image is not a scene corresponding to the applied dictionary, and image processing applied to the general image is executed instead of image processing specialized for the scene. Image processing based on the determination result is executed by the image processing unit 105 of the imaging apparatus shown in FIG.

  As described above, when it is determined that the captured image for which the histogram is generated is a scene image corresponding to the dictionary applied to the scene determination process, image processing specialized for the scene is executed in step S212.

  Specifically, for example, when a dictionary applied to scene determination is a dictionary for underwater scenes, and it is determined that the captured image is an image of an underwater scene, the captured image may be imaged due to underwater light characteristics. The color is more bluish than it actually is. In this case, the process of removing the bluish color and returning it to a natural color is executed in step S212 as an image process specialized for the scene.

  As described above, the scene discrimination processing according to the present invention expresses the color distribution of a captured image by a histogram, and rectangular shape feature determination data (Rectangle Feature) which is dictionary data generated by learning image data corresponding to various scenes. Is applied to perform scene discrimination. With this configuration, it is possible to accurately determine the scene based on the colors included in the image. For example, more accurate scene discrimination can be realized than conventional scene discrimination based on brightness conditions.

  For example, it is impossible to determine whether or not the subject is underwater only by analysis based on brightness. Even if scene discrimination based on brightness is executed, specific image processing corresponding to an image taken underwater is executed. Such processing cannot be performed. Therefore, general image processing must be executed only by scene analysis based on brightness, and the resulting output image becomes an unnatural blue image.

  In contrast, by performing scene discrimination according to the present invention, various scenes (outdoor, sunset, night view, indoor, underwater, etc.) can be determined more accurately, and specialized for each scene. It is possible to execute optimal image correction processing such as optimal color correction, and to generate and output high-quality images corresponding to each scene.

  In addition, since the scene determination processing of the present invention is configured to execute scene determination using the entire color distribution information instead of the average value of the color feature amount, information used for scene determination increases dramatically, It is possible to improve scene discrimination accuracy. For example, if you try to distinguish between an image of a landscape where some blue object appears in the majority of the image and an image of a fish photographed underwater with the average value of the color, The average value is likely to be almost the same value, and scene discrimination becomes difficult. However, the method of the present invention is configured to perform discrimination using color distribution, and the problem in processing using such an average value is solved, and more accurate scene discrimination is realized.

  The processing of the present invention makes it possible to perform correction processing using a more accurate scene discrimination result. For example, an image that is actually blue in non-water is output in blue, and an unnatural blue image is suppressed in water. Thus, it is possible to perform correction processing according to the scene, and it is possible to generate and output a higher quality image.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run. For example, the program can be recorded in advance on a recording medium. In addition to being installed on a computer from a recording medium, the program can be received via a network such as a LAN (Local Area Network) or the Internet, and installed on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of the embodiment of the present invention, the pixel value ratio of each color in the block unit which is the divided region of the RAW image acquired by the image sensor or the reduced RAW image obtained by reducing the RAW image. , And a histogram that is the frequency distribution information of the calculated pixel value ratio is generated, and the generated histogram is compared with a scene discrimination dictionary stored in advance in the storage unit to obtain a RAW image shooting scene A discrimination process is executed, and an image process corresponding to the discrimination result is executed. Specifically, since the scene is discriminated by comparing the histogram and the parameters that define the rectangular feature determination data (Rectangle Feature) prepared for various scenes, brightness and color Compared to analysis using average values, various scenes (outdoor, sunset, night view, indoor, underwater, etc.) can be determined more accurately, and optimal color correction specialized for each scene, etc. Image correction processing can be executed.

It is a figure explaining the structure of the imaging device which concerns on one Example of this invention. It is a figure explaining the example of a RAW image with the data arrangement | sequence of R, G, B. It is a figure explaining the specific example of the process which the image reduction part in the imaging device which concerns on one Example of this invention performs. It is a figure explaining an example of the histogram on the two-dimensional plane which makes R / G the vertical axis | shaft which the feature-value calculation part in the imaging device which concerns on one Example of this invention produces, and makes a horizontal axis B / G. Processing example of acquiring a block from a reduced RAW image, calculating an image feature amount, R / G, B / G, which is a color ratio, from RGB pixel values in the block, and generating an image feature histogram based on the calculation result It is a figure explaining about. It is a figure which shows the flowchart explaining the specific histogram generation sequence in the imaging device which concerns on one Example of this invention. It is a figure explaining the structural example of the dictionary data utilized by the image discrimination | determination part in the imaging device which concerns on one Example of this invention. It is a figure explaining the process example which calculates the difference value of the frequency number total number (integral value) using the rectangular shape characteristic determination data (Rectangle Feature) which the image discrimination | determination part which the imaging device which concerns on one Example of this invention performs. . It is a figure which shows the structural example of the dictionary applied to determination of a certain scene (for example, underwater scene). It is a figure which shows the flowchart explaining the sequence of the scene discrimination | determination process to which dictionary data is applied, and the determination process of the image processing mode based on a discrimination | determination result. It is a figure explaining the specific example of the scene discrimination | determination process to which dictionary data is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 101 Lens 102 Image pick-up element 103 Digital signal conversion part 105 Image processing part 110 Scene discrimination | determination part 111 Image reduction part 112 Feature-value calculation part 113 Image discrimination | determination part

Claims (9)

An imaging device that executes imaging processing and outputs a RAW image;
A RAW image acquired by the image sensor or a reduced RAW image obtained by reducing the RAW image is input, a pixel value ratio of each color is calculated in units of blocks that are segmented regions of the input RAW image, and the calculated pixel value ratio A feature amount calculation unit that generates a histogram which is frequency distribution information;
An image discriminating unit that performs a comparison between a histogram calculated by the feature amount calculating unit and a scene discriminating dictionary stored in a storage unit, and executes a shooting scene discriminating process of the RAW image;
An image processing unit that executes image processing according to a scene determination result of the image determination unit;
An imaging apparatus having The feature amount calculation unit includes:
Ratios R / G and B / G of RGB pixel values are calculated in units of blocks constituting the input RAW image, and a plurality of R / G and B / G values corresponding to all the constituent blocks of the input RAW image are calculated. The imaging apparatus according to claim 1, wherein the imaging apparatus is configured to generate a histogram having an appearance frequency as a frequency distribution. The imaging device
An image reduction unit that generates a reduced RAW image by performing a reduction process of the RAW image output from the image sensor;
The imaging apparatus according to claim 1, wherein the feature amount calculation unit is configured to generate the histogram by inputting the reduced RAW image generated by the image reduction unit. The storage unit stores, as a scene discrimination dictionary corresponding to a plurality of different scenes, parameters defining rectangular shape feature determination data (Rectangle Feature) for determining the frequency distribution tendency of the histogram,
The image discrimination unit
A comparison process between the histogram calculated by the feature amount calculation unit and the parameter stored in the storage unit is executed, and a parameter-corresponding scene having a similarity higher than a specified similarity with the histogram is a scene of the RAW image The imaging apparatus according to claim 1, wherein the imaging apparatus is configured to determine that The storage unit
The imaging apparatus according to claim 4, wherein a result of calculating a parameter that defines the rectangular shape feature determination data (Rectangle Feature) using learning data based on an image corresponding to a specific scene is stored. The image processing unit
The imaging apparatus according to claim 1, wherein the imaging apparatus is configured to execute color correction processing specialized for a scene discrimination result of the image discrimination unit. A RAW image acquired by the image sensor or a reduced RAW image obtained by reducing the RAW image is input, a pixel value ratio of each color is calculated for each block which is a segmented area of the input RAW image, and the frequency of the calculated pixel value ratio A feature amount calculation unit for generating a histogram as distribution information;
An image discriminating unit that performs a comparison between a histogram calculated by the feature amount calculating unit and a scene discriminating dictionary stored in a storage unit, and executes a shooting scene discriminating process of the RAW image;
An image processing unit that executes image processing according to a scene determination result of the image determination unit;
An image processing apparatus. A feature amount calculation unit inputs a RAW image acquired by an image sensor or a reduced RAW image obtained by reducing the RAW image, calculates a pixel value ratio of each color in units of blocks that are divided regions of the input RAW image, and calculates A feature amount calculating step for generating a histogram which is frequency distribution information of the pixel value ratio,
An image discriminating step, wherein the image discriminating unit performs comparison between the histogram calculated in the feature amount calculating step and the scene discriminating dictionary stored in the storage unit, and executes the shooting scene discriminating process of the RAW image;
An image processing unit that performs image processing according to a scene determination result in the image determination step;
An image processing method. A program for causing an image processing apparatus to execute image processing,
A RAW image acquired by the image sensor or a reduced RAW image obtained by reducing the RAW image is input to the feature amount calculation unit, and a pixel value ratio of each color is calculated for each block that is a divided region of the input RAW image. A feature amount calculating step for generating a histogram which is frequency distribution information of the pixel value ratio,
An image discriminating step for causing the image discriminating unit to perform a comparison between the histogram calculated in the feature amount calculating step and the scene discriminating dictionary stored in the storage unit, and to execute a shooting scene discriminating process of the RAW image;
An image processing step for causing the image processing unit to execute image processing according to a scene determination result in the image determination step;
A program with