JP2005062964A - Image processing system, image detection processing method, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce the load related to processing of image detection, and allow detection processing of an image for attention, even in a device of low processing capacity such as a direct photoprinter. <P>SOLUTION: The system judges whether or not chromaticity information and luminance information, obtained for each of partial pixel areas that are compression units of compressed image data is in a range that matches a prescribed definition of the target image (S402), groups the partial pixel areas next to each other of the partial pixel areas decided as being in the range that matches the prescribed definition of the target image (S404), calculates the spatial frequency characteristics of each group (S405), decides whether the calculated spacial frequency characteristics is in a prescribed proper range of the spacial frequency characteristics of the image for attention for each of the groups (S406), and certifies the group that belongs to the target image, according to the result of the decision (S407). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, an image detection processing method, a program, and a recording medium that can decode compressed image data by orthogonal transform processing.
[0002]
[Prior art]
Conventionally, as a method for detecting whether or not an image of interest exists in captured image data stored in the Jpeg file format, as a method for detecting a blue sky region, the entire image is decomposed for each specific region set, A method is known in which the spatial frequency characteristics for each region are calculated, and the determination is made based on the result and hue. Also, Patent Document 1 discloses a technique for detecting sky.
[0003]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2001-195591
[0004]
[Problems to be solved by the invention]
However, the above-described conventional method requires enormous calculation processing capability by calculating the spatial frequency for each specific region of the entire image. Recently, a digital camera having about 4 million pixels that has been widely used has recently been widely used, and even if processing is performed on a personal computer for image data having a large image size output from such a digital camera, there is a large time stress. Will give. In addition, it is practically difficult to execute the above-described processing in a built-in device such as a direct photo printer that directly incorporates and prints a storage medium of a digital camera that has recently been released.
[0005]
Accordingly, an object of the present invention is to reduce the load related to the image detection process, and to enable the target image detection process to be executed even by an apparatus having a low processing capability such as a direct photo printer.
[0006]
[Means for Solving the Problems]
In order to achieve such an object, the image processing apparatus of the present invention is an image processing apparatus capable of decoding compressed image data by orthogonal transform processing, and each of the partial pixel areas which are compression units of the compressed image data. A determination unit that determines whether or not the acquired chromaticity information and luminance information match a range that matches a definition of the target image that is set in advance, and the determination unit determines whether the target image Among the partial pixel regions determined to be in conformity with the definition, a group generation unit that groups adjacent pixel regions and a spatial frequency characteristic of each group generated by the group generation unit are calculated. The calculation means and the spatial frequency characteristic calculated by the calculation means are adapted to an appropriate range of the spatial frequency characteristics of the image of interest set in advance. The whether determined for people each group respectively, and having a certification means for certifying groups that belong to the target image in accordance with the determination result.
[0007]
The image processing apparatus of the present invention is an image processing apparatus capable of decoding compressed image data by orthogonal transform processing, and includes chromaticity information acquired for each of the partial pixel areas that are compression units of the compressed image data, and A first determination unit that determines, for each of the partial pixel areas, whether the luminance information matches a range that matches a predetermined definition of the target image, and the definition of the target image by the first determination unit Among the partial pixel regions determined to be in conformity with the range matching with the group generation means for grouping adjacent partial pixel areas, and the spatial frequency characteristics of each of the first group groups generated by the group generation means And a suitable range of the spatial frequency characteristics of the image of interest in which the spatial frequency characteristics calculated by the first calculation means are preset. 2nd determining means for determining whether each of the first group groups is suitable, and appropriate spatial frequency characteristics of the image of interest whose spatial frequency characteristics are preset by the second determining means A second calculation means for calculating an area ratio of all the second group groups determined to be compatible with the range, and whether the area ratio calculated by the second calculation means exceeds a predetermined area ratio; It is characterized by determining whether each of the second group group determines whether or not and certifying means for certifying a group belonging to the image of interest according to the determination result.
[0008]
The image processing apparatus of the present invention is an image processing apparatus capable of decoding compressed image data by orthogonal transform processing, and includes chromaticity information acquired for each of the partial pixel areas that are compression units of the compressed image data, and A first determination unit that determines, for each of the partial pixel areas, whether the luminance information matches a range that matches a predetermined definition of the target image, and the definition of the target image by the first determination unit Among the partial pixel areas determined to match the range that matches the group generation means for grouping the adjacent partial pixel areas with each other and the first group group generated by the group generation means First calculation means for calculating the area ratio occupied and whether or not the area ratio occupied by all the images exceeds a predetermined area ratio are determined for each of the first group groups. And a portion belonging to the outermost group of each of the second group group determined by the second determination unit and the second group group determined by the second determination unit that the area ratio of the entire image exceeds the predetermined area ratio A second calculating means for calculating a spatial frequency characteristic of an area excluding the pixel area, and the spatial frequency characteristic calculated by the second calculating means is adapted to an appropriate range of the spatial frequency characteristic of the target image set in advance; Whether or not each of the second group group is determined, and a recognition unit that recognizes a group belonging to the target image according to the determination result.
[0009]
The image detection processing method of the present invention is an image detection processing method by an image processing device capable of decoding compressed image data by orthogonal transform processing, wherein the image processing device is a unit of compression of the compressed image data. Whether or not the chromaticity information and luminance information acquired for each of the pixel areas matches a range that matches the definition of the target image set in advance is determined for each of the partial pixel areas, and the target image is defined Among the partial pixel areas determined to match the matching range, adjacent partial pixel areas are grouped together, and the spatial frequency characteristics of each group generated by the grouping are calculated, and the calculated spatial frequency characteristics are preset. Each group is determined whether or not it fits within an appropriate range of the spatial frequency characteristics of the target image to be processed, and the determination result is Characterized in that it authorized the group belonging to the target image Te.
[0010]
The image detection processing method of the present invention is an image detection processing method by an image processing device capable of decoding compressed image data by orthogonal transform processing, wherein the image processing device is a unit of compression of the compressed image data. Whether or not the chromaticity information and luminance information acquired for each of the pixel areas matches a range that matches the definition of the target image set in advance is determined for each of the partial pixel areas, and the target image is defined Among the partial pixel regions determined to match the matching range, the adjacent partial pixel regions are grouped, and the spatial frequency characteristics of the first group group generated by the grouping are calculated, and the calculated spatial frequency It is determined for each of the first group groups whether or not the characteristics are suitable for an appropriate range of the spatial frequency characteristics of the image of interest set in advance. The area ratio of the second group group that has been determined to have a spatial frequency characteristic suitable for the preset appropriate range of the spatial frequency characteristic of the image of interest is calculated, and the calculated area ratio is a predetermined area. Whether or not the ratio exceeds the ratio is determined for each of the second group groups, and the group belonging to the target image is identified according to the determination result.
[0011]
The image detection processing method of the present invention is an image detection processing method by an image processing device capable of decoding compressed image data by orthogonal transform processing, wherein the image processing device is a unit of compression of the compressed image data. Whether or not the chromaticity information and luminance information acquired for each of the pixel areas matches a range that matches the definition of the target image set in advance is determined for each of the partial pixel areas, and the target image is defined Among the partial pixel areas determined to match the matching range, adjacent partial pixel areas are grouped together, and the area ratio of the first group group generated by the grouping to the total image is calculated, and It is determined for each of the first group groups whether the area ratio of all images exceeds a predetermined area ratio, and the area ratio of all images is For each of the second group groups determined to exceed the predetermined area ratio, the spatial frequency characteristic of the area excluding the partial pixel area belonging to the outermost outline of the group is calculated, and the calculated spatial frequency characteristic is preset. It is determined for each of the second group groups whether or not the appropriate range of the spatial frequency characteristics of the target image is satisfied, and a group belonging to the target image is identified according to the determination result.
[0012]
A program according to the present invention causes a computer to execute the image detection processing method. Furthermore, the computer-readable recording medium of the present invention is characterized in that the program is recorded.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
When printing image data in JPEG File format, which is a data compression recording format of a general digital camera, before restoring to uncompressed data, luminance and chromaticity based DCT (discrete) for each block (8 * 8 pixels) Cosine transformed data is also acquired, and the presence and area of a blue sky that can be targeted are detected based on the spatial frequency characteristics, the arrangement of blocks, and the ratio of the blocks to the image. By doing this, it is possible to acquire AC component information and the like for each image data block without performing advanced calculations, and to detect a blue sky region in the image using the characteristics.
[0014]
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.
<First Embodiment>
Now, description will be given of information omission and encoding / decoding of “Jpeg file” of the most common image compression file.
First, encoding is generally performed in digital cameras, digital videos, and the like, in which still images are stored as JPEG files. In this case, a signal containing a CCD, which is a light receiving element of the input device, is A / D converted and then taken into a frame memory to convert RGB or CMY filter information into luminance and chromaticity information. Then, it is divided into 8 * 8 (64) square pixel blocks.
[0015]
(1) in FIG. 3 is a diagram showing an example in which the luminance data bitmap is divided into 8 * 8 blocks. In {circle around (2)}, the pixel value of 0 to 255 is level-shifted and converted to a signal of −128 to 127. In (3), a DCT coefficient is obtained by DCT (discrete cosine transform). (4) is a diagram showing a quantization table in which omission of high-frequency components in consideration of visual characteristics is increased. Using this table, quantization is performed on the DCT coefficient resulting from (3) above. (5) is the result of quantization. This value is entropy encoded and expressed by a Huffman code to generate compressed data that is an encoded signal.
[0016]
Next, in decoding, the reverse process of encoding is performed.
That is, the encoded signal is decoded and decoded into a quantized DCT coefficient value. For this purpose, the DCT coefficient is obtained by multiplying the encoded signal by each corresponding value in the quantization table. Thereafter, the image subjected to the level shift is restored by performing inverse DCT, and the image of each block is decoded by adding the value 128 of the inverse level shift.
[0017]
In the above description, it is omitted to combine the data divided into luminance information and chromaticity information and convert it into an RGB image. However, as shown in FIG. 2, the color image is converted into a luminance component (Y) as shown in FIG. The image is converted into two chromaticity components (Cb, Cr), and each of them is encoded and combined to generate a compressed image.
[0018]
As a method for printing a JPEG image as a compressed image data file as described above, the image data from the input device is loaded into a PC by USB or storage medium, the image is developed, and image correction is performed as necessary. Various types, such as sending data to the printer after adding it, or inputting the image data from the input device directly into the printer, decompressing the image in the printer, performing image correction as necessary, and printing There are options.
[0019]
In any case, in order to print a good image, it is judged whether the photographed image data is a good photographed image or an image that needs to be corrected, and a good image to be printed faithfully and a good quality by performing correction. There is a need to separate what is printed after getting close to the correct image.
[0020]
The following can be considered as a good image.
1) White balance is good.
2) The contrast is appropriate.
3) The tone of the necessary part is assigned. That is, the exposure setting is good.
4) The saturation is appropriate.
5) The finish looks like a silver halide photograph.
6) The image of interest such as a person is corrected mainly.
[0021]
Even in commercially available PC printers and direct printers that do not go through a PC, the items 1) to 5) are performed to some extent. However, the method often uses the same processing constant for all images based on the definition that has been set, and there is no degree of freedom to change the constants specific to each image scene. The effect could not be maximized.
[0022]
In addition, the correction of the image of interest in 6) is not performed because a large amount of processing is necessary for the detection and the method has not been established. However, the present embodiment solves this problem.
[0023]
As the target of the target image, human skin, blue sky, and the like can be candidates for the present invention, but only the case of detecting the blue sky region will be described below, but it is also possible to detect human skin similarly. is there.
[0024]
As a means for reflecting the correction, a method of passing to the whole image correction is performed after detecting the presence of the image of interest from the Jpeg image file and confirming the necessity of correction for the detected image.
[0025]
FIG. 1 is a block diagram showing a process of decompressing a Jpeg file and information acquired at that time.
In the process of converting a Jpeg file into RGB bitmap data, the entropy decoding unit 12 first performs entropy decoding using the code table 11, and then the inverse quantization unit 14 performs inverse using the quantization table 13. After quantization, it is stored as data. The inversely quantized data is frequency-converted as block unit data, and this data is acquired as data for obtaining image frequency characteristics. Thereafter, the inverse DCT unit 15 performs inverse DCT processing and inverse level shift to perform Ycc-RGB conversion, thereby developing normal RGB bitmap data.
[0026]
FIG. 4 is a flowchart showing the flow of the blue sky area existence determination process at a level effective for performing image correction in the captured image data, and the target area designation process when the blue sky area exists.
[0027]
A detection process will be described assuming that an image sample 1 as shown in FIG. 11 is taken as photographed image data. Image sample 1 was taken with a 2 million pixel digital camera and is an image of UXGA (1600 * 1200) size.
[0028]
In step S401, DCT characteristic data in units of 8 * 8 blocks with respect to the entire image file area is acquired in the decompression process of the JPEG file described with reference to FIG.
[0029]
In S402, chromaticity information and luminance information for each 8 * 8 block, which is an image compression unit, are acquired for the entire area of the image file, and the chromaticity or chromaticity matching the preset definition of the target image is acquired. It is determined for each 8 * 8 block whether or not the ratio range and the luminance range are matched, and the matched block is stored.
[0030]
It is conceivable to use one of the following methods as a method for acquiring and generating chromaticity information. A more effective method can be selected depending on the characteristics of the image of interest.
[0031]
FIG. 8 illustrates the chromaticity detection points used in this embodiment. The determination is based on the average chromaticity of the entire 8 * 8 block. As a method for obtaining the average chromaticity in this block, in addition to the method of taking the average value of the pixel values of all 8 * 8 blocks, the chromaticity data (Cb, Cr) before performing the inverse DCT during decompression is included. It is also possible to obtain from the DC component. As an advantage of this method, since the determination can be made based on the color tone of the entire block, higher accuracy can be expected compared with the case where there are few detection points.
[0032]
FIG. 7 shows whether the chromaticity of the pixels at the four corners of the 8 * 8 block is all within the chromaticity range, and if all are within the range, the block is determined to be the appropriate chromaticity. Yes. In FIG. 7, the second block from the left in the upper row and the 1, 2, and 3 blocks from the left row in the lower row correspond. In the upper leftmost block, the chromaticity at the upper left of the four points is determined to be a non-skin color pixel, and therefore a block including this is determined to be outside the skin color range. Similarly, the upper right 1 block and the lower right 2 block are out of range. The example in FIG. 9 has the same idea as the example in FIG. 7, but is for equalizing the detection intervals in the entire image.
[0033]
The relationship between the luminance component and the chromaticity component will be described.
FIG. 23 is a graph showing the chromaticity ratio of the blue sky region in the image data including the blue sky region of various scenes with the digital camera.
The horizontal axis is the chromaticity ratio of red and is calculated by the following formula F.
Red chromaticity ratio = R / (R + G + B) (Formula F)
The vertical axis represents the chromaticity ratio of green and is calculated by the following formula G.
Green chromaticity ratio = G / (R + G + B) (Formula G)
[0034]
It can be seen that this alone can result in a fairly wide chromaticity ratio. In each of these data, the following diagram shows the luminance range divided. FIG. 24 is a graph showing only the luminance range within the range of 0 to 139 in 256 steps. FIG. 25 is a graph showing only the luminance range within the range of 140 to 179 in 256 stages. FIG. 26 is a graph showing only the luminance range within the range of 180 to 199 in 256 steps. FIG. 27 is a graph showing only the luminance range within the range of 200 to 155 in 256 stages.
[0035]
It can be seen from the distributions shown in FIGS. 24 to 27 that the chromaticity ratio range is narrowed by limiting the luminance range. By using this characteristic, it is possible to further increase the detection accuracy of the target image, rather than targeting the luminance-dependent chromaticity ratio range.
[0036]
In this embodiment, the range specified in FIG. 28 is used as an appropriate setting range of the luminance and chromaticity ratio range calculated from this characteristic. However, even if a chromaticity ratio range that does not depend on luminance is set, the accuracy is reduced to some extent, but there is no problem.
[0037]
In S403, it is determined whether or not there is a matching block stored in S402. Here, when there is no matching block, it is determined that there is no image of interest in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a matching block, the process proceeds to the next step.
[0038]
FIG. 12 shows the result of the determination performed on the image in FIG. A portion determined to be a conforming block region and a portion determined to be a non-conforming block region are shown in different colors.
[0039]
In S404, it is determined whether or not all conforming blocks are adjacent to the corresponding block in the eight directions (vertical, left and right, diagonal). If there are adjacent blocks, further search is performed for the adjacent blocks of the block. And form a group. In the formed group, group numbers are assigned in descending order of the number of formed blocks. The largest one (the one with the largest number of blocks included) is group 1.
[0040]
As a group in FIG. 12, a total of three groups, that is, a group 1 that occupies almost the entire area at the top of the image and two upper and lower groups on the upper right of the image divided by the telephone pole, are formed as candidate groups.
[0041]
In S405, the average value of the DCT data of each block acquired in S401 is calculated for each group. As a method for obtaining the DCT characteristic, the maximum spatial frequency information can be obtained by calculating each of the 63 AC components of the DCT characteristic in Jpeg. In the embodiment, with respect to 63 frequency components, 6 characteristics and the remaining 3 (61 to 63) characteristics are calculated for 10 frequency components from the lowest frequency component, and 63 characteristics are calculated. Judgment data is created with the frequency component as the frequency component characteristics of seven groups.
[0042]
In S406, it is determined whether the spatial frequency characteristics of each candidate group calculated in step S405 are suitable for the appropriate range of the spatial frequency characteristics for each target image set in advance. Here, when there is no suitable candidate group, it is determined that the image of interest does not exist in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a suitable candidate group, the process proceeds to the next step.
[0043]
The appropriate range of the spatial frequency characteristics for each image of interest that has been set in advance, since the frequency characteristics vary depending on the image size, the characteristics should be set for each size, and are set arithmetically with respect to the reference image May be used.
[0044]
In S407, the candidate group matched in S406 is recognized as the target image, and various information including the area information is set, and the process ends. Here, examples of connecting the detection of the blue sky region in the image to high-quality image correction include the following.
1) Image correction reflecting the characteristics of the blue sky region
2) Up / down judgment by analyzing and utilizing the characteristics of the blue sky region
[0045]
<Second Embodiment>
FIG. 5 shows determination control for determining the presence of a blue sky region at a level effective for performing image correction in the captured image data and specifying the target region when it exists in the same configuration as the first embodiment. It is a thing.
[0046]
FIG. 5 will be described. In step S501, DCT characteristic data in units of 8 * 8 blocks for the entire image file area is acquired in the process of decompressing the Jpeg file described with reference to FIG.
[0047]
In S502, chromaticity information and luminance information for each 8 * 8 block, which is an image compression unit, are acquired for the entire area of the image file, and the chromaticity or chromaticity that matches the preset definition of the target image is acquired. It is determined for each 8 * 8 block whether it matches the ratio range and the luminance range, and the matching block is stored.
[0048]
In S503, it is determined whether or not there is a matching block stored in S502. Here, when there is no matching block, it is determined that there is no image of interest in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a matching block, the process proceeds to the next step.
[0049]
In S504, it is determined in all the matching blocks whether the matching block is adjacent in the eight directions (up, down, left, right, diagonal). If there is an adjacent block, the adjacent block of the block is further searched. And form a group. In the formed group, group numbers are assigned in descending order of the number of formed blocks. The largest one (the one with the largest number of blocks included) is group 1.
[0050]
In S505, the average value of the DCT data of each block acquired in S501 is calculated for each group. As a method for obtaining the DCT characteristic, the maximum spatial frequency information can be obtained by calculating each of the 63 AC components of the DCT characteristic in Jpeg. In the embodiment, for the 63 frequency components, 10 characteristics from the lowest frequency component are collectively calculated and 6 characteristics and the remaining 3 (61 to 63) characteristics are calculated. Judgment data is created with the frequency component as the frequency component characteristics of seven groups.
[0051]
In S506, it is determined whether or not the spatial frequency characteristics of each candidate group calculated in step S505 are suitable for the appropriate range of the spatial frequency characteristics for each target image set in advance. Here, when there is no suitable candidate group, it is determined that the image of interest does not exist in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a suitable candidate group, the process proceeds to the next step.
[0052]
The appropriate range of the spatial frequency characteristics for each image of interest that has been set in advance, since the frequency characteristics vary depending on the image size, the characteristics should be set for each size, and are set arithmetically with respect to the reference image May be used.
[0053]
In S507, the area ratio of all the images of the candidate group determined to be suitable in S506 is calculated. In step S508, it is determined whether the area ratio of each candidate group in the entire image is larger than a preset authorized reference area ratio. If the area ratio is smaller, it is determined that the candidate group is too small as a target image. Each candidate group is sequentially determined from candidate group 1 having a larger size. When the candidate group 1 does not satisfy the determination criterion, it is determined that the image of interest does not exist in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a suitable candidate group, the process proceeds to the next step.
[0054]
In S509, the candidate group matched in S508 is recognized as the target image, and various information including the area information is set, and the process is terminated.
[0055]
Although this embodiment requires more processing than the first embodiment, the determination of the area ratio in the entire image provides a blue sky or false detection that has only a small area that has little effect when viewed from the entire image. The performance to prevent is improved.
[0056]
<Third Embodiment>
FIG. 6 shows determination control for determining the presence of a blue sky region at a level effective for performing image correction in the captured image data and specifying the target region when it exists in the same configuration as in the first embodiment. It is a thing.
[0057]
FIG. 6 will be described. In step S601, DCT characteristic data in units of 8 * 8 blocks with respect to the entire area of the image file is acquired in the JPEG file decompression process described with reference to FIG.
[0058]
In S602, chromaticity information and luminance information for each 8 * 8 block, which is an image compression unit, are acquired for the entire area of the image file, and the chromaticity or chromaticity matching the preset definition of the target image is acquired. It is determined for each 8 * 8 block whether it matches the ratio range and the luminance range, and the matching block is stored.
[0059]
In S603, it is determined whether or not there is a matching block stored in S602. Here, when there is no matching block, it is determined that there is no image of interest in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a matching block, the process proceeds to the next step.
[0060]
In S604, it is determined whether or not the matching blocks are adjacent in all the matching blocks in the eight directions (up, down, left and right, diagonal). If there are adjacent blocks, further search is performed for the neighboring blocks of the block. And form a group. In the formed group, group numbers are assigned in descending order of the number of formed blocks. The largest one (the one with the largest number of blocks included) is group 1.
[0061]
In S605, the area ratio of the candidate group determined to be suitable in S604 with respect to all images is calculated. In step S606, it is determined whether the area ratio of each candidate group in the entire image is larger than a preset authorized reference area ratio. If the area ratio is smaller, it is determined that the candidate group is too small as a target image. If the candidate group 1 is determined from a larger candidate group 1 and the candidate group 1 does not satisfy the determination criterion, it is determined that no image of interest exists in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. To do. If there is a suitable candidate group, the process proceeds to the next step.
[0062]
In S607, the average value of the spatial frequency component characteristics is calculated from the 8 * 8 blocks in the outermost portion of the candidate group for which the area ratio has been calculated. This is because when the DCT characteristic is determined, the spatial frequency characteristic of the blue sky may not contain much high-frequency components, but the 8 * 8 block in the outermost part of the detected candidate group has a boundary with other regions. Therefore, when calculating the spatial frequency component in the candidate group, there is a possibility that it contains a lot of high-frequency components of the spatial frequency component. By removing the 8 * 8 block from the calculation target of the spatial frequency characteristics, it becomes possible to detect the spatial frequency characteristics of the target image to be detected with higher accuracy.
[0063]
FIG. 13 shows the image sample 1 shown in FIG. A portion 1301 in FIG. 13 is an 8 * 8 block of the outermost portion of the candidate group 1. 13 is the outermost part of the candidate group 2, and the lower part 1303 is the outermost part of the candidate group 3.
[0064]
In S608, a determination is made as to whether or not the spatial frequency characteristics of each candidate group calculated in step S607 are suitable for the appropriate range of the spatial frequency characteristics for each target image set in advance. Here, when there is no suitable candidate group, it is determined that there is no image of interest in the image, and determination information indicating that there is no image of interest is set, and the detection process ends. If there is a suitable candidate group, the process proceeds to the next step.
[0065]
The appropriate range of the spatial frequency characteristics for each image of interest that has been set in advance, since the frequency characteristics vary depending on the image size, the characteristics should be set for each size, and are set arithmetically with respect to the reference image May be used.
[0066]
In step S609, the candidate group matched in step S608 is recognized as the target image, and various information including the area information is set, and the process ends.
[0067]
<Fourth Embodiment>
If various information including the presence / absence of a blue sky region and designation of the region can be extracted from the photographed image file by the method of the first to third embodiments, more advanced information is obtained from the information. be able to. Specifically, a method for obtaining the top / bottom determination will be described.
[0068]
An image sample 2 for explaining the present embodiment is shown in FIG. Image sample 2 was taken with a 1.3 million pixel digital camera and is an SXGA (1280 * 1024) size image.
[0069]
FIG. 18 shows a block distribution as a result of detecting this image with the matching chromaticity according to the third embodiment. As a group, there are a candidate group 1 having the largest area including the central portion and a candidate group 2 in a portion surrounded by a barbed wire near the right center of the image.
[0070]
FIG. 14 shows the final determination of the spatial frequency characteristics. Candidate group 2 was not recognized as a blue sky because the high-frequency component of the spatial frequency was increased by the net part of the barbed wire and was out of the adaptation range defined as the blue sky. Therefore, the candidate group 1 is recognized as a blue sky.
[0071]
The above-mentioned detection area is obtained by slicing the space from the sky to the vicinity of the ground into 28, classifying from “Top1” to “Top28” from the sky, and extracting the luminance and RGB values (8 bits each). 16 shows.
[0072]
The horizontal axis of the graph is divided into 28 steps from the upper left “Top1” to the right “Top28” near the ground, and the vertical axis has a scale of 256 gradations expressed in 8 bits.
[0073]
As the transition of each value of luminance and RGB, the blue saturation in the sky is high, and it can be seen that the ratio of red increases relatively toward the ground and the hue changes from blue to white.
[0074]
FIG. 17 is a graph showing the results of analyzing the chromaticity distribution in the blue sky region of the same blue sky scene taken by digital cameras of five companies (A, B, C, D, E). The horizontal axis is divided into 5 steps from the top left “Top1” to the right “Top5” near the ground. The vertical axis divides the red and green chromaticity ratio and luminance value (0 to 255) by 1000. Represents what you did. Even in this case, the same tendency as the result of FIG. 16 can be seen. Therefore, the up / down determination can be performed by using the chromaticity ratio or by looking at the transition of red or the like. The same up / down determination can also be executed based on the change in the chromaticity change. Such transition tendency of the chromaticity distribution is almost universal, and it is possible to determine the upper and lower sides on the ground by analyzing the chromaticity distribution in the blue sky region.
[0075]
A chromaticity distribution analysis method will be described with reference to FIG. 14 showing a result of detecting a blue sky region from the image of FIG.
Chromaticity distribution analysis is performed in the horizontal direction (H direction) and vertical direction (V direction) of the original image when the Jpeg image is developed. In many cases, photographing with a digital camera is performed in a portrait or landscape orientation, and the processing becomes complicated if analysis in an oblique direction is performed. In this embodiment, the two directions shown in FIG. Perform a distribution analysis. However, more analyzes than two directions may be performed in order to increase accuracy.
[0076]
The analysis method in the two directions is shown in the flowchart of FIG. In S1001, a line in which the maximum number of blocks in the candidate group 1 are arranged in the horizontal direction is detected. As S1002, the continuous blocks of the maximum line in the horizontal direction (H direction) detected in S1001 are grouped into five groups in the horizontal direction, and G1 (H1), G2 (H2), G3 (H3) from the reference side (left side). , G4 (H4), G5 (H5), the average value of the red values in the group is calculated.
[0077]
In FIG. 14, a continuous line of 140 blocks was detected at a maximum of 160 blocks. In order to make this into five groups, a group of 28 consecutive in the horizontal direction is created, and an average value of red values in the group is obtained.
[0078]
In S1003, a line in which the maximum number of blocks in the candidate group 1 are arranged in the vertical direction is detected. In S1004, the continuous blocks of the maximum lines in the vertical direction (V direction) detected in S1003 are grouped into five groups in the horizontal direction, and G1 (V1), G2 (V2), G3 (V3) from the reference side (left side). , G4 (V4), G5 (V5), the average value of the red values in the group is calculated.
[0079]
In FIG. 14, a continuous line of 85 blocks was detected for a maximum of 128 blocks. In order to make this into 5 groups, 17 groups are created in the vertical direction, and the average value of red values in the group is obtained.
[0080]
FIG. 19 is a graph of the calculated values of the grouped items. The horizontal axis is group 1 (G1) from the left, and group 5 (G5) is on the right. The vertical axis is the average value of red in the group. The horizontal (H) and vertical (V) are shown.
[0081]
As S1005, the absolute value of the difference between “G1 (H1)” and “G5 (H5)” in the horizontal direction is calculated and stored as ΔH, as shown in the following (formula A).
| H1-H5 | = ΔH (formula A)
[0082]
As S1006, the absolute value of the difference between “G1 (V1)” and “G5 (V5)” in the horizontal direction is calculated and stored as ΔV as shown below (Formula B).
| V1-V5 | = ΔV (Formula B)
[0083]
FIG. 21 is a flowchart showing the chromaticity distribution analysis process in the horizontal direction. As S2101, it is calculated whether the condition 2 of the following (formula C) is met.
H1 ≦ H2 <H3 <H4 or H2>H3> H4 ≧ H5 (Formula C)
[0084]
In S2102, it is determined whether the condition 2 of S2102 is satisfied. If the condition 2 is not satisfied, it is determined that there is no change in chromaticity distribution in the image search direction. In S2109, the horizontal determination information “Hset” is set to 0, and the detection process ends. If the condition 2 is satisfied, the process proceeds to the next.
[0085]
As S2103, the following (formula D) is calculated. The difference between the values of group 5 (H5) and group 1 (H1) where the chromaticity difference is the largest is obtained.
ΔH ′ = H5−H1 (formula D)
[0086]
As S2104, a positive or negative sign is determined as a result of S2103. If the determination is positive, “H1” has a smaller red value, and it can be seen that group 1 may be empty. On the other hand, if the determination is negative, “H5” has a smaller red value, and thus it can be seen that group 5 may be in the sky.
[0087]
In step S2105, “H1” is set to Htop, which is the temporary designation in the horizontal direction based on the above result. Thereafter, the process proceeds to S2107. As S2106, “H5” is set to Htop which is the temporary designation in the horizontal direction from the above result. Thereafter, the process proceeds to S2107.
[0088]
As S2107, the absolute value of the difference between the values of group 5 (H5) and group 1 (H1) where the largest chromaticity difference is calculated is read in step S1005.
[0089]
In S2108, it is determined whether the value of “ΔH” is 30 or more. This is to determine whether or not the absolute amount of the change in value has a difference greater than or equal to a set value serving as a determination criterion for up / down determination. Therefore, when the reference value is exceeded, it is determined that there is a chromaticity distribution difference in which the up / down determination is permitted. Here, if Yes, the process proceeds to S2110, and if No, the process proceeds to S2109. As S2110, 1 is set to “Hset”. As S2109, 0 is set to “Hset”.
[0090]
Next, FIG. 20 is a flowchart showing a chromaticity distribution analysis process in the vertical direction (V direction) which is the second direction. The operation of the flowchart shown in FIG. 21 is basically the same as the operation shown in the flowchart of FIG.
[0091]
The processing shown in the flowchart of FIG. 22 is executed in response to the results of the flowcharts of FIGS. The operation shown in the flowchart of FIG. 22 will be described.
[0092]
In S2201, the value of “Vset”, which is a determination as to whether or not there is a chromaticity distribution difference that allows vertical determination in the vertical direction, is determined. If “1 (present)”, the process proceeds to S2202, and if “0 (not present)”, the process proceeds to S2203.
[0093]
In S2202, the value of “Hset”, which is a determination as to whether or not there is a chromaticity distribution difference in which the vertical determination in the horizontal direction is recognized, is determined. In the case of “1 (present)”, since a difference in chromaticity distribution is recognized in both the vertical and horizontal directions, it is not possible to determine which of at least two directions is up or down, and the process proceeds to S2204. In the case of “0 (not present)”, since only the distribution characteristic in the vertical direction satisfies the condition, it is determined that the up / down determination is possible, and the process proceeds to S2205 to determine which is in the vertical direction.
[0094]
In S2205, it is determined whether or not the value of “Vtop”, which is a temporary designation in the vertical direction, is V1. If it is V1, the process proceeds to S2206. If it is not V1, the process proceeds to S2207.
[0095]
In S2206, it is determined that the vertical direction in the image is the vertical direction and the upper side is the V1 side, and finally V1 is set as Top and the process ends.
[0096]
In S2207, it is determined that the vertical direction in the image is the vertical direction and the upper side is the V1 side, and finally V5 is set as Top and the process ends.
[0097]
In S2203, the value of “Hset”, which is a determination as to whether or not there is a chromaticity distribution difference in which vertical determination in the horizontal direction is recognized, is determined. In the case of “0 (not present)”, since there is no difference in chromaticity distribution in both the vertical and horizontal directions, it is not possible to determine whether it is up or down. In the case of “1 (present)”, since only the distribution characteristic in the horizontal direction satisfies the condition, it is determined that the up / down determination is possible, and the process proceeds to S2208 to determine which is in the horizontal direction.
[0098]
In S2204, as a result of the chromaticity distribution analysis of the blue sky detection area, the up / down determination is impossible. Set indeterminate and exit.
[0099]
In S2208, it is determined whether or not the value of “Htop”, which is a temporary designation in the horizontal direction, is H1. If it is H1, the process proceeds to S2209, and if it is not H1, the process proceeds to S2210.
[0100]
In S2209, it is determined that the vertical direction in the image is the horizontal direction and the top is the H1 side, and finally H1 is set as Top and the process ends.
[0101]
In S2210, it is determined that the vertical direction in the image is the horizontal direction and the upper side is the H5 side, and finally H5 is set as Top and the process ends.
[0102]
As described above, according to the above-described embodiment, it is possible to determine the top / bottom of an image by analyzing the chromaticity distribution of the blue sky region, and compared with a PC, such as when directly printing from a digital camera. It is possible to detect a blue sky region in an image as a means for performing advanced correction processing on a compressed image file to be printed by processing within a range that can be used as a product even in a built-in device with low capability. .
[0103]
Conventionally, when automatically editing a shot image and using it in an automatic layout application such as an album, there is a problem that the top and bottom of the target image cannot be determined, and the laid out image is placed sideways or upside down. According to the above-described embodiment, it is possible to determine the up / down direction of the image by analyzing the chromaticity distribution in the detected region.
[0104]
Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.
[0105]
In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.
[0106]
As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
[0107]
Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (basic system or operating system) running on the computer based on the instruction of the program code. Needless to say, a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0108]
Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.
[0109]
【The invention's effect】
According to the present invention, the spatial frequency characteristics are calculated for each group composed of a plurality of partial pixel regions, and whether or not those spatial frequency characteristics are suitable for the appropriate range of the spatial frequency characteristics of the image of interest, Since it is configured to detect a group belonging to the image of interest, the load related to the image detection processing is reduced compared to the conventional technology in which the entire image is subjected to calculation of the spatial frequency characteristics, and the processing capability of, for example, a direct photo printer is reduced The target image can be detected even with a low device.
[Brief description of the drawings]
FIG. 1 is a conceptual diagram illustrating a flow of acquiring data necessary for decompressing a Jpeg image.
FIG. 2 is a conceptual diagram showing a flow of a process for converting image data into a Jpeg format.
FIG. 3 is a diagram showing a process of conversion into Jpeg format taking 8 * 8 Block, which is a Jpeg image compression unit, as an example;
FIG. 4 is a flowchart showing a blue sky detection process according to the first embodiment of the present invention.
FIG. 5 is a flowchart showing blue sky detection processing according to a second embodiment of the present invention.
FIG. 6 is a flowchart showing a blue sky detection process according to the third embodiment of the present invention.
FIG. 7 is a diagram for explaining a chromaticity detection method in 8 * 8 Block, which is a Jpeg file image compression unit.
FIG. 8 is a diagram for explaining a chromaticity detection method using a DC component in 8 * 8 Block which is a Jpeg file image compression unit.
FIG. 9 is a diagram illustrating a detection state in 8 * 8 Block when detection is performed using 3-bit decimation.
FIG. 10 is a flowchart showing image up / down determination processing using a detected blue sky region according to the fourth embodiment of the present invention.
FIG. 11 is a diagram illustrating an image sample that is a target of blue sky detection.
12 is a diagram showing a result of detecting only the chromaticity ratio and the luminance information as a blue sky detection method for the sample image of FIG. 11. FIG.
13 shows an area detected when a group adjacent to the partial pixel area is set as a blue sky detection method for the sample image 1 in FIG. 11, and an outermost partial pixel area of the area. It is the figure which showed the detection result when detecting.
14 is a diagram showing a result of detecting a blue sky region from the image sample of FIG.
FIG. 15 is a diagram showing an image sample for explaining an up / down determination process after detection of a blue sky;
FIG. 16 is a diagram showing actual data that is a basis for a change in chromaticity ratio for determining the up / down determination process after the blue sky is detected.
FIG. 17 is a diagram showing real data (luminance and chromaticity ratio) obtained by photographing a blue sky with products of digital camera companies as a region distribution.
18 is a diagram showing a result of appropriate detection of (8 * 8) blocks from image sample 2 in FIG.
FIG. 19 is a diagram showing an example when V (vertical direction) / H (horizontal direction) detected consecutive blocks in a detected area are grouped into five groups;
FIG. 20 is a flowchart showing an image up / down determination process using a detected blue sky region;
FIG. 21 is a flowchart showing image up / down determination processing using a detected blue sky region;
FIG. 22 is a flowchart showing an image up / down determination process using a detected blue sky region;
FIG. 23 is a graph showing the chromaticity ratio of the blue sky region in the captured image data including the blue sky region.
FIG. 24 is a diagram showing, as a graph, the chromaticity ratio of only those in which the luminance range in the blue sky region in the captured image data including the blue sky region falls within the range of 0 to 139.
FIG. 25 is a graph showing the chromaticity ratio of only those whose luminance range in the blue sky region in the captured image data including the blue sky region falls within the range of 140 to 179.
FIG. 26 is a graph showing the chromaticity ratio of only those in which the luminance range in the blue sky region in the captured image data including the blue sky region falls within the range of 180 to 199.
FIG. 27 is a graph showing the chromaticity ratio of only those in which the luminance range in the blue sky region in the captured image data including the blue sky region falls within the range of 200 to 255.
FIG. 28 is a diagram illustrating a relationship between a luminance range and a chromaticity ratio range for designating a blue sky region.
[Explanation of symbols]
11 Code table
12 Entropy decoding unit
13 Quantization table
14 Inverse quantization part
15 Reverse DCT section

Claims (12)

An image processing apparatus capable of decoding compressed image data by orthogonal transform processing,
Whether or not the chromaticity information and the luminance information acquired for each of the partial pixel areas, which are compression units of the compressed image data, conform to a range that matches a predetermined definition of the target image, Determining means for determining
Group generating means for grouping adjacent partial pixel areas among the partial pixel areas determined by the determining means to match the range matching the definition of the target image;
Calculating means for calculating the spatial frequency characteristics of each group generated by the group generating means;
It is determined for each of the groups whether or not the spatial frequency characteristic calculated by the calculation unit is matched with a preset appropriate range of the spatial frequency characteristic of the target image, and the target image is determined according to the determination result. An image processing apparatus comprising: a recognition unit that identifies a group to which the image belongs. An image processing apparatus capable of decoding compressed image data by orthogonal transform processing,
Whether or not the chromaticity information and the luminance information acquired for each of the partial pixel areas, which are compression units of the compressed image data, conform to a range that matches a predetermined definition of the target image, First determining means for determining
Group generating means for grouping adjacent partial pixel areas among the partial pixel areas determined by the first determination means to match the range matching the definition of the target image;
First calculation means for calculating a spatial frequency characteristic of each of the first group groups generated by the group generation means;
Second determination for determining whether or not the spatial frequency characteristic calculated by the first calculation means is suitable for a preset appropriate range of the spatial frequency characteristic of the target image. Means,
A second calculating unit that calculates an area ratio of the second group group determined by the second determination unit to occupy an appropriate range of the spatial frequency characteristic of the image of interest set in advance in the entire image; Means for calculating
Certification for determining whether each of the second group groups determines whether the area ratio calculated by the second calculation means exceeds a predetermined area ratio, and certifying the group belonging to the target image according to the determination result And an image processing apparatus. An image processing apparatus capable of decoding compressed image data by orthogonal transform processing,
Whether or not the chromaticity information and the luminance information acquired for each of the partial pixel areas, which are compression units of the compressed image data, conform to a range that matches a predetermined definition of the target image, First determining means for determining
Group generating means for grouping adjacent partial pixel areas among the partial pixel areas determined by the first determination means to match the range matching the definition of the target image;
First calculation means for calculating an area ratio of each of the first group groups generated by the group generation means in all images;
A second determination unit that determines, for each of the first group groups, whether or not an area ratio of the entire image exceeds a predetermined area ratio;
The spatial frequency of the region excluding the partial pixel region belonging to the outermost group of the group for each of the second group groups determined by the second determination means that the area ratio of the entire image exceeds the predetermined area ratio A second calculating means for calculating the characteristics;
It is determined for each of the second group whether or not the spatial frequency characteristic calculated by the second calculation means is matched with a preset appropriate range of the spatial frequency characteristic of the image of interest, and the determination result An image processing apparatus comprising: an authenticating unit that authorizes a group belonging to the target image according to Analyzing means for analyzing a chromaticity distribution on at least one of the groups certified as belonging to the target image by the certification means;
4. The image according to claim 1, further comprising: a determination unit configured to determine whether the image data decoded from the compressed image data is up-down based on an analysis result by the analysis unit. 5. Processing equipment. The analysis unit analyzes a chromaticity distribution on a line in a plurality of directions on the group to be analyzed for chromaticity distribution, and the determination unit is based on an analysis result of the chromaticity distribution on the line in a plurality of directions. The image processing apparatus according to claim 4, wherein the image data decoded from the compressed image data is determined to be up and down. The image processing apparatus according to claim 1, wherein the attention image is a blue sky image. The image processing apparatus according to claim 1, wherein the compressed image data is a JPEG image file. An image detection processing method by an image processing apparatus capable of decoding compressed image data by orthogonal transform processing,
The image processing apparatus includes:
Whether or not the chromaticity information and the luminance information acquired for each of the partial pixel areas, which are compression units of the compressed image data, conform to a range that matches a predetermined definition of the target image, Judgment about
Among the partial pixel areas determined to conform to the range that matches the definition of the image of interest, grouping adjacent partial pixel areas,
The spatial frequency characteristic of each group generated by the grouping is calculated, and it is determined for each group whether or not the calculated spatial frequency characteristic matches a preset appropriate range of the spatial frequency characteristic of the target image. And
An image detection processing method, wherein a group belonging to the image of interest is recognized according to the determination result. An image detection processing method by an image processing apparatus capable of decoding compressed image data by orthogonal transform processing,
The image processing apparatus includes:
Whether or not the chromaticity information and the luminance information acquired for each of the partial pixel areas, which are compression units of the compressed image data, conform to a range that matches a predetermined definition of the target image, Judgment about
Among the partial pixel areas determined to conform to the range that matches the definition of the image of interest, grouping adjacent partial pixel areas,
While calculating the spatial frequency characteristics of each first group generated by the grouping, whether or not the calculated spatial frequency characteristics are suitable for the appropriate range of the spatial frequency characteristics of the image of interest set in advance, Determine for each of the first group,
The area ratio of the second group group that has been determined to have a spatial frequency characteristic suitable for the preset appropriate range of the spatial frequency characteristic of the image of interest is calculated, and the calculated area ratio is a predetermined area. Determining whether or not the ratio is exceeded for each of the second group of groups,
An image detection processing method, wherein a group belonging to the image of interest is recognized according to the determination result. An image detection processing method by an image processing apparatus capable of decoding compressed image data by orthogonal transform processing,
The image processing apparatus includes:
Whether or not the chromaticity information and the luminance information acquired for each of the partial pixel areas, which are compression units of the compressed image data, conform to a range that matches a predetermined definition of the target image, Judgment about
Among the partial pixel areas determined to conform to the range that matches the definition of the image of interest, grouping adjacent partial pixel areas,
For each of the first group groups, the area ratio of the first group group generated by the grouping is calculated for the entire image, and whether the area ratio of the first group group exceeds a predetermined area ratio is determined for each of the first group groups. Judgment,
For each of the second group groups determined that the area ratio of the entire image exceeds the predetermined area ratio, the spatial frequency characteristics of the area excluding the partial pixel area belonging to the outermost group of the group were calculated and calculated. It is determined for each of the second groups whether or not the spatial frequency characteristics are matched with an appropriate range of the spatial frequency characteristics of the image of interest set in advance.
An image detection processing method, wherein a group belonging to the image of interest is recognized according to the determination result. A program for causing a computer to execute the image detection processing method according to any one of claims 8 to 10. The computer-readable recording medium which recorded the program of Claim 11.

Images