JP5263010B2 - Image processing apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and a program for specifying a main object included in an image even if a whole area of the image is focused. <P>SOLUTION: A band image which is obtained by extracting an image of each frequency band is produced in each previously determined frequency band in which an outline of a focused object is extracted from image information indicating an object image to be processed, and in each at least one frequency band which is lower than the previously determined frequency band, then a feature portion which becomes to be a feature relating to each produced band image is extracted. When a proportion occupied by the feature portion within the band image of the predetermined frequency band in which the feature portion is extracted becomes not less than a previously determined proportion, an object area of the image indicated by the image information is specified based on the band image of each frequency band from which the band image of the previously determined frequency band is removed. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an image processing apparatus and a program.

  Patent Document 1 describes an image processing apparatus that processes a two-dimensional image input from an imaging device such as a television camera and determines the type / number / classification / normal / abnormality of an object present in the image. Filtering means for applying one or more kinds of spatial bandpass filters to the input image, ternary processing means for processing the output of the spatial bandpass filter with two threshold values to obtain a ternary image, and ternary values Local feature detection means for detecting the similarity between an image and a local pattern, threshold means for generating an output when the similarity between a local pattern and a local pattern exceeds a certain value, and the output of the threshold means in the image at several points A feature vector extracting means having a plurality of local feature counting means for counting whether or not, an output of the feature vector extracting means and one or more standard vectors stored in advance Obtains the similarity between the Le, the image processing apparatus having a feature vector determination unit for outputting a final determination results is described.

  Patent Document 2 discloses a method for partitioning image data, in which an image feature space is defined based on frequency information, the image data in the image feature space is filtered using a morphological tool, and the filtering process is performed. A method is described in which a region of the image feature space is assigned as an initial subject, the boundary of the initial subject of the filtered image feature space is identified, and the size of the initial subject is determined with respect to the image data size.

  Patent Document 3 discloses an image processing apparatus that extracts an area where a subject is present from an image, an image input unit that inputs an image, and resolution conversion that converts the input image into resolution-converted images having a plurality of different resolutions. A feature extraction unit that calculates a predetermined feature amount for each of the resolution conversion images, an attention degree calculation unit that calculates an attention level of each of the resolution conversion images based on the predetermined feature amount, and a resolution conversion image The image processing apparatus includes an attention level synthesis unit that synthesizes the attention level of each and generates an attention level map, and an area determination unit that determines a subject area based on the attention level map.

JP-A-7-93545 JP 2004-164624 A JP 2007-122101 A

  An object of the present invention is to provide an image processing apparatus and a program that can identify a main subject included in an image even if the image is in focus.

  According to the first aspect of the present invention, a predetermined frequency band in which a contour of a focused subject is extracted from image information indicating an image to be processed and at least one frequency band lower than the predetermined frequency band are extracted. Generating means for generating a band image obtained by extracting an image of each frequency band; extracting means for extracting a characteristic part for each band image generated by the generating means; and Based on the band images of each frequency band excluding the band image of the predetermined frequency band when the proportion of the feature portion in the extracted band image of the predetermined frequency band is greater than or equal to the predetermined ratio Specifying means for specifying the subject area of the image indicated by the image information.

  According to a second aspect of the present invention, in the first aspect of the invention, the specifying unit specifies a region where characteristic portions overlap in a band image of each frequency band as a subject region of an image indicated by the image information. To do.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the predetermined frequency band is 128 / long side length to 64 / long side of the image indicated by the image information. The frequency band is longer than either length.

  On the other hand, in the invention according to claim 4, the computer is caused to extract a contour of a focused subject from image information indicating an image to be processed, and at least a predetermined frequency band lower than the predetermined frequency band. Generating means for generating a band image obtained by extracting an image of each frequency band for each frequency band; extracting means for extracting a characteristic portion that is characteristic for each band image generated by the generating means; and the extraction Each frequency band excluding the band image of the predetermined frequency band when the ratio of the characteristic part in the band image of the predetermined frequency band from which the characteristic part has been extracted by the means is equal to or greater than a predetermined ratio And a specifying unit that specifies a subject area of an image indicated by the image information based on the band image.

  According to the first and fourth aspects of the invention, there is an excellent effect that the main subject included in the image is specified even if the image is in focus.

  Further, according to the second aspect of the present invention, there is an excellent effect that the subject can be discriminated more accurately than in the case where the subject is discriminated separately from each band image.

  In addition, according to the third aspect of the present invention, there is an excellent effect that the contour of the subject is extracted with higher accuracy than the band images of other frequency bands.

It is a block diagram which shows schematic structure of PC which concerns on embodiment. It is the block diagram which showed the subject specific process which concerns on embodiment functionally. It is a schematic diagram which shows the change of the image at the time of performing the to-be-photographed object process which concerns on embodiment on the image of pan focus. (A) is a pan-focus image, and (B) is a high-frequency band image. (A) is a band image in the middle frequency band, and (B) is a band image in the low frequency band. It is a schematic diagram which shows the change of the image at the time of performing the to-be-photographed object process which concerns on embodiment on the image of pan focus. It is a figure which shows the result of having specified the to-be-photographed object area | region of the image of pan focus. It is a schematic diagram which shows the change of the image at the time of performing the to-be-photographed object process which concerns on embodiment with respect to the image which is not pan focus. (A) is an image which is not pan focus, and (B) is a high frequency band image. (A) is a band image in the middle frequency band, and (B) is a band image in the low frequency band. It is a schematic diagram which shows the change of the image at the time of performing the to-be-photographed object process which concerns on embodiment with respect to the image which is not pan focus. It is a figure which shows the result of having specified the to-be-photographed object area | region of the image which is not pan focus. It is a flowchart which shows the flow of a process of the to-be-photographed object processing program which concerns on embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Hereinafter, a case where the present invention is applied to a personal computer (hereinafter referred to as “PC”) as an image processing apparatus will be described.

  FIG. 1 shows a schematic configuration of the PC 12 according to the present embodiment.

  The PC 12 includes a display 50 that displays an operation menu, a message, and the like, and an operation input unit 52 such as a keyboard and a pointing device for inputting various operation instructions from the user.

  The PC 12 temporarily stores a CPU (Central Processing Unit) 54 that controls the operation of the entire apparatus, a ROM (Read Only Memory) 56 as a recording medium in which various programs including a control program are stored in advance, and various types of information. A RAM 58 (Random Access Memory) to be stored, an HDD 60 that stores and holds various installed application software and various information, a display driver 62 that controls display of various information on the display 50, and an operation input unit 52 An operation input detection unit 64 that detects an operation state of the device, a media I / F unit 66 that writes and reads information to and from a portable storage medium 65 such as a USB memory or an SD (Secure Digital) card, and the network 18. And a communication I / F unit 68 for performing communication.

  The CPU 54, ROM 56, RAM 58, HDD 60, display driver 62, operation input detection unit 64, media I / F unit 66, and communication I / F unit 68 are connected to each other via a system bus BUS. Accordingly, the CPU 54 accesses the ROM 56, RAM 58, and HDD 60, controls the display of various information on the display 50 via the display driver 62, and the portable storage medium 65 via the media I / F unit 66. Control of writing and reading information to and from the network 18 via the communication I / F unit 68 is performed. Further, the CPU 54 grasps the operation state of the user with respect to the operation input unit 52 via the operation input detection unit 64.

  Incidentally, the PC 12 according to the present embodiment performs predetermined image processing for correcting an image with respect to image information. The PC 12 acquires the image information by reading the image information from the portable storage medium 65 attached to the media I / F unit 66 or receiving the image information via the communication I / F unit 68.

  Here, it is often judged that the main subject (for example, a human face region) in the image is satisfactory if it is finished with an appropriate density and color balance.

  For this reason, in the PC 12 according to the present embodiment, the subject identification processing is performed to identify the main subject from the image indicated by the image information, and the identified main subject is finished with an appropriate density and color balance. The image is corrected.

  FIG. 2 is a block diagram functionally showing the subject specifying process according to the present embodiment. This block diagram divides the subject specifying process functionally, and does not limit the hardware configuration.

  The subject specifying process according to the present embodiment includes a band image generation unit 70 that generates a band image obtained by extracting an image of each frequency band for each of a plurality of predetermined frequency bands from image information indicating an image to be processed; A feature extraction unit 72 that extracts a feature portion that is a feature of each band image generated by the band image generation unit 70, and an image in which an image to be processed is focused on the entire image (a so-called pan focus image) ) And a subject region specifying unit 76 for specifying the subject region of the image indicated by the image information.

  The band image generation unit 70 according to the present embodiment includes, as a plurality of frequency bands, a high frequency band in which the contour of a subject included in the image is extracted, a medium frequency band in which the main image configuration of the image is extracted, and an image A band image of three frequency bands of a low frequency band from which a region of interest is extracted is extracted, and a high pass filter 80, low pass filters 82A and 82B, an average value filter 84, and a subtraction processing unit 86A. , 86B.

  Image information indicating an image to be processed is input to the high-pass filter 80, the low-pass filters 82A and 82B, and the average value filter 84.

  The high-pass filter 80 extracts an image having a frequency band greater than or equal to 128 / long side length to 64 / long side length of the image to be processed to generate a high frequency band image.

  The low-pass filter 82A extracts an image having a frequency band equal to or less than 64 / long side length to 16 / long side length of the image to be processed. The low-pass filter 82B extracts an image in a frequency band equal to or less than any of 16 / long side length to 4 / long side length of the image to be processed. The average value filter 84 obtains an average value of pixel values of an image to be processed, and generates an average value image using the average value as each pixel value.

  The subtraction processing unit 86A subtracts each pixel value of the image extracted by the low-pass filter 82B from each pixel value of the image extracted by the low-pass filter 82A for each pixel corresponding to a position in the image, and A band image is generated.

  The subtraction processing unit 86B subtracts each pixel value of the image generated by the average value filter 84 from each pixel value of the image extracted by the low-pass filter 82B for each pixel corresponding to a position in the image, and outputs the low frequency band. A band image is generated.

  The feature extraction unit 72 according to the present embodiment is configured to extract a feature portion that is a feature of the band image of each band generated by the band image generation unit 70. A processing unit 92 is provided.

  For each band image, the nonlinear conversion processing unit 90 obtains an average value of each pixel value of the image, and generates an image having a difference between the average value and the pixel value of each pixel as a new pixel value. Then, the non-linear conversion processing unit 90 performs non-linear conversion on the pixel value of each image so that the increase rate of the pixel value increases as the new pixel value increases. In this non-linear conversion, for example, an exponential function or the like may be used so that the pixel value increases as the pixel value increases, and as the pixel value increases step by step using a plurality of threshold values. You may convert so that an increase rate may become large.

  The expansion processing unit 92 performs an expansion process on the image of each band processed by the nonlinear conversion processing unit 90 to extract a feature portion having a large amount of information in the image.

  FIGS. 3 to 6 show examples of results obtained by performing processing by the band image generation unit 70 and the feature extraction unit 72 on the pan-focus image 100. 4 and 5 are enlarged views of the image 100, the high-frequency band image 102A, the middle-frequency band image 102B, and the low-frequency band image 102C in FIG.

  As illustrated in FIG. 3, the image 100 is converted into a high-frequency band image 102 </ b> A, a medium-frequency band image 102 </ b> B, and a low-frequency band image 102 </ b> C by processing by the band image generation unit 70.

  As shown in FIGS. 4 and 5, the outline of the focused subject in the image 100 is extracted from the band image 102A, and the main image configuration of the image 100 is extracted from the band image 102B. In the band image 102 </ b> C, a region with a large contrast that is easily noticed as human visual characteristics of the image 100 is extracted as a region of interest.

  As shown in FIG. 6, the band images 102A to 102C are then converted into images 104A to 104C from which feature portions have been extracted by the processing by the nonlinear conversion processing unit 90, and the images 104A to 104C are processed by the expansion processing unit 92. Thus, the images of the characteristic portions are converted into the expanded images 106A to 106C.

  Here, when the image 100 is a pan-focus image, since the entire image is in focus, the contour is extracted from the entire band image 102A. For this reason, in the image 106A, the proportion of the characteristic portion increases.

  Therefore, the pan focus determination unit 74 binarizes the image 106A with a predetermined threshold value, and the ratio of the portion that has the same value as the feature portion (white in the image 107) in the binarized image is a predetermined ratio. If it is (for example, 60%) or more, it is determined that the image 100 is a pan-focus image. As this predetermined ratio, a value obtained in advance by an experiment using a real machine, a computer simulation, or the like is applied as a value for accurately discriminating a pan focus image.

  As a result of the determination by the pan focus determination unit 74, the subject region specifying unit 76 specifies a region where the characteristic portions overlap in the images 106A to 106C as the subject region of the image when it is determined that the image 100 is not a pan focus image. When it is determined that the image 100 is a pan-focus image, an area where the characteristic portions overlap in the images 106B and 106C is specified as the subject area of the image except for the image 106A.

  In FIG. 6, the area occupied by the characteristic portion in the images 106 </ b> B and 106 </ b> C (the white area of the image 108 in FIG. 6) is the subject of the image 100, where the ratio of the characteristic portion in the image 106 </ b> A is equal to or greater than the predetermined ratio. Identified as an area.

  FIG. 7 shows the result of specifying the subject area of the image 100 by the subject specifying process according to the present embodiment.

  On the other hand, FIGS. 8 to 11 show an example of the result of performing the same processing on the image 120 that is not pan-focused. 9 and 10 are enlarged views of the image 100 of FIG. 8, the high-frequency band image 102A, the medium-frequency band image 102B, and the low-frequency band image 102C.

  When the image 120 is not a pan-focus image, since a part of the image (the horse part in the image 120) is in focus, a contour is extracted from a part of the band image 122A. For this reason, as shown in FIG. 11, in the image 126A, the proportion of the characteristic portion is reduced, and it is determined that the image 120 is not a pan-focus image.

  In FIG. 11, the area where the characteristic portion overlaps in the images 126 </ b> A to 126 </ b> C (the white area of the image 128 in FIG. 11) is the subject of the image 120, where the proportion of the characteristic portion in the image 126 </ b> A is less than the predetermined ratio. Identified as an area.

  FIG. 12 shows the result of specifying the subject area of the image 120 by the subject specifying process according to the present embodiment.

  In the PC 12 according to the present embodiment, the processing by each component of the subject specifying process (the band image generating unit 70, the feature extracting unit 72, the pan focus determining unit 74, and the subject region specifying unit 76) executes a program. Therefore, it is realized by a software configuration using a computer. However, the implementation is not limited to the software configuration, and may be realized by a hardware configuration or a combination of the hardware configuration and the software configuration.

  In the following, a case will be described in which the PC 12 according to the present embodiment realizes processing by each of the above-described components by executing a subject specifying processing program.

  FIG. 13 shows a flowchart showing the flow of the subject specifying processing program executed by the CPU 54. The program is stored in advance in the HDD 60 as a recording medium.

  In step 150 of the figure, from the image information indicating the image to be processed, a high frequency band greater than or equal to 128 / long side length to 64 / long side length of the image, 64 / long side length of the image. A medium frequency band less than or equal to 16 / long side length and an image in a low frequency band less than or equal to 16 / long side length to 4 / long side length of the image are respectively extracted. Band images of a frequency band, a medium frequency band, and a low frequency band are generated.

  In the next step 152, an average value of each pixel value of the image is obtained for each band image generated in the above step 150, and the difference between the average value and the pixel value of each pixel is used as a new pixel value. Is generated.

  In the next step 154, the pixel value of each image is nonlinearly converted so that the pixel value becomes larger as the pixel value of each pixel generated in step 152 becomes larger.

  In the next step 156, expansion processing is performed on the image converted in step 154.

  In the next step 158, it is determined whether or not the proportion of the feature portion in the high frequency band image subjected to the expansion process in step 156 is equal to or greater than the predetermined proportion. The process proceeds to 160, and if a negative determination is made, the process proceeds to step 162.

  In step 160, an area where the characteristic portions overlap in the image in the medium frequency band and the image in the low frequency band is specified as the subject area of the image except for the image in the high frequency band.

  On the other hand, in step 162, an area where the characteristic portions overlap in the high frequency band image, the medium frequency band image, and the low frequency band image is specified as the subject area of the image.

  In this way, when the image to be processed is a pan-focus image in which the entire image is in focus, the subject area of the image is identified based on the band image of each frequency band excluding the band image of the high frequency band As a result, the main subject of the image to be processed is identified with high accuracy.

  In the above embodiment, the case where the present invention is applied to the PC 12 has been described. However, the present invention is not limited to this, for example, a photographing apparatus such as a digital camera or a video camera, a copying machine, a printer, or the like. The present invention may be applied to other image forming apparatuses. The image processing apparatus of the present invention includes a photographing apparatus and an image forming apparatus. In the case of a photographing apparatus, a main subject may be identified and focused on the main subject, or image correction may be performed so that the main subject has an appropriate density and color balance. In the case of an image forming apparatus, image correction may be performed so that a main subject has an appropriate density and color balance.

  Further, in the above embodiment, a case has been described in which a feature portion having a large amount of information is extracted from an image by obtaining a difference between the pixel value of each pixel and the average value for each band image. The invention is not limited to this, and the amount of information may be calculated using pixel dispersion, the amount of edge, and the like, and the method of extracting the feature portion is not limited to this. For example, a portion having a large entropy may be extracted as a feature portion by obtaining entropy.

  Further, in the above-described embodiment, the case where the region of the characteristic portion is extracted by performing the expansion process on the image of each band has been described. However, the present invention is not limited to this, and for example, the expansion is performed. It is also possible to perform processing for closing the contour of the image by repeating the processing and the contraction processing, and extract the region surrounded by the contour as the region of the feature portion.

  In the above embodiment, the case of extracting the medium frequency band and the low frequency band image together with the high frequency band image has been described. However, the present invention is not limited to this, and the medium frequency band is extracted. Alternatively, only one of the low-frequency band images may be used.

  The subject identification processing program according to the above embodiment is stored in advance in the HDD 60, stored in the ROM 56 in advance, or on a computer-readable storage medium such as a CD-ROM or DVD-ROM. The present invention can be applied to a form provided in a stored state, a form distributed via wired or wireless communication means, and the like.

12 PC
54 CPU
70 Band Image Generation Unit 72 Feature Extraction Unit 74 Pan Focus Determination Unit 76 Subject Area Identification Unit

Claims (4)

An image of each frequency band is extracted for each of a predetermined frequency band from which the contour of the focused subject is extracted from image information indicating an image to be processed and at least one frequency band lower than the predetermined frequency band Generating means for generating a band image;
Extraction means for extracting a characteristic portion that is characteristic for each band image generated by the generation means;
In the band image of the predetermined frequency band in which the characteristic part is extracted by the extracting unit, when the proportion of the characteristic part is equal to or higher than a predetermined ratio, each of the band images except for the predetermined frequency band is excluded Specifying means for specifying a subject region of an image indicated by the image information based on a band image of a frequency band;
An image processing apparatus. The image processing apparatus according to claim 1, wherein the specifying unit specifies a region where characteristic portions overlap in a band image of each frequency band as a subject region of an image indicated by the image information. The image processing apparatus according to claim 1, wherein the predetermined frequency band is a frequency band greater than or equal to 128 / long side length to 64 / long side length of the image indicated by the image information. . Computer
An image of each frequency band is extracted for each of a predetermined frequency band from which the contour of the focused subject is extracted from image information indicating an image to be processed and at least one frequency band lower than the predetermined frequency band Generating means for generating a band image;
Extraction means for extracting a characteristic portion that is characteristic for each band image generated by the generation means;
When the proportion of the feature portion in the band image of the predetermined frequency band from which the feature portion has been extracted by the extraction unit is equal to or greater than a predetermined proportion, Specifying means for specifying a subject region of an image indicated by the image information based on a band image of a frequency band;
Program to function as.