JP2005339076A - Object area extraction device, extraction method, program for the method, and recording medium recorded with the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To stably extract an object of low luminance and achromatic color from an image. <P>SOLUTION: An image converting means 101 converts the image into a color coordinate system having chroma, hue and luminance. An area cutting means 102 cuts out a rectangular area from the image shown by chroma, hue and luminance. A reliability calculating means 103 calculates reliability from chroma and luminance information of the rectangular area, a characteristic and a image system of an image pickup system based on hue and luminance in the area of the image. An object feature calculating means 104 calculates the feature obtained by quantizing and histogramming the hue of the color coordinate system. When reliability of the hue is low, luminance of the image is quantized and a luminance histogram is calculated. A hue histogram and the luminance histogram are put together and one histogram feature is calculated. A calculation means of objective image feature 105 calculates the feature of the whole image. An object extraction means 106 extracts the area where the object exists from the objective image based on the feature. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and method for searching an area similar to a template from search target images, and in particular, in a color image obtained by shooting an object, the object to be shot exists near the center of the image. Assuming that, the present invention relates to an object region extracting apparatus and method for stably narrowing down the position where an object exists by using the color information of the object that has been formed into a histogram.

  As a method for extracting an object region from a color image, there is a method of extracting a character region from a color image by color matching line processing (see, for example, Non-Patent Document 1). In this method, first, a single character is detected from a color matching line image generated from an image, and a character region is extracted using the property that characters are arranged in a straight line in a character string.

As another method, a method of extracting a road sign from a color road scene image has been studied (for example, see Non-Patent Document 2). In this method, signs are extracted from an image using hue information and shape information.
Ryukoku University, Ueda / Takeda / Okada, IEICE Technical Report, PRMU94-28. September 1994, pp.9-16 Nara National College of Technology, Nakai / Matsuo / Ueda, Osaka Electro-Communication University, Umeda, IEICE Technical Report, PRMU95-158. November 1995, pp.5-8

  Since the color information of the image is used in the prior art, correct hue information can be obtained when the luminance of the target object is significantly reduced due to an image including a shadow with low luminance or backlight. However, there is a problem that the accuracy is lowered.

  In addition, since the achromatic object does not include hue information, there is a problem in that it cannot be identified.

  The present invention has been made in view of such circumstances, and an object area extraction apparatus, an extraction method, a program for this method, and a recording in which this program is recorded, which can stably extract low-luminance or achromatic objects from an image. The purpose is to provide a medium.

  The present invention can extract low luminance and achromatic objects stably by extracting saturation, hue, and luminance components from an image and flexibly changing characteristics using the saturation and luminance as keys. In order to stably search for an object from the target image, the captured image is converted into the XYZ color system, which is color information independent of the characteristics of the imaging system, and the color from the XYZ color system is changed. Image conversion that can be converted to Munsell color system with degree, hue, and luminance information, assuming that an object exists near the center of the captured image, and cut out the vicinity of the image center in a rectangular area of a predetermined size The reliability of the hue information of the image is calculated from the saturation and luminance information of the clipped area, the characteristics of the imaging system, and the image format of the captured image, and the local area of the clipped area and the entire image is calculated. The hue histogram and luminance histogram in the image are calculated, and a new histogram is created from the hue and luminance histogram of the entire image extracted based on the reliability, and the histogram of the extracted image and the histogram of the entire image are created. By calculating the similarity, a region is stably extracted even if the object present in the search target image has low luminance or achromatic color.

  From the above, the present invention is characterized by the following apparatus, method, program, and recording medium.

(1) An object area extraction device for extracting an area where an unknown object exists in an image,
Image conversion means capable of converting from an image to a color system having image saturation, hue, and luminance;
Assuming that an object exists near the center of the target image, a region cutout of a rectangular region of a predetermined size from the vicinity of the center of the image information indicated by the saturation, hue, and luminance calculated based on the image conversion means Means,
The saturation, luminance information, and imaging system characteristics of an appropriately sized rectangular area near the center of the image cut out by the area cutting means, and the hue and luminance changes in the area of the image from the image format are determined. A reliability calculation means for calculating the reliability based on;
A feature in which the hue of the color system converted by the image conversion unit is quantized and histogrammed is calculated, and the hue of the image is histogrammed when the reliability of the hue calculated by the reliability calculation unit is low. In addition, an object feature calculation unit that calculates the luminance histogram by quantizing the luminance of the image, and calculates the feature of the region cut out by the region cut-out unit as one histogram feature by combining the hue histogram and the luminance histogram;
A feature in which the hue of the color system converted by the image conversion unit is quantized and histogrammed is calculated, and the hue of the image is histogrammed when the reliability of the hue calculated by the reliability calculation unit is low. In addition, the target image feature calculation means for calculating the brightness histogram by quantizing the brightness of the image, calculating the feature of the entire target image as one histogram feature by combining the hue histogram and the brightness histogram,
An object extraction unit that searches for an object from a target image based on features obtained from the object feature calculation unit and the target image feature calculation unit, and extracts an area where the object exists from the image;
An object area extraction apparatus comprising:

  (2) In the object feature calculation unit and the target image feature calculation unit described in (1) above, when the reliability of the hue of the image is low, in addition to the feature histogram, the edge strength and / or the edge strength from the luminance An object region extracting apparatus characterized in that the object extracting unit extracts an object as a histogram together with a hue histogram.

  (3) In the feature calculation means described in the above (1) or (2), the hue and the luminance histogram are changed by changing the quantization step size of the hue and the luminance histogram depending on the reliability of the hue when generating the histogram feature. An object region extraction apparatus characterized in that features are extracted by changing luminance weights.

(4) An object area extraction method for extracting an area where an unknown object exists in an image,
An image conversion process that can convert an image into a color system having the saturation, hue, and luminance of the image,
Assuming that an object exists near the center of the target image, a region cut out of a rectangular region of a predetermined size from the vicinity of the center of the image information indicated by the saturation, hue, and luminance calculated based on the image conversion process Process,
The saturation, brightness information, and imaging system characteristics of an appropriately sized rectangular area near the center of the image cut out by the area cutout process, and the magnitude of changes in hue and brightness in the area of the image from the image format. A reliability calculation process for calculating the reliability based on
A feature obtained by quantizing the hue of the color system converted by the image conversion process and calculating a histogram, and if the reliability of the hue calculated by the reliability calculation process is low, a histogram of the hue of the image In addition, an object feature calculation process that calculates a luminance histogram by quantizing the luminance of the image, calculates a feature of the region extracted by the region extraction process as one histogram feature by combining the hue histogram and the luminance histogram,
A feature obtained by quantizing the hue of the color system converted by the image conversion process and calculating a histogram, and if the reliability of the hue calculated by the reliability calculation process is low, a histogram of the hue of the image In addition, a luminance histogram is calculated by quantizing the luminance of the image, a target image feature calculation process for calculating a feature of the entire target image as one histogram feature by combining the hue histogram and the luminance histogram,
An object extraction step of searching for an object from the target image based on the features obtained from the object feature calculation step and the target image feature calculation step, and extracting an area where the object exists from the image;
An object region extraction method characterized by comprising:

  (5) In the object feature calculation process and the target image feature calculation process described in (4) above, when the reliability of the hue of the image is low, in addition to the feature histogram, the edge intensity and / or the edge intensity from the luminance The object region extraction method is characterized in that an object is extracted in the object extraction process as a histogram together with a hue histogram.

  (6) In the feature calculation process described in (4) or (5) above, by changing the quantization step size of the hue and luminance histogram depending on the reliability of the hue when generating the histogram feature, An object region extraction method characterized by extracting features by changing luminance weights.

  (7) A program characterized in that the object region extraction method according to any one of (4) to (6) above is described in a computer program and is executable.

  (8) A recording medium recorded with a computer readable recording of a program configured to execute the object region extraction method according to any one of (4) to (6) above.

  As described above, as a technique for extracting an area in which an unknown object in the search target image is likely to exist, a hue that is robust to the luminance change of the image has been conventionally used. However, this technique has a problem that the luminance is remarkably lowered or it is difficult to extract a region where an achromatic object exists.

  The present invention solves these problems, and can stably extract low-luminance or achromatic objects.

  FIG. 1 shows an overall configuration diagram of an embodiment of the present invention. In FIG. 1, 101 is an image conversion means, 102 is a region cutout means, 103 is a reliability calculation means, 104 is an object feature calculation means, 105 is a search target image feature calculation means, and 106 is an object extraction means.

  In this embodiment, first, the image conversion unit 101 converts the pixel value of the search target image into the Munsell color system having information on saturation, hue, and luminance. In the conversion, first, conversion to the XYZ color system defined by the CIE independent of the imaging form used for image acquisition is performed, and then conversion to the Munsell color system is performed. As a means of conversion, a color chart having a sufficient number of colors is photographed, a pixel value output as a result, and a neural network that learns XYZ values measured under the same conditions as when photographing, and corresponding values are shown. Use the conversion table. It is assumed that an existing conversion table is used for conversion from the XYZ value to the Munsell color system. At this time, there is a case where a conversion is performed by creating a table that collectively converts the pixel values of the imaging system to the Munsell color system without converting to the XYZ color system.

  Next, the region cutout unit 102 cuts out a region near the center of the search target image with a rectangular region having an appropriate size. As shown in FIG. 2, the area is specified when displaying as a guide of several patterns at the time of shooting, by specifying a guide when extracting an object from an already shot image, or by a predetermined size. It shall be.

  Next, the reliability calculation means 103 lowers the hue reliability according to the value when the saturation or luminance reliability of the image is low, and when the CCD is used as the imaging system, the dark current, When the CMOS is used, the reliability of the pixel value at low luminance is lowered due to the influence of switching noise. Therefore, the reliability of the hue is lowered according to the reliability of the pixel value at low luminance of the imaging system. In addition to this, when color information is greatly impaired due to the type of compression format of an image, such as a JPEG image, the reliability of hue is lowered. As a specific reliability calculation method, a method is conceivable in which a statistical S / N ratio with respect to an input signal is measured in advance and the value is used as the reliability as it is. By this method, a reliability table relating to luminance and a reliability table relating to hue can be prepared, and these tables will be referred to for calculation of reliability.

Next, the object feature calculation unit 104 creates a hue histogram and a luminance histogram from the object features cut out by the region cutout unit 102 by counting the number of pixels included in a certain hue or luminance range. Then, the created histogram is weighted by the reliability calculated by the reliability calculation means 103 to create a histogram having the characteristics of the object. The histogram is weighted by changing the quantization step size of the hue and luminance as shown in FIG. Quantization step size changing method, from the reliability R _H of the reliability R _I and hue luminance calculated by the reliability calculation unit 103,

Thus, the luminance quantization step width W _I and the hue quantization step width W _H can be calculated. The functions f _I and f _H are, for example,

Etc. can be considered. Here, P is a parameter. When the hue reliability is low, the edge strength of the image and the dispersion of the edge strength can also be used as the feature of the object.

  Next, the search target image feature extraction unit 105 extracts a region having the same size as the region extracted by the region extraction unit 102 with the search position appropriately determined from the search target image as the center, and is the same as the object feature calculation unit 104. By performing the process, the feature of the search target image at the position is calculated. Here, the search position in the search target image is selected by shifting in the vertical and horizontal directions of the image for every pixel or for each appropriately determined step size.

  Finally, the object extraction unit 106 normalizes the cross-correlation between the characteristics of each search point extracted by the search target image feature extraction unit 105 and the object features calculated by the object feature calculation unit 104 (for example, non-patent document: Makoto Nagao: “Digital Image Processing” pp 306-310, Modern Science Co., Ltd., 1995), or normalizes the features of the object and the search target image, and calculates the distance between the normalized histograms. When using the normalized cross-correlation, a search point having a correlation equal to or larger than a predetermined threshold is output, and when using a distance between histograms, a search point having a distance equal to or smaller than the predetermined threshold is output as an object region.

  FIG. 4 is a diagram showing a processing flow of the image conversion unit 101, the region cutout unit 102, the reliability calculation unit 103, and the object feature calculation unit 104 in the present invention. First, a search target image is read in S1, and in S2 and S3, conversion into an XYZ color system and a Munsell color system is performed, respectively. Here, the image conversion means converts to the Munsell color system after converting it to the XYZ color system, but it is also possible to batch convert the pixel values of the read image to the Munsell color system without going through the XYZ color system. is there.

  The pixel value converted to the Munsell color system in S3 is sent to S4 and S13 shown in FIG. In S4, the size of the area to be cut out as an object area from the image is designated. The size of the area to be cut out is selected from existing sizes, or an appropriately determined size is selected, or a predetermined size is set. After specifying the size in S4, based on the size, the object region is cut out from the center of the image in S5.

  In S6, the average value, median value, or mode value of luminance / hue is calculated from the cut image, and the reliability is calculated by referring to the luminance reliability table and the hue reliability table. The calculated reliability is sent to S7 and S13.

  In S7, a hue and luminance histogram is created. At this time, on the basis of the hue reliability calculated in S6, the hue and luminance quantization step widths at the time of creating the histogram are changed according to the above formulas (1), (2), (3), and (4). Also, the calculated hue and luminance histograms are created as one histogram, and this is used as the object histogram. In S8, it is determined whether or not to use the edge strength of the image and the variance of the edge strength based on the reliability of the hue. If the edge strength and the variance of the edge strength are used, the process proceeds to S9. If not, the process proceeds to S12. Move on to processing.

  In S9, differentiation processing is performed on the luminance of the image, and in S10, the edge strength is calculated by taking the square of the differentiation processing result, and the variance of the edge strength calculated in S11 is calculated. In S10, the average edge strength in all pixels of the cut-out image is also calculated. In S12, the feature of the object is calculated using the calculated histogram and the average and variance of the edge strength.

  FIG. 5 is a diagram showing a processing flow of the target image feature calculating unit 105 and the object extracting unit 106. In S13, the pixel value converted into the Munsell color system of the search target image calculated in S3 and the hue reliability calculated in S6 are read, and the hue and luminance histograms at all search points in the image are created. At this time, based on the hue reliability calculated in S6, the quantization step size of the hue and luminance at the time of creating the histogram is changed. Also, a histogram of hue and luminance calculated at each search point is created as one histogram, and this is used as the histogram at that search point. As search points, all pixels of the search target image or positions sampled in the vertical and horizontal directions of the image at a predetermined step size are used. In S14, it is determined whether or not to use the edge strength of the image and the variance of the edge strength based on the reliability of the hue. If the edge strength and the variance of the edge strength are used, the process proceeds to S15. If not, the process proceeds to S18. Move on to processing.

  In S15, differentiation processing is performed on the luminance of the image, and in S16, the edge strength is calculated by taking the square of the differentiation processing result, and the variance of the edge strength calculated in S17 is calculated. In S16, the average edge strength in all pixels of the cut-out image is also calculated. In S18, the feature at each search point of the search target image is calculated using the calculated histogram and the average and variance of the edge strength.

  In S19, the search position of the object is designated, and in S20, the feature of the object calculated in S12 is read, and the similarity with the object is calculated at the designated search position. As the similarity, normalized cross-correlation of the features of the object and the search target image, or normalization of each feature, and the distance between the features is used. If the normalized cross-correlation value is equal to or greater than a predetermined threshold value, or the distance between features is equal to or smaller than the predetermined threshold value, the search point is output as a region where the object exists. In S21, it is determined whether or not the similarity has been calculated for all the search points. If there is a search point for which the similarity is not calculated, the search point is selected again until the search at all the search points is completed. , S20 is repeated. If the calculation of the similarity at all search points has been completed, the process ends.

  In the present invention, some or all of the processing functions of the methods shown in FIGS. 4 and 5 can be configured as a program and executed by a computer. In addition, a program for causing a computer to execute the processing procedure is recorded on a recording medium readable by the computer, for example, a flexible disk, an MO, a ROM, a memory card, a CD, a DVD, a removable disk, and stored. Or can be distributed, and can be distributed via a communication network such as the Internet.

1 is an overall configuration diagram showing an embodiment of the present invention. An example of a cutout area and cutout area selection in the embodiment. The example of the weighting of the histogram in embodiment. The flowchart (the 1) which shows the process sequence in embodiment. The flowchart (the 2) which shows the process sequence in embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image conversion means 102 Area extraction means 103 Reliability calculation means 104 Object feature calculation means 105 Target image feature calculation means 106 Object extraction means

Claims (8)

An object area extraction device for extracting an area where an unknown object exists in an image,
Image conversion means capable of converting from an image to a color system having image saturation, hue, and luminance;
Assuming that an object exists near the center of the target image, a region cutout of a rectangular region of a predetermined size from the vicinity of the center of the image information indicated by the saturation, hue, and luminance calculated based on the image conversion means Means,
The saturation, luminance information, and imaging system characteristics of an appropriately sized rectangular area near the center of the image cut out by the area cutting means, and the hue and luminance changes in the area of the image from the image format are determined. A reliability calculation means for calculating the reliability based on;
A feature in which the hue of the color system converted by the image conversion unit is quantized and histogrammed is calculated, and the hue of the image is histogrammed when the reliability of the hue calculated by the reliability calculation unit is low. In addition, an object feature calculation unit that calculates the luminance histogram by quantizing the luminance of the image, and calculates the feature of the region cut out by the region cut-out unit as one histogram feature by combining the hue histogram and the luminance histogram;
A feature in which the hue of the color system converted by the image conversion unit is quantized and histogrammed is calculated, and the hue of the image is histogrammed when the reliability of the hue calculated by the reliability calculation unit is low. In addition, the target image feature calculation means for calculating the brightness histogram by quantizing the brightness of the image, calculating the feature of the entire target image as one histogram feature by combining the hue histogram and the brightness histogram,
An object extraction unit that searches for an object from a target image based on features obtained from the object feature calculation unit and the target image feature calculation unit, and extracts an area where the object exists from the image;
An object area extraction apparatus comprising:   The object feature calculation unit and the target image feature calculation unit according to claim 1, wherein when the reliability of the hue of the image is low, in addition to the feature histogram, the edge strength and / or the variance of the edge strength is calculated from the luminance. An object area extracting apparatus, wherein the object extracting means extracts an object as a histogram together with a hue histogram.   3. The feature calculation unit according to claim 1, wherein the weight of the hue and the luminance is changed by changing a quantization step size of the histogram of the hue and the luminance depending on the reliability of the hue when generating the histogram feature. An object region extraction apparatus characterized by extracting features. An object area extraction method for extracting an area where an unknown object exists in an image,
An image conversion process that can convert an image into a color system having the saturation, hue, and brightness of the image,
Assuming that an object exists near the center of the target image, a region cutout of a rectangular region of a predetermined size from the vicinity of the center of the image information indicated by the saturation, hue, and luminance calculated based on the image conversion process Process,
The saturation, luminance information, and characteristics of the imaging system in the vicinity of the center of the image extracted by the region extraction process, the characteristics of the imaging system, and the hue and luminance changes in the region of the image from the image format are determined. A reliability calculation process for calculating the reliability based on
A feature in which the hue of the color system converted by the image conversion process is quantized and histogrammed is calculated, and the hue of the image is histogrammed when the reliability of the hue calculated by the reliability calculation process is low In addition, an object feature calculation process that calculates a luminance histogram by quantizing the luminance of the image, and calculates a feature of the region extracted by the region extraction process as one histogram feature by combining the hue histogram and the luminance histogram;
A feature in which the hue of the color system converted by the image conversion process is quantized and histogrammed is calculated, and the hue of the image is histogrammed when the reliability of the hue calculated by the reliability calculation process is low In addition, a luminance histogram is calculated by quantizing the luminance of the image, and a target image feature calculation process for calculating a feature of the entire target image as one histogram feature by combining the hue histogram and the luminance histogram,
An object extraction step of searching for an object from a target image based on the feature obtained from the object feature calculation step and the target image feature calculation step, and extracting an area where the object exists from the image;
An object region extraction method comprising:   5. In the object feature calculation process and the target image feature calculation process according to claim 4, when the reliability of the hue of the image is low, the edge strength and / or the dispersion of the edge strength is calculated from the luminance in addition to the feature histogram. An object region extraction method comprising extracting an object in the object extraction process as one histogram together with a hue histogram.   6. In the feature calculation process according to claim 4, the hue and brightness weights are changed by changing the quantization step size of the hue and brightness histogram depending on the reliability of the hue when generating the histogram feature. An object region extraction method characterized by extracting features.   A program characterized in that the object region extraction method according to any one of claims 4 to 6 is described by a computer program and can be executed.   A recording medium recorded with a computer-readable program configured to execute the object region extraction method according to any one of claims 4 to 6 by a computer.

Images