JP6614658B2 - Color recognition apparatus, color recognition method, and program - Google Patents

Description

  The present invention relates to a color recognition apparatus and a color recognition method for recognizing a specific color, and further relates to a program for realizing these.

  In recent years, extraction of a specific color portion from an image has been performed in various fields. For example, in order to detect a human hand or face, a region is extracted from an image (see, for example, Patent Document 1).

  Specifically, Patent Document 1 discloses a hand shape recognition device that recognizes a human hand by extracting a skin color region on a screen. The hand shape recognition device disclosed in Patent Document 1 calculates a correlation value with a template value of a skin color created in advance for an input image, and based on the correlation value, determines the skin color of the skin color from the input image. Extract regions.

  Further, in the hand shape recognition device disclosed in Patent Document 1, the template value is given as RGB data and is updated as appropriate. The template value is updated by obtaining the average of the extracted skin color area average and the template value, and setting the obtained value as the latest template value. For this reason, according to the hand shape recognition device disclosed in Patent Document 1, it is considered that the skin color region can be accurately extracted while reducing the burden on the system.

JP-A-11-167455

  However, since an auto white balance function is usually added to a camera serving as an image means, the brightness of a skin color region in an image greatly varies depending on the environment where the image is taken. For this reason, in the hand shape recognition device disclosed in Patent Document 1, the detection accuracy is greatly reduced, for example, in a situation where the subject moves from the sun to the shade, although the template value is appropriately executed. May end up.

  An object of the present invention is to provide a color recognition device, a color recognition method, and a program capable of solving the above-described problem and detecting a specific color with high accuracy without being influenced by a shooting environment.

In order to achieve the above object, a color recognition apparatus according to one aspect of the present invention is an apparatus for recognizing a specific color region in a target image,
From the calibration image having the specific color area, the R value, the G value, and the B value are specified for each pixel constituting the area, and each value is multiplied by a plurality of coefficients set in stages. An extended array creating unit for creating an extended array of the calibration images;
An array conversion unit that converts the extended array of the calibration image into an array defined by hue, saturation, and brightness;
A histogram creation unit that creates a two-dimensional histogram from the expanded array of the calibration images after conversion, with one axis as hue and the other axis as saturation;
An array of the target image defined by hue, saturation, and brightness is created from the R value, G value, and B value of each pixel constituting the target image, and the created array of the target image and the histogram And a color recognition unit for recognizing the specific color region in the target image,
It is characterized by having.

In order to achieve the above object, a color recognition method according to one aspect of the present invention is a method for recognizing a specific color region in a target image,
(A) A R value, a G value, and a B value are specified for each pixel constituting the calibration image having the specific color region, and a plurality of coefficients set in stages for each value. Multiplying to create an expanded array of the calibration images; and
(B) converting the expanded array of the calibration images into an array defined by hue, saturation, and brightness;
(C) creating a two-dimensional histogram from the expanded array of the converted calibration images, with one axis as hue and the other axis as saturation;
(D) An array of the target image defined by hue, saturation, and brightness is created from the R value, G value, and B value of each pixel constituting the target image, and the created target image array Recognizing a region of the specific color in the target image using the histogram and the histogram; and
It is characterized by having.

Furthermore, in order to achieve the above object, a program according to one aspect of the present invention is a program for recognizing a specific color region in a target image by a computer,
In the computer,
(A) R value, G value, and B value are specified for each pixel constituting the calibration image having the specific color area, and a plurality of coefficients set stepwise for each value Multiplying to create an expanded array of the calibration images; and
(B) converting the expanded array of the calibration images into an array defined by hue, saturation, and brightness;
(C) creating a two-dimensional histogram from the expanded array of the converted calibration images, with one axis as hue and the other axis as saturation;
(D) An array of the target image defined by hue, saturation, and brightness is created from the R value, G value, and B value of each pixel constituting the target image, and the created target image array Recognizing a region of the specific color in the target image using the histogram and the histogram; and
Is executed.

  As described above, according to the present invention, it is possible to accurately detect a specific color without being influenced by the shooting environment.

FIG. 1 is a block diagram showing a schematic configuration of a color recognition apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing a specific configuration of the color recognition apparatus according to the embodiment of the present invention. FIG. 3 is a diagram showing an example of a calibration image used in the embodiment of the present invention. FIG. 4 is a diagram showing an example of the extended array created in the embodiment of the present invention. FIG. 5 is a diagram showing an example of a lookup table used in the embodiment of the present invention. FIG. 6 is a flowchart showing an operation during the calibration process of the color recognition apparatus according to the embodiment of the present invention. FIG. 7 is a flowchart showing the operation during the recognition processing of the color recognition apparatus in the embodiment of the present invention. FIG. 8 is a block diagram illustrating an example of a computer that implements the color recognition apparatus according to the embodiment of the present invention.

(Embodiment)
Hereinafter, a color recognition device, a color recognition method, and a program according to an embodiment of the present invention will be described with reference to FIGS.

[Device configuration]
First, the configuration of the color recognition apparatus according to the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of a color recognition apparatus according to an embodiment of the present invention.

  A color recognition apparatus 100 according to the present embodiment shown in FIG. 1 is an apparatus for recognizing a specific color area in a target image. As shown in FIG. 1, the color recognition apparatus includes an extended array creation unit 10, an array conversion unit 20, a histogram creation unit 30, and a color recognition unit 40.

  The extended array creation unit 10 identifies the R value, the G value, and the B value for each pixel constituting the calibration image having a specific color area, and is set stepwise for each value. Multiply multiple coefficients to create an expanded array of calibration images.

  The array conversion unit 20 converts the extended array of calibration images created by the extended array creation unit 10 into an array defined by hue, saturation, and brightness. The histogram creation unit 30 creates a two-dimensional histogram from the expanded array of converted calibration images with one axis as hue and the other axis as saturation.

  First, the color recognition unit 40 creates an array of target images defined by hue, saturation, and brightness from the R value, G value, and B value of each pixel constituting the target image. Subsequently, the color recognizing unit 40 recognizes a specific color region in the target image by using the created array of target images and the histogram created by the histogram creating unit 30.

  As described above, in this embodiment, first, since the histogram is created from the calibration image, the bin emphasizes a specific color. Further, when the histogram is created, the RGB values of the calibration image are expanded and the color space is converted thereon, so that the histogram corresponds to a change due to a specific color environment. For this reason, according to the present embodiment, it is possible to accurately recognize a specific color without being influenced by the shooting environment.

  Next, the configuration of the color recognition apparatus in the embodiment of the present invention will be described more specifically with reference to FIG. FIG. 2 is a block diagram showing a specific configuration of the color recognition apparatus according to the embodiment of the present invention. Further, hereinafter, a case where the specific color to be recognized is a skin color will be described as an example.

  As shown in FIG. 2, the color recognition apparatus 100 according to the present embodiment includes a calibration processing unit in addition to the above-described extended array creation unit 10, array conversion unit 20, histogram creation unit 30, and color recognition unit 40. 50, a storage unit 60, and an image acquisition unit 70.

  Among these, the image acquisition unit 70 receives the image data of the target image (hereinafter referred to as “target image data”) and the image data that is the basis of the calibration image from the outside. To do. Further, when the target image data is acquired, the image acquisition unit 70 inputs the target image data to the color recognition unit 40.

  Furthermore, when the image data that is the basis of the calibration image is input, the image acquisition unit 70 extracts only the image data of the portion that becomes the calibration image from the image data. Then, the image acquisition unit 70 inputs the extracted image data (hereinafter referred to as “calibration image data”) to the extended array creation unit 10.

  In the present embodiment, for example, as shown in FIG. 3, when image data that is the basis of a calibration image is input, the image acquisition unit 70 displays an image 71 specified by the image data. It can also be displayed on the screen of a display device (not shown in FIG. 2). In this case, as shown in FIG. 3, the image acquisition unit 70 further receives designation of a part to be the calibration image 72, extracts image data of the designated part (calibration image 72), and calibrates this. The image data is input to the extended array creation unit 10 as image data.

  FIG. 3 is a diagram showing an example of a calibration image used in the embodiment of the present invention. In the example of FIG. 3, since the specific color is a skin color, the image 71 is an image in which a human hand is the subject. Further, as the calibration image 72, an image of a part of the palm is used.

  Further, when the calibration image data is input, the extended array creation unit 10 specifies the R value, the G value, and the B value for each pixel from the calibration image data, and gradually increases the specified value to each specified value. Multiply the set coefficient. Specifically, in the present embodiment, as shown in FIG. 4, the extended array creation unit 10 uses coefficients that are in the range of 0.7 to 1.3 for each of the R value, the G value, and the B value. Is multiplied by 0.05 to create an expanded array.

  FIG. 4 is a diagram showing an example of the extended array created in the embodiment of the present invention. In the example of FIG. 4, only the extended array for one pixel is shown. For example, if the number of pixels of the calibration image is 50 × 50, the size of the expanded array is 50 × 50 × 3 × 13. Moreover, although the above-mentioned coefficient is an example when skin color is recognized, the interval between coefficients and the number of coefficients are appropriately set according to the performance of the color recognition apparatus 100.

  Furthermore, in the present embodiment, the coefficient to be multiplied is set according to the color to be recognized. For example, each coefficient is set by setting only a coefficient to be multiplied by a specific value or increasing the value of the coefficient to be multiplied by the specific value according to the composition of the wavelength component. Specifically, if a color with many red wavelength components is to be recognized, the coefficient value multiplied by the R value is increased, or only the coefficient multiplied by the R value is provided. Each coefficient is set.

  In the present embodiment, the array conversion unit 20 can convert an extended array into an array defined by hue, saturation, and brightness by using a conversion process from a general RGB space to an HSV space. it can.

In the present embodiment, the histogram creation unit 30 removes the lightness component from the array after conversion, and creates a two-dimensional histogram using the remaining components. Further, at this time, the histogram creation unit 30 can remove bins whose hue values are equal to or less than the threshold or less than the threshold from the created histogram in order to remove noise. Specifically, the histogram creation unit 30 identifies bins where the hue value is less than or less than the threshold value, and sets the hue value of the identified bin to zero.

  The calibration processing unit 50 obtains a covariance matrix using the histogram, and calculates a Mahalanobis distance for each combination of the hue value and the saturation value in the calibration image using the obtained covariance matrix.

Specifically, when the hue value in each bin of the histogram is H _i and the saturation value is S _j , the coordinates (polar coordinate system) on the HS plane indicated by H _i and S _j are orthogonal coordinates. When the converted value is an input vector x _k , the number of input vectors is n, and the average vector of the input vectors is μ, the calibration processing unit 50 obtains a covariance matrix Σ using the following formula 1, and the covariance matrix Σ obtained by applying the following equation 2, H _i and S combination of _j (H _{i, S j)} can be calculated Mahalanobis distance D for each. Note that i and j are arbitrary integers from 0 to 255, respectively.

  The calibration processing unit 50 stores the calculated Mahalanobis distances in a two-dimensional array (hereinafter referred to as “look-up table”) defined by hue and saturation. In the present embodiment, the lookup table in which the Mahalanobis distance is stored is stored in the table storage unit 60. FIG. 5 is a diagram showing an example of a lookup table used in the embodiment of the present invention.

  In the present embodiment, the color recognition unit 40 creates an array of target images using a conversion process from the RGB space to the HSV space in the same manner as the array conversion unit 20 described above. In this embodiment, the color recognition unit 40 collates the created array of target images with the above-described lookup table, and specifies the Mahalanobis distance for each pixel constituting the target image. Subsequently, the color recognition unit 40 recognizes a specific color region in the target image based on the specified Mahalanobis distance.

  Specifically, the color recognition unit 40 specifies the hue value H and the saturation value S from the array of the target image for each pixel of the target image, and further specifies the specified value (H, S). The Mahalanobis distance corresponding to the specified value (H, S) is specified by collating with the lookup table. Subsequently, for each pixel, the color recognition unit 40 compares the specified Mahalanobis distance with a set threshold value, specifies a pixel whose Mahalanobis distance is equal to or less than the threshold value, and specifies an area composed of the specified pixels. It is determined that the color area.

[Device operation]
Next, the operation of the color recognition apparatus 100 according to the embodiment of the present invention will be described with reference to FIGS. In the following description, FIGS. 1 to 5 are referred to as appropriate. In the present embodiment, the color recognition method is implemented by operating the color recognition apparatus 100. Therefore, the description of the color recognition method in the present embodiment is replaced with the following description of the operation of the color recognition apparatus 100.

  First, the operation when the color recognition apparatus 100 executes the calibration process will be described with reference to FIG. FIG. 6 is a flowchart showing an operation during the calibration process of the color recognition apparatus according to the embodiment of the present invention.

  As shown in FIG. 6, first, the image acquisition unit 70 acquires image data that is the basis of a calibration image (step A1). Specifically, in the present embodiment, when image data of a human hand or face image data is transmitted from an external device (not shown), the image acquisition unit 70 is transmitted. Get image data.

  Next, the image acquisition unit 70 extracts image data of a portion to be a calibration image from the image data acquired in step A1 (step A2). Specifically, as shown in FIG. 3, the image acquisition unit 70 displays an image 71 specified by the image data acquired in step A <b> 1 on the screen of the display device, and specifies a portion to be the calibration image 72. When the designation is performed, the image data of the designated portion is extracted.

  Next, the extended array creation unit 10 acquires the R value, G value, and B value of each pixel of the image data extracted in step A2 (step A3). Subsequently, the expanded array creating unit 10 creates an expanded array by multiplying each of the R value, G value, and B value of each pixel by a coefficient set in stages (step A4).

  Next, the array conversion unit 20 converts the color space of the extended array from the RGB space to the HSV space, and creates an array defined by the pigment, saturation, and lightness (step A5).

  Next, the histogram creating unit 30 removes the lightness component from the array in which the color space is converted, and calculates a two-dimensional histogram based on hue and saturation (step A6). In Step A6, in order to remove noise, the histogram creation unit 30 identifies a bin whose hue value is less than or less than a threshold value, and sets the hue value of the identified bin to zero.

  Next, the calibration processing unit 50 obtains a covariance matrix using the histogram created in step A6 (step A7). In step A7, a color space corresponding to the skin color is specified.

  Thereafter, the calibration processing unit 50 calculates the Mahalanobis distance for each combination of the hue value and the saturation value, and stores the obtained Mahalanobis distance in the lookup table (step A8). Since the look-up table is completed by executing step A8, the calibration process ends.

  Next, an operation when the color recognition apparatus 100 executes the recognition process will be described with reference to FIG. FIG. 7 is a flowchart showing the operation during the recognition processing of the color recognition apparatus in the embodiment of the present invention.

  As shown in FIG. 7, first, the image acquisition unit 70 acquires image data (target image data) of an image to be recognized (step B1). Specifically, in the present embodiment, when target image data is transmitted from an external device (not shown), the image acquisition unit 70 acquires the transmitted image data.

Next, the color recognition unit 40 acquires the R value, the G value, and the B value of each pixel of the image data acquired in step B1 (step B2). Subsequently, the color recognition unit 40 performs a conversion process from the RGB space to the HSV space based on the acquired R value, G value, and B value, and the target image defined by the hue, saturation, and brightness. Create an array of (Step B3).

  Next, the color recognizing unit 40 specifies a hue value H and a saturation value S for each pixel of the target image from the array obtained in step B3. Subsequently, the color recognition unit 40 collates the specified value (H, S) with the lookup table obtained in step A8, and specifies the Mahalanobis distance corresponding to the specified value (H, S). (Step B4).

  Next, the color recognition unit 40, for each pixel, compares the Mahalanobis distance specified in step B4 with the set threshold value, specifies a pixel whose Mahalanobis distance is equal to or less than the threshold value, and includes the specified pixels. Is determined to be a specific color area (step B5).

  As described above, in the present embodiment, the RGB space of the calibration image is expanded, and a histogram necessary for recognition of a specific color is generated from the expanded RGB space. Therefore, it becomes possible to recognize a specific color with high accuracy.

  In the present embodiment, the Mahalanobis distance for each set of hue and saturation is registered in the lookup table in advance, and at the time of recognition, the Mahalanobis distance for each pixel is obtained simply by obtaining the hue and saturation of each pixel. Is immediately determined, and it can be immediately determined whether each pixel has a specific color. For this reason, it is possible to reduce the load in the recognition process of a specific color.

  In the present embodiment, the calibration process may be completed in steps A1 to A6. In this case, in the recognition process, after execution of step B3, steps A7 and A8 are executed, and thereafter steps B4 and B5 are executed.

[program]
The program in the embodiment of the present invention may be a program that causes a computer to execute steps A1 to A8 shown in FIG. 6 and steps B1 to B5 shown in FIG. By installing and executing this program on a computer, the color recognition apparatus and the color recognition method in the present embodiment can be realized. In this case, a central processing unit (CPU) of the computer functions as an extended array creation unit 10, an array conversion unit 20, a histogram creation unit 30, a color recognition unit 40, a calibration processing unit 50, and an image acquisition unit 70, and performs processing. Do.

  Note that the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as the extended array creation unit 10, the array conversion unit 20, the histogram creation unit 30, the color recognition unit 40, the calibration processing unit 50, and the image acquisition unit 70, respectively. The storage unit 60 may be constructed on a computer different from the computer that executes the program in the present embodiment.

  Here, a computer that realizes the color recognition apparatus 100 by executing the program according to the present embodiment will be described with reference to FIG. FIG. 8 is a block diagram illustrating an example of a computer that implements the color recognition apparatus according to the embodiment of the present invention.

  As shown in FIG. 8, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These units are connected to each other via a bus 121 so that data communication is possible.

The CPU 111 performs various calculations by developing the program (code) in the present embodiment stored in the storage device 113 in the main memory 112 and executing them in a predetermined order. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Further, the program in the present embodiment is provided in a state of being stored in a computer-readable recording medium 120. Note that the program in the present embodiment may be distributed on the Internet connected via the communication interface 117.

  Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and a mouse. The display controller 115 is connected to the display device 119 and controls display on the display device 119.

  The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, and reads a program from the recording medium 120 and writes a processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.

  Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic storage media such as a flexible disk, or CD- An optical storage medium such as ROM (Compact Disk Read Only Memory) can be used.

  As described above, according to the present invention, it is possible to accurately detect a specific color, for example, a skin color, regardless of the shooting environment. The present invention is useful for a system that needs to recognize a specific color, for example, a system that performs input by hand recognition, a system that recognizes a human face, and the like.

DESCRIPTION OF SYMBOLS 10 Extended arrangement | sequence creation part 20 Array conversion part 30 Histogram creation part 40 Color recognition part 50 Calibration process part 60 Memory | storage part 70 Image acquisition part 100 Color recognition apparatus 110 Computer 111 CPU
112 Main Memory 113 Storage Device 114 Input Interface 115 Display Controller 116 Data Reader / Writer 117 Communication Interface 118 Input Device 119 Display Device 120 Recording Medium 121 Bus

Claims (12)

A device for recognizing a specific color area in a target image,
From the calibration image having the specific color area, the R value, the G value, and the B value are specified for each pixel constituting the area, and each value is multiplied by a plurality of coefficients set in stages. An extended array creating unit for creating an extended array of the calibration images;
An array conversion unit that converts the extended array of the calibration image into an array defined by hue, saturation, and brightness;
A histogram creation unit that creates a two-dimensional histogram from the expanded array of the calibration images after conversion, with one axis as hue and the other axis as saturation;
An array of the target image defined by hue, saturation, and brightness is created from the R value, G value, and B value of each pixel that constitutes the target image, and from the created array, for each pixel Identifying a hue value and a saturation value, and using the hue value and the saturation value specified for each pixel and the two-dimensional histogram, the specific color of the target image A color recognition unit that recognizes an area;
A color recognition apparatus comprising: A covariance matrix is obtained using the two-dimensional histogram, and a Mahalanobis distance for each combination of a hue value and a saturation value in the calibration image is calculated using the obtained covariance matrix. A calibration processing unit for storing each Mahalanobis distance in a two-dimensional array defined by hue and saturation;
The color recognition unit collates the hue value and the saturation value specified for each pixel with the two-dimensional array, specifies the Mahalanobis distance for each pixel constituting the target image, and specifies Recognizing the specific color region in the target image based on Mahalanobis distance;
The color recognition apparatus according to claim 1. The histogram creation unit removes bins whose hue values are less than or less than a threshold value from the two-dimensional histogram,
The calibration processing unit obtains a covariance matrix using the two-dimensional histogram from which the bins have been removed;
The color recognition apparatus according to claim 2.   The color recognition device according to claim 1, wherein the plurality of coefficients are set according to the specific color. A method for recognizing a specific color area in a target image,
(A) A R value, a G value, and a B value are specified for each pixel constituting the calibration image having the specific color region, and a plurality of coefficients set in stages for each value. Multiplying to create an expanded array of the calibration images; and
(B) converting the expanded array of the calibration images into an array defined by hue, saturation, and brightness;
(C) creating a two-dimensional histogram from the expanded array of the converted calibration images, with one axis as hue and the other axis as saturation;
(D) creating an array of the target image defined by hue, saturation, and brightness from the R value, G value, and B value of each pixel constituting the target image, and from the created array, For each pixel, specify a hue value and a saturation value, and use the hue value and the saturation value specified for each pixel and the two-dimensional histogram to specify the specification in the target image. Recognizing the color region of the step,
A color recognition method characterized by comprising: (E) A covariance matrix is obtained using the two-dimensional histogram, and a Mahalanobis distance for each combination of hue value and saturation value in the calibration image is calculated using the obtained covariance matrix. And storing each calculated Mahalanobis distance in a two-dimensional array defined by hue and saturation,
In the step (d), the hue value and the saturation value specified for each pixel are collated with the two-dimensional array obtained in the step (e) to construct the target image. Identifying the Mahalanobis distance for each pixel to be recognized, and recognizing the specific color region in the target image based on the identified Mahalanobis distance;
The color recognition method according to claim 5. In the step (c), bins whose hue values are less than or less than a threshold value are removed from the two-dimensional histogram,
In the step (e), a covariance matrix is obtained using the two-dimensional histogram from which the bin has been removed.
The color recognition method according to claim 6.   The color recognition method according to claim 5, wherein the plurality of coefficients are set according to the specific color. A program for recognizing a specific color area in a target image by a computer,
In the computer,
(A) A R value, a G value, and a B value are specified for each pixel constituting the calibration image having the specific color region, and a plurality of coefficients set in stages for each value. Multiplying to create an expanded array of the calibration images; and
(B) converting the expanded array of the calibration images into an array defined by hue, saturation, and brightness;
(C) creating a two-dimensional histogram from the expanded array of the converted calibration images, with one axis as hue and the other axis as saturation;
(D) creating an array of the target image defined by hue, saturation, and brightness from the R value, G value, and B value of each pixel constituting the target image, and from the created array, For each pixel, specify a hue value and a saturation value, and use the hue value and the saturation value specified for each pixel and the two-dimensional histogram to specify the specification in the target image. Recognizing the color region of the step,
A program that executes In the computer,
(E) A covariance matrix is obtained using the two-dimensional histogram, and a Mahalanobis distance for each combination of hue value and saturation value in the calibration image is calculated using the obtained covariance matrix. And storing each calculated Mahalanobis distance in a two-dimensional array defined by hue and saturation,
In the step (d), the hue value and the saturation value specified for each pixel are collated with the two-dimensional array obtained in the step (e) to construct the target image. Identifying the Mahalanobis distance for each pixel to be recognized, and recognizing the specific color region in the target image based on the identified Mahalanobis distance;
The program according to claim 9. In the step (c), bins whose hue values are less than or less than a threshold value are removed from the two-dimensional histogram,
In the step (e), a covariance matrix is obtained using the two-dimensional histogram from which the bin has been removed.
The program according to claim 10. The program according to any one of claims 9 to 11, wherein the plurality of coefficients are set according to the specific color.

Images