JP2012094050A - Image processing method, and image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing method for extracting an object in an image.SOLUTION: A camera 100 includes: dividing means for dividing an image into plural sections in a color difference space, which is represented in two dimension, based on color difference value; binary image creating means for binarizing an image for each of the plural sections divided by the dividing means, and creating binary images; and object extracting means for extracting an object in an image based on the binary images created by the binary image creating means.

Description

  The present invention relates to an image processing method and an image processing apparatus.

  The following image processing apparatus is known. This image processing apparatus converts image data into image data expressed in color difference space coordinates, and then detects a specific color based on the converted image data (for example, Patent Document 1).

Japanese Patent No. 3716466

  However, when a specific color is detected based on the converted image data after converting the image data into image data represented by color difference space coordinates as in the conventional image processing apparatus, the discrimination of the intermediate color is good. Absent. For this reason, if an attempt is made to extract a subject in an image based on a specific color specified by a conventional method, the subject extraction accuracy may be reduced.

In the image processing method according to the present invention, in a color difference space expressed in two dimensions, a division procedure for dividing an image into a plurality based on a color difference value, and binarizing the image for each of the plurality of divisions divided in the division procedure A binarized image generating procedure for generating a binarized image; and a subject extracting procedure for extracting a subject from the image based on the binarized image generated by the binarized image generating procedure. .
An image processing apparatus according to the present invention binarizes an image for each of a plurality of sections divided by the classifying means and a classifying means for classifying the image into a plurality of colors based on the color difference value in a two-dimensional color difference space. A binarized image generating unit that generates a binarized image, and a subject extracting unit that extracts a subject from the image based on the binarized image generated by the binarized image generating unit. And

  According to the present invention, it is possible to improve the discrimination accuracy of intermediate colors and improve the accuracy of subject extraction.

It is a block diagram which shows the structure of one Embodiment of a camera. It is a figure which shows the specific example of a CIELAB chromaticity diagram. It is a figure which shows the specific example of the binarization division image produced based on the CIELAB chromaticity diagram. It is a flowchart which shows the flow of a subject specific process. It is a figure which shows the specific example at the time of dividing | segmenting into the color difference space 8 division. It is a figure which shows the example which produced the binarized image of 8 sheets by binarizing into 8 divisions by making a CIELAB chromaticity diagram into object image. It is a figure which shows the specific example at the time of dividing | segmenting into 16 divisions of color difference space.

  FIG. 1 is a block diagram illustrating a configuration of an embodiment of a camera according to the present embodiment. The camera 100 includes an operation member 101, a lens 102, an image sensor 103, a control device 104, a memory card slot 105, and a monitor 106. The operation member 101 includes various input members operated by the user, such as a power button, a release button, a zoom button, a cross key, an enter button, a play button, and a delete button.

  The lens 102 is composed of a plurality of optical lenses, but is representatively represented by one lens in FIG. The image sensor 103 is an image sensor such as a CCD or a CMOS, for example, and captures a subject image formed by the lens 102. Then, an image signal obtained by imaging is output to the control device 104.

  The control device 104 generates image data in a predetermined image format, for example, JPEG format (hereinafter referred to as “main image data”) based on the image signal input from the image sensor 103. Further, the control device 104 generates display image data, for example, thumbnail image data, based on the generated image data. The control device 104 generates an image file that includes the generated main image data and thumbnail image data, and further includes header information, and outputs the image file to the memory card slot 105. In the present embodiment, it is assumed that both the main image data and the thumbnail image data are image data expressed in the RGB color system.

  The memory card slot 105 is a slot for inserting a memory card as a storage medium, and the image file output from the control device 104 is written and recorded on the memory card. The memory card slot 105 reads an image file stored in the memory card based on an instruction from the control device 104.

  The monitor 106 is a liquid crystal monitor (rear monitor) mounted on the back surface of the camera 100, and the monitor 106 displays an image stored in a memory card, a setting menu for setting the camera 100, and the like. . Further, when the user sets the mode of the camera 100 to the shooting mode, the control device 104 outputs image data for display of images acquired from the image sensor 103 in time series to the monitor 106. As a result, a through image is displayed on the monitor 106.

  The control device 104 includes a CPU, a memory, and other peripheral circuits, and controls the camera 100. Note that the memory constituting the control device 104 includes SDRAM and flash memory. The SDRAM is a volatile memory, and is used as a work memory for the CPU to develop a program when the program is executed or as a buffer memory for temporarily recording data. The flash memory is a non-volatile memory in which data of a program executed by the control device 104, various parameters read during program execution, and the like are recorded.

  In the present embodiment, the control device 104 divides an image acquired based on image data captured by the image sensor 103 into a plurality of regions using colors in the image, and selects a specific color from the image. The subject is extracted.

  2. Description of the Related Art Conventionally, there is known a method of dividing an image into a plurality of regions by colors by dividing the CIELAB chromaticity diagram shown in FIG. 2 into a plurality of regions for each color. However, when the subject is extracted by dividing the image into a plurality of regions using this method, there is a problem that the subject extraction accuracy is not high because the discrimination of the intermediate colors is low. Hereinafter, the problems of the conventional method will be specifically described.

In the conventional method, the image data of the target image expressed in the RGB color system is converted into an image in the YCbCr format, and a Y component image (Y plane image), a Cr component image (Cr plane image), and Cb Component images (Cb plane images) are respectively generated. Specifically, the target image represented in the RGB color system is represented by a Y plane image that is a luminance image composed of luminance components (Y component) in the YCbCr color space using the following equations (1) to (3): Conversion into a Cb plane image and a Cr plane image, which are color difference images composed of color difference components (Cb component, Cr component). Also, a Y complement plane image is generated by inverting the white pixels and black pixels of the Y plane image.
Y = 0.299R + 0.587G + 0.114B (1)
Cb = −0.169R−0.332G + 0.500B (2)
Cr = 0.500R−0.419G−0.081B (3)

  Then, for each of the generated Y plane image, Cb plane image, Cr plane image, and Y complement plane image, the density value of all pixels in the image is examined, and the average value M of each density value and the standard deviation σ of each density And calculate. Further, each pixel of the Y plane image, the Cb plane image, the Cr plane image, and the Y complement plane image is binarized with an average value M + standard deviation σ, an average value M, and an average value M−standard deviation σ, respectively. Two binarized segment images are created.

  Thereby, 16 binarized segment images as shown in FIGS. 3A to 3P are generated based on the CIELAB chromaticity diagram shown in FIG. According to the binarized segment image shown in FIG. 3, in the binarized segment image for each plane image shown in a vertical row, there is little overlap of white pixel areas between the four images. When viewed as a whole, white pixel regions overlap in a large range between the 16 images, and the color discrimination is not good.

  For example, four binarized segment images for the Cb plane images shown in FIGS. 3 (e) to 3 (h) and four for the Cr plane images shown in FIGS. 3 (i) to 3 (l). Focusing on the two columns of the binarized segment images, the four binarized segment images shown in FIGS. 3 (f), (g), (j), and (k) have a central portion of a circle corresponding to an intermediate color. In this case, the white pixel areas are overlapped, and the distinguishability of intermediate colors is deteriorated.

  On the other hand, in the two binarized segmented images shown in FIGS. 3E and 3H, the white pixel region is divided into an upper side of a circle corresponding to a blue color and a lower side of a circle corresponding to a yellow color. There is no overlap. However, according to the two binarized segmented images, the blue color and the yellow color can be discriminated, but the red color and the green color cannot be discriminated. 3 (i) and (l), the white pixel region is divided into a right side of a circle corresponding to a red color and a left side of a circle corresponding to a green color. There is no overlap. However, according to the two binarized segmented images, the red color and the green color can be discriminated, but the blue color and the yellow color cannot be discriminated. Further, if the four binarized segment images of FIGS. 3E, 3H, 3I, and 1L are used, the blue color, the yellow color, the red color, and the green color are displayed. Each color can be discriminated, but the middle color at the center of the circle cannot be discriminated.

  As described above, in each of the Y plane image, the Cb plane image, the Cr plane image, and the Y complement plane image, in the conventional method using 16 binarized segment images, the blue color, the yellow color, The discrimination between intermediate colors other than red and green colors is poor, and when images are classified using colors by this method, there is a possibility that the accuracy of identifying a subject may be reduced.

  Therefore, in the present embodiment, the control device 104 executes the processing shown in FIG. 4 to improve the discrimination of intermediate colors and to divide the image area, thereby enabling to specify the subject with high accuracy. . FIG. 4 is a flowchart showing the flow of the subject specifying process in the present embodiment. The process shown in FIG. 4 is performed as a program that starts when a through image input from the image sensor 103 is started. 104.

  In step S10, the control device 104 sets the input through image as the target image, and converts the image data of the target image expressed in the RGB color system into image data in the YCbCr format using the equations (1) to (3). Convert. As a result, the above-described Y component image (Y plane image), Cr component image (Cr plane image), and Cb component image (Cb plane image) are generated. Then, it progresses to step S20.

  In step S20, the control device 104 binarizes the target image into 8 sections on the color difference space using the image data converted into the YCbCr format. In the present embodiment, as shown in FIG. 5, the control device 104 has eight color difference spaces (Cb-Cr spaces) expressed in two dimensions with the vertical axis representing the Cb value and the horizontal axis representing the Cr value. Divide into sections 5a to 5h. Then, the control device 104 prepares eight segment images having the same size as the target image in association with each of the segments 5a to 5h, and sets all the pixel values of the eight segment images to zero.

The control device 104 binarizes the target image into eight sections 5a to 5h using the following formulas (4) to (11) using pixel values of corresponding pixels of the Cb plane image and the Cr plane image. To do. That is, the control device 104 binarizes the target image for each of the sections 5a to 5h based on the magnitude relationship between the Cb value and the Cr value of the target image, the sign of the Cb value, and the sign of the Cr value. .
Cb ≧ 0 and Cr ≧ 0 and | Cr | ≧ | Cb | = section 5a (4)
Cb ≧ 0 and Cr ≧ 0 and | Cr | <| Cb | = section 5b (5)
Cb ≧ 0 and Cr <0 and | Cr | ≦ | Cb | = section 5c (6)
Cb ≧ 0 and Cr <0 and | Cr |> | Cb | = section 5d (7)
Cb <0 and Cr <0 and | Cr |> | Cb | = section 5e (8)
Cb <0 and Cr <0 and | Cr | ≦ | Cb | = section 5f (9)
Cb <0 and Cr ≧ 0 and | Cr | <| Cb | = section 5g (10)
Cb <0 and Cr ≧ 0 and | Cr | ≧ | Cb | = section 5h (11)

  Specifically, when the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (4), the control device 104 sets the pixel value of the corresponding pixel of the section image of the section 5a to 1 Change to Further, when the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (5), the control device 104 changes the pixel value of the corresponding pixel of the segment image of the segment 5b to 1. . Similarly, when the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (6), the control device 104 changes the pixel value of the corresponding pixel of the segment image of the segment 5c to 1. If the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (7), the pixel value of the corresponding pixel of the segment image of the segment 5d is changed to 1.

  In addition, when the pixel value of the corresponding pixel of the Cb plane image and the Cr plane image satisfies Expression (8), the control device 104 changes the pixel value of the corresponding pixel of the segment image of the segment 5e to 1. When the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (9), the pixel value of the corresponding pixel of the segment image of the segment 5f is changed to 1. In addition, when the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (10), the control device 104 changes the pixel value of the corresponding pixel of the segment image of the segment 5g to 1. When the pixel values of the corresponding pixels of the Cb plane image and the Cr plane image satisfy Expression (11), the pixel value of the corresponding pixel of the segment image of the segment 5h is changed to 1.

  The control device 104 performs determination using the above formulas (4) to (11) for all corresponding pixels of the Cb plane image and the Cr plane image, so that eight images are obtained based on the eight divided images. The binarized image is generated. Thereby, the target image can be binarized into 8 sections of sections 5a to 5h. FIG. 6 uses the CIELAB chromaticity diagram shown in FIG. 2 as a target image and binarizes it into 8 sections using equations (4) to (11), thereby creating eight binarized images 6a to 6h. An example is shown. In the eight binarized images shown in FIG. 6, white pixels do not overlap between all the binarized images, and all the pixels are 1 when the logical sum of the pixels is taken for all the images. Become.

  As shown in FIG. 6, in the eight binarized images 6a to 6h created by the method in the present embodiment, all colors of blue, yellow, red, and green can be distinguished. Since there is no overlap of white pixels even at the center of the circle, intermediate colors can also be distinguished.

  Thereafter, the process proceeds to step S30, and the control device 104 extracts a subject from the target image based on the eight binarized images 6a to 6h created by the above-described processing. As described above, if all the colors including the intermediate colors can be discriminated by the eight binarized images 6a to 6h, it is possible to accurately extract a subject having a specific color existing in the target image. .

  For example, when the color of the subject is red, the subject can be extracted from the binarized image 6h. That is, the position of the red subject in the target image can be specified by the position of the white pixel region in the binarized image 6h, and the size of the red subject can be determined based on the size and shape of the white pixel region. The shape can be specified. When the subject is green, the subject can be specified by the binarized images 6d, 6e, and 6f. That is, the white pixel regions of the binarized images 6d, 6e, and 6f are combined, and the position, size, and shape of the green subject in the target image are based on the position, size, and shape of the combined white pixel region. Can be specified. The control device 104 can extract a subject having a specific color from the target image based on the position, size, and shape of the subject thus identified.

  Note that the color of the subject to be specified may be specified in advance by the user, or the control device 104 prompts the user to photograph the subject in advance, and the subject is based on the photographed image as a result. The color may be specified.

  Thereafter, the process proceeds to step S40, and the control device 104 determines whether or not the input of the through image has been completed. If a negative determination is made in step S40, the process returns to step S10, and the above-described processing is executed for the next frame. As a result, it is possible to extract a subject in each frame of a through image input in time series and track the subject between frames. On the other hand, when an affirmative determination is made in step S40, the process ends.

According to the present embodiment described above, the following operational effects can be obtained.
(1) The control device 104 divides the target image into eight sections 5a to 5h based on the color difference values in a two-dimensional color difference space with the vertical axis representing the Cb value and the horizontal axis representing the Cr value. A target image is binarized for each section to generate a binarized image. Then, the control device 104 extracts a subject from the target image based on the generated binarized image. Accordingly, it is possible to improve the discrimination of intermediate colors and extract a subject of a specific color from the target image with high accuracy.

(2) The control device 104 binarizes the target image for each section based on the magnitude relationship between the Cb value and the Cr value, the sign of the Cb value, and the sign of the Cr value. As a result, a binarized image can be generated so that white pixels do not overlap between all the binarized images, so that all of the blue, yellow, red, and green colors can be generated by the binarized image. The color can be discriminated, and the discrimination of intermediate colors can also be improved.

-Modification-
The camera according to the above-described embodiment can be modified as follows.
(1) In the above-described embodiment, the control device 104 binarizes the target image into 8 sections on the color difference space using the image data converted into the YCbCr format according to the equations (4) to (11). An example was described. However, the target image may be binarized by dividing it into a number other than 8 in the color difference space. For example, as shown in FIG. 7, the control device 104 divides the color difference space (Cb-Cr space) into 16 sections 7a to 7p at equiangular intervals, and the following equations (12) to (27) The image may be binarized into 16 sections on the color difference space.

Cb> 0 and Cr ≧ 0 and Cb ≦ 0.5Cr = section 7a (12)
Cb> 0 and Cr ≧ 0 and Cr ≧ Cb> 0.5Cr = section 7b (13)
Cb> 0 and Cr ≧ 0 and 2Cr ≧ Cb> Cr = section 7c (14)
Cb> 0 and Cr ≧ 0 and Cb> 2Cr = section 7d (15)
Cb ≧ 0 and Cr <0 and Cb ≧ −2Cr = section 7e (16)
Cb ≧ 0 and Cr <0 and −Cr ≦ Cb <−2Cr = section 7f (17)
Cb ≧ 0 and Cr <0 and −0.5Cr ≦ Cb <−Cr = section 7g (18)
Cb ≧ 0 and Cr <0 and Cb <−0.5Cr = section 7h (19)
Cb <0 and Cr <0 and Cb ≧ 0.5Cr = section 7i (20)
Cb <0 and Cr <0 and Cr ≦ Cb <0.5Cr = section 7j (21)
Cb <0 and Cr <0 and 2Cr ≦ Cb <Cr = section 7k (22)
Cb <0 and Cr <0 and Cb <2Cr = section 7l (23)
Cb ≦ 0 and Cr ≧ 0 and Cb <−2Cr = section 7m (24)
Cb ≦ 0 and Cr ≧ 0 and −2Cr ≦ Cb <−Cr = section 7n (25)
Cb ≦ 0 and Cr ≧ 0 and −Cr ≦ Cb <−0.5Cr = section 7o (26)
Cb ≦ 0 and Cr ≧ 0 and Cb ≧ −0.5Cr = section 7p (27)

(2) In the above-described embodiment, the example in which the control device 104 executes the process illustrated in FIG. 4 for a through image and extracts a subject in each frame of the through image has been described. However, the subject may be extracted by executing steps S10 to S30 in FIG. 4 for one still image. Alternatively, the processing of FIG. 4 may be executed for each frame of moving image data composed of a plurality of frames, and a subject may be extracted from each frame constituting the moving image, and the subject may be tracked between the plurality of frames.

(3) In the above-described embodiment, the case where the present invention is applied to a camera has been described. However, the present invention can also be applied to other devices that can take an image, such as a portable terminal with a camera.

  Note that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired. Moreover, it is good also as a structure which combined the above-mentioned embodiment and a some modification.

100 Camera, 101 Operation member, 102 Lens, 103 Image sensor, 104 Control device, 105 Memory card slot, 106 Monitor

Claims (6)

A division procedure for dividing an image into a plurality of images based on a color difference value on a two-dimensional color difference space;
A binarized image generating procedure for binarizing the image for each of a plurality of sections divided by the sectioning procedure to generate a binarized image;
An image processing method comprising: a subject extraction procedure for extracting a subject from the image based on the binarized image generated by the binarized image generation procedure. The image processing method according to claim 1,
2. The image processing method according to claim 1, wherein the color difference space is a color difference space expressed two-dimensionally with a vertical axis representing a Cb value and a horizontal axis representing a Cr value. The image processing method according to claim 2,
The binarized image generation procedure binarizes the image for each of the plurality of sections based on the magnitude relationship between the Cb value and the Cr value, the sign of the Cb value, and the sign of the Cr value. Image processing method. Classification means for dividing an image into a plurality of images based on a color difference value on a two-dimensional color difference space;
Binarized image generating means for binarizing the image for each of a plurality of sections divided by the dividing means to generate a binarized image;
An image processing apparatus comprising: a subject extracting unit that extracts a subject from the image based on the binarized image generated by the binarized image generating unit. The image processing apparatus according to claim 4.
2. The image processing apparatus according to claim 1, wherein the color difference space is a color difference space expressed in two dimensions with a vertical axis representing a Cb value and a horizontal axis representing a Cr value. The image processing apparatus according to claim 5.
The binarized image generating means binarizes the image for each of the plurality of sections based on the magnitude relationship between the Cb value and the Cr value, the sign of the Cb value, and the sign of the Cr value. An image processing apparatus.

Images