JP2005121402A - Color image processing unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a color image processing unit capable of rapid detection processing even if the posture of a detection target is changed. <P>SOLUTION: While captured image data from an image sensor 2 are being transferred, only the color of a pixel is read from the captured image data, and a center-of-gravity position for each detection target color featuring at least two colors selected for the captured image simultaneously is computed, thus converting the posture of the captured image so as to coincide with that of a reference image from the center-of-gravity position, and determining whether the detection target M is conforming or not rapidly. Therefore, even if the posture of the detection target M is changed, it is not necessary to store a number of images and search a feature point as conventionally, thus achieving rapid detection processing. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a color image processing apparatus that detects the color of a detection object and determines the quality of the detection object, and more particularly to an apparatus that can determine in a short time even if the attitude of the detection object changes.

  2. Description of the Related Art Conventionally, there is known a color image processing apparatus that compares a captured image of a detection target with a reference image that is captured in advance and determines whether the detection target is good or bad. When the posture such as the position and angle of the detection object changes, either the captured image or the reference image is sequentially rotated and compared so that the images match.

In this case, it is necessary to store and compare a number of images that are sequentially rotated and moved, and there is a problem that the processing time is increased at the same time as the memory capacity increases. For this reason, a plurality of characteristic points of the reference image are registered, each feature point is searched from the captured image, an angle around the reference point of each feature point is obtained, and the angle is rotated. An image processing apparatus to be compared is known (for example, Patent Document 1).
Japanese Patent Laid-Open No. 11-53564

  However, in Patent Document 1, since it is not necessary to store a large number of images, the memory capacity is reduced. However, when searching for feature points, a plurality of pixels are made into blocks based on captured images stored in the memory. There is a problem in that it is necessary to search for blocks including feature points while sequentially shifting the blocks, which takes time and still takes processing time.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a color image processing apparatus that can solve the above-described problems and can perform a rapid detection process even if the posture of a detection object changes.

    In order to achieve the above object, a color image processing apparatus according to one configuration of the present invention includes an image sensor that captures a color image of a detection target, and pixels from captured image data of the detection target transferred from the image sensor. Color extraction means for reading out and extracting a color for each pixel, captured image storage means for storing a captured image of a detection object from which a color has been extracted for each pixel, and characteristics of two or more colors in the captured image A detection target color selecting unit that selects a detection target color and a detection target color for each detection target color based on a captured image of the detection target in which a color is extracted for each pixel during transfer of the captured image data from the image sensor. Based on the center-of-gravity position calculation means for calculating the center-of-gravity position, the center-of-gravity position of the captured image for each color to be detected, and the reference center-of-gravity position of the reference image registered in advance, the relative position of the captured image with respect to the reference image A captured image calculation means for calculating a relative angle and a relative size, and an imaging for converting the posture of the captured image to match the posture of the reference image based on the calculated relative position, relative angle and relative size. The image conversion unit includes a determination unit that compares the reference image and the converted captured image, calculates the degree of coincidence, and determines the quality of the detection target.

  According to this configuration, during the transfer of the captured image data from the image sensor, only the pixel color is read from the captured image data, and the barycentric position for each of two or more characteristic detection target colors selected for the captured image is simultaneously determined. By calculating, the orientation of the captured image can be converted to match the orientation of the reference image based on the center of gravity position and the reference center of gravity position, and the quality of the detection target can be quickly determined. Even if the posture of an object changes, it is not necessary to store a large number of images or search for feature points as in the conventional case, and a rapid detection process is possible.

  Preferably, the center-of-gravity position calculating means determines a detection target color for each pixel of the captured image, detects the positions of a plurality of points of the same color, and calculates the center-of-gravity position from the average value of the coordinates. Therefore, it is possible to accurately calculate the barycentric position for each color to be detected.

  Preferably, the reference image calculation means calculates a reference image from an intermediate position between the centroid positions of the detection target colors selected in the captured image and an intermediate position between the reference centroid positions of the detection target colors selected in the reference image. The relative position of the picked-up image with respect to the image is calculated, and the reference image is picked up from the arc tangent of each centroid position of the selected detection target color of the picked-up image and the arc tangent of each reference centroid position of the selected detection target color of the reference image. The relative angle of the image is calculated, and the distance between each centroid position of the selected detection target color of the captured image and the distance between each reference centroid position of the selected detection target color of the reference image are calculated. The relative size is calculated. Therefore, the relative position, relative angle, and relative size of the captured image with respect to the reference image can be accurately calculated.

  Preferably, the ratio of the three primary colors is calculated for each pixel during transfer from the image sensor according to the brightness change around the detection target, and white and black in which the ratio of the three primary colors is the same are detected by the calculation. Brightness correction means for correcting the luminance of each pixel by performing processing is provided. Therefore, by obtaining the ratio of the three primary colors for each pixel, the shadow of the detection target and the uneven brightness of the illumination are corrected, so that the luminance of each pixel is stably corrected.

A color image processing apparatus according to another configuration of the present invention reads an image sensor that captures a color image of a detection object and captured image data of the detection object transferred from the image sensor, and extracts a color for each pixel. A color extraction unit that performs detection, a captured image storage unit that stores a captured image of a detection target in which a color is extracted for each pixel, and a detection target color selection that selects a detection target color having one characteristic in the captured image Centroid position calculating means for calculating the centroid position of the detection target color based on the captured image of the detection target object in which the color is extracted for each pixel during transfer of the captured image data from the image sensor; Based on the number-of-pixels measuring means for measuring the number of pixels of the detection target color, the center of gravity position of the captured image of the detection target color, and the reference center of gravity position of the reference image registered in advance, the phase of the captured image with respect to the reference image A captured image calculation means for calculating a position and calculating a relative size of the captured image with respect to the reference image based on the number of pixels of the captured image of the detection target color and the reference pixel number of the reference image registered in advance; Captured image conversion means for converting the orientation of the captured image to match the orientation of the reference image based on the calculated relative position and relative size;
A determination unit that compares the reference image with the captured image after conversion, calculates the degree of coincidence, and determines whether the detection target is good or bad is provided.

  According to this configuration, during the transfer of the captured image data from the image sensor, only the color of the pixel is read from the captured image data, and at the same time, the barycentric position and the number of pixels of the characteristic detection target color selected for the captured image By calculating, the orientation of the captured image matches the orientation of the reference image based on the center of gravity position, the reference center of gravity position, the number of pixels, and the number of reference pixels when only one characteristic detection target color exists. Similarly, even if the posture of the detection target changes, a large number of images can be stored or feature points can be stored as in the past. There is no need to search, and a quick detection process is possible.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a color image processing apparatus according to the first embodiment of the present invention. The apparatus 1 is provided on a production line, for example, a label (FIG. 3) having two colors (for example, red and blue) in which a detection object M is pasted on a bottle, and the detection objects having two or more selected colors. Based on the color, the orientation of the captured image is matched with the orientation of the reference image, and a non-defective product or a defective product is discriminated.

  The apparatus 1 captures a color image of the detection object M while scanning, for example, an image sensor 2 such as a CMOS image sensor, and reads from the captured image data of the detection object M transferred from the image sensor 2. For each pixel, a color extraction unit 3 is provided for extracting the color by converting the color of the detection object M into, for example, three primary colors of R (red), G (green), and B (blue). In this case, only the color of the pixel is read from the captured image data. The color extracting unit 3 extracts each color from the luminance of the color designated by each pixel of the captured image based on the detected hue and the color margin. The detected hue is calculated for each specified color by the sum of the brightness of the specified color / the brightness of the three primary colors. For example, the brightness of red is 100, the brightness of green is 0, the brightness of blue is 0, and red is specified. In this case, 100 / (100 + 0 + 0) = 1.

  In addition, the apparatus 1 includes a captured image storage unit (captured image memory) 4, a gravity center position calculation unit 7, a captured image calculation unit 8, a captured image conversion unit 9, a determination unit 12, and a display unit 14. The CPU 10 has, for example, a detection target color selection unit 5 that selects a detection target color that has characteristics of two colors in the captured image, that is, is different and can be distinguished from the background, and a predetermined position, angle, and size of the detection target. And a reference image registration means 6 for registering in advance a reference image having a posture composed of and a reference barycentric position for each color to be detected in the reference image. Further, the apparatus 1 includes brightness correction / color ratio calculation means 13, detection target color range detection means 15 for detecting each detection target color range selected by the detection target color selection means 5 in the captured image, A pixel number measuring unit 17 that measures the number of pixels of the detection target color and a reference image memory 16 that stores a reference image registered in advance by the reference image registration unit 6 and a reference centroid position for each detection target color are provided.

  In the captured image memory 4, a captured image of the detection target M from which the color is extracted for each pixel by the color extraction unit 3 is written, and the captured image is stored. The brightness of each pixel is corrected by brightness correction / color ratio calculation means 13. The center-of-gravity position calculation means 7 is for each detection target color, that is, red, blue, based on the captured image of the detection target M from which the color has been extracted for each pixel during transfer of the captured image data from the image sensor 2. Each centroid position in the color range is calculated.

For example, the barycentric position calculation means 7 determines the color to be detected for each pixel of the captured image, detects the positions of a plurality of points of the same color (red or blue), and determines the barycentric position from the average value of the coordinates. It is to calculate. That is,
Center of gravity = Σ (detection target color coordinates) ÷ Σ (number of detection target color pixels)
Is calculated for each of the X coordinate and the Y coordinate.
In this case, while the captured image data from the image sensor 2 is being transferred, the color of each pixel of the captured image is sequentially captured, and if the pixel is a detection target color, the coordinates are sequentially added. Then, the calculation of the center of gravity position is completed by the above calculation at substantially the same timing as when the transfer of the captured image data ends.

  The captured image calculation means 8, after the transfer of the captured image data from the image sensor 2, the barycentric position of the calculated captured image for each color to be detected, and this read out from the reference image memory 16. Based on the reference centroid position of the reference image of the same color, the relative position, relative angle, and relative size of the captured image with respect to the reference image are calculated. Hereinafter, an example of the captured image calculation unit 8 will be described.

The relative position (movement amount) of the captured image with respect to the reference image from the intermediate position between the centroid positions of the selected detection target color of the captured image and the intermediate position between the reference centroid positions of the selected detection target color of the reference image Is calculated. For example,
Relative position = (intermediate position between each barycentric position of the two selected colors of the captured image)
-(Intermediate position between the reference centroid positions of the two selected colors of the reference image)
Can be obtained.

From the arc tangent (tan ⁻¹ ) of each centroid position of the selected detection target color of the captured image and the arc tangent of each reference centroid position of the selected detection target color of the registered image, the relative angle of the captured image with respect to the reference image ( Rotation angle) is calculated. For example,
Relative angle = (inverse tangent of each gravity center position of the two selected colors of the captured image)
-(Inverse tangent of each reference barycentric position of the two selected colors of the reference image)
Can be obtained.

The relative size (magnification) of the captured image with respect to the reference image is determined from the distance between the centroid positions of the selected detection target color of the captured image and the distance between the reference centroid positions of the selected detection target color of the reference image. Calculated. For example,
Relative size = (distance between each barycentric position of the two selected colors of the captured image)
÷ (Distance between each center of gravity of the selected two colors of the reference image)
Can be obtained.

The captured image conversion unit 9 matches the orientation of the captured image with the orientation of the reference image (original image) based on the relative position, relative angle, and relative size calculated by the captured image calculation unit 8. Convert. For example,
X coordinate after conversion = {{(COS (rotation angle) × original X coordinate) − (SIN (rotation angle) × original Y coordinate)} ÷ magnification ratio} + movement amount Y coordinate after conversion = {{( SIN (rotation angle) × original X coordinate) + (COS (rotation angle) × original Y coordinate)} ÷ magnification} + movement amount. The captured image after conversion is transferred to the display unit 14 when displayed on the display unit 14 and stored in the memory.

  The determination unit 12 compares the reference image read from the reference image memory 16 with the converted captured image, calculates the degree of coincidence, and based on a predetermined threshold value of coincidence, The quality of the detection object M is determined. The display unit 14 displays the determination result. Moreover, since the display part 14 also displays a reference | standard image and the converted captured image, it can judge the quality of the detection target M also visually.

    In addition, when the posture of the detection target M on the production line changes, the shadow of the detection target M changes accordingly, or when the illumination is applied to the detection target M, the brightness unevenness varies depending on the location of the illumination. If there is a change in shadow or uneven brightness of illumination, the same article may be erroneously detected as a different article. The brightness correction / color ratio calculation means 13 prevents this erroneous detection. The brightness correction / color ratio calculation means 13 includes a color ratio calculation unit that calculates the ratio of the three primary colors for each pixel, and a black and white detection processing unit that performs detection processing for white and black that have the same ratio of the three primary colors by the calculation. The two-stage processing of color ratio calculation and black and white detection processing is performed.

  The color ratio calculation unit calculates the color ratio of each pixel from the luminance of the three primary colors (RGB) of each pixel of the received image data, for example. The ratio of colors is obtained by the ratio of R (red) R / (R + G + B), the ratio of G (green) G / (R + G + B), and the ratio of B (blue) B / (R + G + B). For example, when the luminances of R, G, and B are 50, 100, and 0, respectively, the ratio of R is 50/150 = 1/3, the ratio of G is 100/150 = 2/3, and the ratio of B is 0. Become.

  Next, with only the color ratio calculation described above, 100 / (100 + 100 + 100) = 1/3 when one pixel is white, and 1 / (1 + 1 + 1) = 1/3 when one pixel is black. The ratio is exactly the same, and white and black cannot be detected. For this reason, the black and white detection processing unit calculates the total value of (R + G + B) based on a predetermined threshold, and corrects black without correcting white, for example, to detect white and black. Like to do. For example, when the total value of (R + G + B) is equal to or less than the threshold value 20, it is corrected to 20. Then, the ratio of the three primary colors of black becomes a correction value of 1/20, which can be distinguished from the ratio of the three primary colors of white which is a value of 1/3, and black can be detected.

  If the change in the posture of the detection object M does not cause a change in shadow or uneven brightness of the illumination, the brightness correction / color ratio calculation means 13 may not be provided.

  FIG. 2 is a time chart showing the operation of the color image processing apparatus having the above configuration. First, in step a, the captured image data is transferred from the image sensor 2, the color is extracted for each pixel, and the captured image is written in the captured image memory 4. At almost the same timing (slightly later), brightness correction and color ratio calculation are performed to correct the brightness of each pixel, and the detection of the range of each detection target color and the calculation of its center of gravity are performed.

  Next, in step b, the relative position, relative angle, and relative size of the captured image with respect to the reference image are calculated.

  In step c, the captured image is read from the captured image memory 4, and coordinate conversion of the captured image is performed based on the calculated relative position, relative angle, and relative size, and the converted captured image is The reference image is read into the memory of the display unit 14 and read out from the reference image memory 16. The degree of coincidence between the reference image and the converted captured image is calculated at substantially the same timing (slightly later), and the quality of the detection object M is determined.

  FIG. 3 is a diagram showing a coordinate conversion operation in step c. In (A), the captured image before coordinate conversion is taken into the memory according to the memory take-in direction after coordinate conversion. Then, the posture of the captured image in (A) matches the posture of the reference image shown in (B).

  As described above, during transfer of the captured image data from the image sensor 2, the present apparatus reads out the captured image data of the detection target and extracts a color for each pixel, and the barycentric position for each selected detection target color. Is calculated. Next, based on the centroid position for each color to be detected and the reference centroid position of the reference image, the relative position, relative angle, and relative size of the captured image with respect to the reference image are calculated, and the orientation of the captured image is changed to the orientation of the reference image. To match. Then, the quality of the detection target is determined from the degree of coincidence between the reference image and the converted captured image.

  Thereby, during the transfer of the captured image data from the image sensor 2, only the color of the pixel is read from the captured image data, and at the same time, the barycentric position is calculated for each of two or more characteristic detection target colors selected for the captured image. Thus, the orientation of the captured image can be converted to match the orientation of the reference image based on the center of gravity position and the reference center of gravity position, and the quality of the detection object can be quickly determined. Even if the posture changes, it is not necessary to store a large number of images or search for feature points as in the prior art, and a quick detection process is possible.

  Next, a color image processing apparatus according to the second embodiment of the present invention will be described. Unlike the first embodiment, the second embodiment is used when there is only one characteristic detection target color in the captured image. The detection target color selection unit 5, the gravity center position calculation unit 7, the imaging The image calculation means 8, the captured image conversion means 9, and the pixel number measurement means 17 are different from the first embodiment. Other configurations are the same as those in the first embodiment. The detection target color selection unit 5 of the second embodiment selects a detection target color having one color characteristic in the captured image, and the pixel number measurement unit 17 measures the number of pixels of the detection target color.

Further, the center-of-gravity position calculation unit 7 calculates the center of gravity of the detection target color (one color) based on the captured image of the detection target M from which the color is extracted for each pixel during the transfer of the captured image data from the image sensor 2. Calculate the position. That is,
Center of gravity = Σ (detection target color coordinates) ÷ Σ (number of detection target color pixels)
Is calculated by

The captured image calculation means 8 calculates the relative position of the captured image with respect to the reference image based on the centroid position of the captured image of the detection target color and the reference centroid position of the reference image registered in advance, and captures the detection target color. The relative size of the captured image with respect to the reference image is calculated based on the number of pixels of the image and the reference pixel number of the reference image registered in advance. That is,
Relative position = (centroid position of one color selected in the captured image)
-(Reference centroid position of one color selected in the reference image)
Relative size = √ {(number of pixels of one color selected in the captured image)
÷ (number of reference pixels of one color selected in the reference image)}
Can be obtained. The relative angle is 0 ° because it cannot be calculated when the detection target color is one color.

  The captured image conversion unit 9 converts the orientation of the captured image to match the orientation of the reference image based on the relative position and relative size calculated by the captured image calculation unit 8. Similarly, the determination unit 12 determines the quality of the detection object M from the reference image and the converted captured image.

  As a result, during the transfer of the captured image data from the image sensor 2, only the pixel color is read out from the captured image data, and at the same time, the barycentric position and the number of pixels of the characteristic detection target color selected for the captured image are calculated. By doing so, when only one characteristic detection target color exists, the posture of the captured image matches the posture of the reference image based on the centroid position, the reference centroid position, the number of pixels, and the reference pixel number. Since it can be converted and the quality of the detection object can be quickly determined, as in the first embodiment, even if the posture of the detection object changes, a large number of images can be stored as in the past, There is no need to search for feature points, and a quick detection process is possible.

  In the above-described embodiment, the detection target color is one color or two colors, but may be three or more colors.

1 is a block diagram illustrating a color image processing apparatus according to a first embodiment of the present invention. It is a time chart figure which shows operation | movement of the color image processing apparatus of FIG. It is a figure which shows the coordinate transformation operation | movement of the color image processing apparatus of FIG.

Explanation of symbols

1: Color image processing device 2: Image sensor 3: Color extraction means 4: Captured image storage means (captured image memory)
5: Detection target color selection means 6: Reference image registration means 7: Center of gravity position calculation means 8: Captured image calculation means 9: Captured image conversion means 12: Determination means 13: Brightness correction / color ratio calculation means 14: Display unit M : Detection target

Claims (5)

An image sensor that captures a color image of the detection object;
Color extraction means for reading out the captured image data of the detection target transferred from the image sensor and extracting a color for each pixel;
Captured image storage means for storing a captured image of a detection object from which a color is extracted for each pixel;
Detection target color selection means for selecting a detection target color having two or more characteristics in the captured image;
Centroid position calculating means for calculating a centroid position for each detection target color based on a captured image of the detection target in which a color is extracted for each pixel during transfer of the captured image data from the image sensor;
A captured image calculation means for calculating the relative position, relative angle, and relative size of the captured image with respect to the reference image based on the centroid position of the captured image for each color to be detected and the reference centroid position of the reference image registered in advance;
Based on the calculated relative position, relative angle, and relative size, captured image conversion means for converting the orientation of the captured image to match the orientation of the reference image;
A color image processing apparatus comprising: a determination unit that compares a reference image and the converted captured image, calculates a degree of coincidence thereof, and determines whether the detection target is good or bad.   2. The center-of-gravity position calculating means according to claim 1, wherein the center-of-gravity position calculating means determines a detection target color for each pixel of the captured image, detects positions of a plurality of points of the same color, and calculates a center-of-gravity position from an average value of the coordinates. A color image processing apparatus. The reference image calculation means according to claim 1 or 2,
The relative position of the captured image with respect to the reference image is calculated from the intermediate position between the gravity center positions of the selected detection target colors of the captured image and the intermediate position between the reference gravity center positions of the selected detection target colors of the reference image,
Calculate the relative angle of the captured image with respect to the reference image from the arc tangent of each gravity center position of the selected detection target color of the captured image and the arc tangent of each reference gravity center position of the selected detection target color of the reference image,
The relative size of the captured image with respect to the reference image is calculated from the distance between the centroid positions of the selected detection target color of the captured image and the distance between the reference centroid positions of the selected detection target color of the reference image. A color image processing apparatus.   3. The ratio of the three primary colors is calculated for each pixel during transfer of the captured image from the image sensor according to a change in brightness around the detection target, and the ratio of the three primary colors is the same by the calculation. A color image processing apparatus provided with brightness correction means for correcting the luminance of each pixel by performing detection processing for white and black. An image sensor that captures a color image of the detection object;
Color extraction means for reading out the captured image data of the detection target transferred from the image sensor and extracting a color for each pixel;
Captured image storage means for storing a captured image of a detection object from which a color is extracted for each pixel;
Detection target color selection means for selecting a detection target color having one color characteristic in the captured image;
During the transfer of the captured image data from the image sensor, the center-of-gravity position calculating means for calculating the center-of-gravity position of the detection target color based on the captured image of the detection target from which the color is extracted for each pixel;
A pixel number measuring means for measuring the number of pixels of the detection target color;
Based on the gravity center position of the captured image of the detection target color and the reference gravity center position of the reference image registered in advance, the relative position of the captured image with respect to the reference image is calculated, and the number of pixels of the captured image of the detection target color is registered in advance. Captured image calculation means for calculating the relative size of the captured image with respect to the reference image based on the reference pixel number of the reference image that has been performed;
Based on the calculated relative position and relative size, captured image conversion means for converting the orientation of the captured image to match the orientation of the reference image;
A color image processing apparatus comprising: a determination unit that compares a reference image and the converted captured image, calculates a degree of coincidence thereof, and determines whether the detection target is good or bad.

Images