JPH04505817A - Method and apparatus for processing image signals - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] Technical field: method and device for processing image signals The present invention particularly, but exclusively, provides a method for processing signals representing images of objects. shadows that have complex shapes and make it difficult to process the visual image signal. Relating to a method and apparatus for processing visual image signals of three-dimensional objects such as objects The present invention is directed to the guidance of a robot 7) device used for micro-propagation of plant materials, for example. has particular but not exclusive application in the processing of images of plant matter for Ru.

<Background technology> Growth from tissue culture of plants, i.e. micropropagation, probably improves desirable properties such as disease resistance. capable of producing in a short time a large number of genetically equivalent plants possessing the quality It's technology. Dissecting and transplanting such plants is labor intensive and repetitive. speed, sterility and labor that can be achieved through the use of robots. Cost gains make automation attractive to rapidly expanding micropropagation industries. It will give you perspective. However, automation is difficult and We need a method of robot guidance that can handle natural variability. Visual processing is agricultural Similar problems in other areas of industry, such as tomato classification, fruit harvesting, and plant identification This is a method of sensory control that has been widely used in the past. However, these techniques are It is applied to micropropagation due to its quality and characteristics such as plant nodes that need to be located. It is difficult to do so.

The problem to be solved in the context of micropropagation is that the plant material to be analyzed is three-dimensional. This means that the entire plant casts a shadow on the surface of the background.

You can then analyze the image and identify the dark areas of the plant and the dark shadowed background surface. areas, as well as the brightest parts of the plant and the illuminated areas of the background surface. It becomes difficult to distinguish between regions.

According to the invention, a signal representing an image of an object formed by electromagnetic radiation is processed. The method of The step of generating an image signal representing the image, and analyzing the color of the pixels of the image signal. providing coordinates for a pixel representing the color of the pixel, and the coordinates of the pixel are The location in the color space that represents the color of the background through a range of different values of background illumination above. Pixels representing the background are classified according to whether they are inside or outside a certain range. A method is provided that includes classifying a pixel as a pixel representing a pixel or an object.

When the term ``color'' is used in connection with an image signal, what is meant by it? If the electromagnetic radiation that defines this color is within the visual range or outside the visual range, e.g. Electromagnetic radiation represented by an image signal, even in the line or ultraviolet range is the frequency component of

In a preferred application, the method uses pixels related to the object and derived from said classified pixels. and generating an output signal including the extracted position signal. another preferred The step is to create an image of the object containing information derived from the classified pixels. and generating an output signal representative of the .

In a preferred form, the method analyzes the color of pixels to create an image in a three-dimensional color space. The pixel's color coordinates approximately encompass the area representing the background color. Pixels according to whether they are inside or outside a given volume in the above three-dimensional color space This includes classifying the pixel as a pixel representing a background or a pixel representing an object.

substantially surrounding the background color area of the predetermined volume and facing the origin of the color space; Preferably, it is a conically shaped volume.

According to the favorable property, the method uses images that lie closer to the origin than a given threshold. element, whether or not such a pixel is within the above-mentioned predetermined volume in the color space. It includes a stage of classification. This is essentially a cone of the above in color space The volume of the cone is truncated close to the origin and extends beyond the truncated base of the cone. This means that any pixel that is present will be identified as an object.

When a specific interval in an object is identified as an object by making this assignment Even in this case, the dark shadows are very close to the edge of the material of the object, so such areas are Become very close to the body itself.

In one practical configuration, the substantially conically shaped volume represents a background color. can be defined by a cone pointing from a region in color space to the origin. . The color of the background in normal illumination is represented by a sphere in a three-dimensional color space. If the cone taken from this sphere pointing towards the origin of the three-dimensional color space is It has been found to contain color coordinates corresponding to the shadow areas of the scene surface. visual image In the analysis of the image, any pixel whose color lies within this cone is compared to the background. If assigned, the compound that would otherwise have been confused with the substance of the object itself is It is possible to pick out even more distinct intervals or spaces in a cluttered object.

In a particular use of the method of the invention, the result of the pixel classification is an output representing the contour of the object. It can be used to generate force signals. In another use, the classification results are Deriving spatial coordinates of localization in objects with certain specific features, e.g. plant nodes It can be used for.

In one preferred practical example of this method, the classification step comprises (i) one pixel , α is the average vector in the background color color space and the average vector in the color space of this pixel. Calculate the value of cosα given by the following formula at the angle with the vector Steps to calculate: rr+gg+rb sentry (r”+g”+b”)”(r”+g”+b’)””cos et(this Here, r, g, and 5 are the red, green, and blue coordinates of the average vector of the background color, and r, g and b are the color coordinates of the pixel vector) (ii) For one pixel, cos The above conical shape taken from the area where α represents the background color and φ points towards the origin. Determine whether it is larger than a certain cosφ, which is a predetermined constant representing the half-angle of the volume. and (iii) this pixel if the inequality cogα>cosφ holds. Stage of classification as background may include.

In such a configuration, the method detects values below a predetermined threshold (threshold) Classifying pixels that develop color coordinates as objects, and in the above-mentioned bar graph may include using a defined step only for pixels above a predetermined intensity threshold. Ru.

In this method, the parameters of the background region in the color space are It may include a learning phase computed from the resulting pixel samples. For practical applications and this is accomplished once for the entire series of images with the same background. All you have to do is To perform this learning step, the method uses a background color image generates a constituent image signal representing the background color and analyzes a sample of background pixels to determine the background color. from these sample color coordinates to the origin in the three-dimensional color space. By joining a volume that approximately surrounds the color coordinates of the background pixel sample in by calculating the average vector angle and return half-angle of the cone formed by , the above predetermined region representing the background, e.g. a substantially cone-shaped region in color space. It may include a preliminary step of calculating the parameters of the area.

The features of the invention that have been described above with respect to the method of the invention are such that the device according to the invention It is equally applicable to

In particular, according to the invention, a signal representing an image of an object formed by electromagnetic radiation In an apparatus for processing three Means for generating an image signal representing an image of a dimensional object, and the signal and for analyzing the color of a pixel and giving the coordinates for the pixel representing the color of this pixel. The coordinates of this pixel represent the background color through a range of different values of background illumination. Depending on whether the pixel is inside or outside a given region in color space, it can be An apparatus is provided that includes signal processing means for classifying pixels as representing objects.

<Brief explanation of the drawing> , an embodiment of the invention will now be described by way of example with reference to the accompanying drawings, in which: In these drawings, FIG. 1 shows a method for generating a visual image signal of a plant and using this signal. 1 is a diagram of an apparatus implementing the present invention for loponotically engineered micropropagation of substances; FIG. the law of nature, FIG. 2 shows a routine for processing visual image signals in one embodiment of the invention. is a flowchart of FIG. 3 is a black and white image of lag microproliferation on a blue background surface; Figure 4 is a diagram of a three-dimensional color space, and the points shown in Figure 4 are relative to the background. Figures 4a, 4b, and Figure 4a is a three-dimensional diagram showing groups of pixels representing the plants and background shown in Figure 3. It shows a dot diagram that is a side view and a top view of the color space, respectively. Figures 5a, 5b and 5a are for images of a uniformly illuminated blue background surface. shows a dot diagram that is a double-sided view and a plan view, respectively, of a three-dimensional color space showing the color values to be and Figures 6a, 6b and 60 show the same image under slanted illumination to produce shadows. 5a to 50 for the blue background surface of The figure is a diagram of a three-dimensional color space showing a cone of color values used according to the invention, FIG. 8 shows the 1 co where α for one pixel is the angle between the pixel vector and the average background vector It is a frequency distribution diagram of sα.

<Explanation based on examples> Embodiments of the present invention include automatic or semi-automatic operation in micro-propagating plants mt@fertilization, e.g. For example, it is mentioned in connection with cutting nodes from plants. In Figure 1, the conveyor bell The transporter 11 carries the plant under the solid state color video camera 12 and then The carrier passes just below the Cartesian robot system shown generally at 13. The motor 14 is controlled by motors 15 and 16 which are controlled by a control device 17. It can be moved to any selected position on the bare belt 11. conveyor belt The speed of A 11 is such that the position of the plant observed by camera 12 is below carriage 14 by a control device 18 so as to be correlated with the subsequent position of the plant.

The carriage 14 carries an actuating device, e.g. a plant, which operates under the control device 20. A cutter 19 for cutting is carried. Control devices 17, 18 and 20 are all It is controlled by a main microcomputer 21.

Video camera 12 sends a first color visual image signal to signal processing means 22 embodying the invention. a signal, whose output may represent, for example, a binary visual image of the outline of a plant. 2 image signals are generated. The second image signal is supplied to the monitor 23 and It is also supplied to the computer 21. Images according to the present invention (described hereafter) Plain classification provides additional information to existing color information. Therefore, the improved 2nd place power is Gray level or color information could also be used, although this is a possibility and should not be ignored. You don't have to.

The operation of this system is semi-automatic or fully automatic. When semi-automatic, the operator Observe the image on the monitor 23 and operate the keyboard of the microcomputer 21. By doing this, for example, you can cut a required node of a plant. The localization cut by the canter 19 is selected on the monitor 23 in order to obtain the image. Full self In the dynamic system, the microcomputer 21 further processes the second image signal; The knot is automatically positioned and the movement of the cutter 19 is instructed. In some configurations, the signal The processing means 22 and the microcomputer 21 carry out all necessary functions. may be provided by one signal processing computer. In other configurations, signal processing Further processing is carried out in the means 22, and the signal processing means 22 is input to the combiator 21. Only the position information for the desired localization in the image is sent.

FIG. 2 shows a flowchart for setting the main stages of image signal processing in the signal processing means 22. – Shows the chart in the form of a line diagram. This chart provides a general overview of the operation of this example. The method will be described in detail below after it has been described.

solid-state color video camera The video signal from video caa+era) is decoded by a PAL decoder. Split into red, green and blue components. These red, green and blue components are the original black and white components. It is digitized together with the video image, and a frame grabber card (Frame G Rabber card).

Each complete image has 256x256 pixels and 255 grays for each pixel. Each pixel in a zero-color image consists of four component images with possible levels The color and brightness of is determined by its gray level values in the red, green and blue component images. For example, a red pixel in a color image has a higher value in the red component than in green or blue. have

In the example that follows, the image used is against a blue surface background. This is a set of microphytes of the genus Rhododendron, and this background is mainly red and green. It has been found to give the best contrast to objects. Figure 3 is a typical black Shows a white image. A known method for classifying such images is gray levels. Recitalding, in which pixels have their gray level below the threshold. It is called ``dark'' or ``light'' depending on whether it is above or below. This technology The gray level value of the intended category in the page, e.g. the overlap between plants and background. It does not function well when there is a problem. For example, in Figure 3, the inner diameter of the plant stem is The shadows are brighter than the background, but some of the shadows in the background are darker, like the plants. , so the result of gray level thresholding is that many pixels are misclassified. It will end up being put away. These pixels are correctly separated as described according to the present invention. Similarity requires more information, such as color.

The color image produced by a video camera is completed by its red, green and blue components. Because it is fully represented, each pixel is a vector or point (R, C, B) in color space. It can be expressed as Figure 4 shows the Cartesian coordinates representing red, green and blue. si-an coordinates) tertiary defined by R, G, B It is a diagram of the original color space. A completely black pixel has zero values in all three components. Therefore, the brightest white pixel is seen at the point (0,0,0) in the color space. (255, 255, 255). Gray with equal value in all colors Color pixels are seen along the line R-G-B.

The points shown in Figure 4 are for plants against a background of substantially uniform color (e.g. It shows the color image of plants (shown in black and white). As shown in Figure 4. The diagram takes each pixel of a color image of a plant against a background of substantially uniform color, and Determine the color coordinates of a pixel, then print a dot representing this pixel at the appropriate coordinates in the color space. This is achieved by making lots. The background and plant colors are shown in line diagrams. Two groups of dots are shown in χ and Y. However, the group of these points is purely diagrammatic and is shown in the following Figures 4a, 4b and 4a. The location does not correspond exactly to the plant diagram in the figure.

A color image equivalent to FIG. 3 can be decomposed into its red, green and blue components.

The distribution of pixels in the color space is shown in the dot diagrams shown in Figures 4a, 4b and 40. In these point diagrams, the color components of each pixel are are plotted against each other. These point charts show the density of pixels at a certain point. FIG. 2 is a side view and a top view of a color space cube represented by FIG.

In the dot diagrams in Figures 4a, 4b and 4a, two groups are visible, one One corresponds to the plants and the other corresponds to the background. Pixels from a bright background are dominantly different plants because they have greater values in all three colors. They form a group far from the origin. In Figure 4b, which is a red-blue dot diagram, the background group is Because of its blue color, it is above the line R-B. The same tendency toward the front axis is the blue-green point in Figure 4C. As can be seen in the diagram, the red-green dot diagram in Figure 4a is the red and green dot diagram for the plants in Figure 3. There is almost no difference between the red and green values in any group. Not yet.

Figures 5a, 5b and 5c show the color space distribution of a uniformly illuminated pure blue background. Figures 6a, 6b, and 60 are point diagrams shown at an angle to the image. shows a dot diagram of the same background surface after a shadow has been cast to produce the same illumination. Ru. The distributions in Figures 5a to 50 show that all pixel vectors are in the color space. A small oval indicating that similar red, green and blue levels exist near a particular point. It is a thing. When a shadow is cast on the image, as shown in Figures 6a-60, , this ellipse is created if the color vectors change their coefficients but not their directions extends the tail of the cone towards the origin as expected in It is said that the degree or intensity is a relationship between its coefficients and its "color" is a function of its direction. consistent with the interpretation. The conic distributions of point diagrams 6a, b, and C are three-dimensional diagrams of the color space. The shape is shown diagrammatically in FIG. This cone is designated Z and is of course shown in Figure 6a. The shape has been simplified from the actual experimental results shown in FIGS.

As seen in Figures 5a-50 and 6a-60, the same color However, due to shadows, pixels of different intensities form a cone-shaped group in color space. In the proposed algorithm of this embodiment of the invention, pixels have their color depending on whether the vector lies inside or outside the cone defined by the sample background group. are classified as background or object.

The initial stage of this algorithm is the learning process, in which the cone The axial vector and half-angle of are derived from a typical sample of the background, in this case There is a wide slip around the edge of the image. Depending on whether the sample contains shadows or not First, the mean background vector must lie on the group axis, so the axial vector Tor can be set to the average of the samples. Sample covariance matrix Although it is possible in principle to derive an appropriate half-angle from is known to not change much, and is simply set as a constant before processing. was selected.

By taking the scalar product, the average background vector (lower, g, b) and the arbitrary vector ( The angle α between r, g, b) is rr+gg10bb= (1+120 koku”)”(r”10g”+b”)”cos a--- (1) and has an axial vector (r, g, b) and a half-angle φ The vector (r, g, b) inside the cone is. . sa＞. . 5φ・・・−・・ - It turns out that (2) is true.

The main part of this algorithm is to calculate cos α for each pixel from equation (1) and This pixel is compared with cosφ in inequality (2), and inequality (2) holds this pixel. The amount of processing per 0 pixel that consists of allocating to the background is 7. The terms g and ■ are all established as constants during the learning stage, and the square root operation looks This is achieved through look-up tables, cosφ Since is a pre-processing constant, it is not that large.

Figure 8 shows all the colors in the color image corresponding to the plant image seen in Figure 3. A frequency distribution diagram of cos α calculated from equation (1) for a pixel is shown. this can be used to determine the actual value of the half-angle φ. To cos α-1 or α-01 The peaks in 0 are off scale and these two in 0 color space are removed. The different directions of the groups are cos α − 0,980 or the difference between them at α 11°. It will be remembered how he was chosen by the valley of his will. This value of α is the background cone Since it emphasizes the edge of , it is the best choice for the angle φ in equation (2). .

In figure 3a! ! Approximately 5 images of 111 m are processed in the same way, and this value is α- It was found that it does not change by more than 2 degrees from 10@. This value is displayed on a blue background. It was expected that the results would be the same for images other than images of microphytes of the genus Rhododendron. However, for such images, the change between them is small, and φ−10° Fixed as a pre-processing constant (pre-processiB constant) This simplifies this calculation.

The selection of this half-width determines the degree of severity of the classification, that is, what kind of errors are present (back It affects whether a scene is classified as an object or vice versa. In theory, this angle The degree of separation in the color space of samples of background pixels, and possibly samples of object pixels. You can choose from specific learning cloths. A very simple strategy is to The first step is to select the smallest angle that allows all pixels to lie within the cone.

One practical method is to experiment with different angles and evaluate this division by eye. It is to be. Another practical method is to create a frequency distribution map (as shown in Figure 8). The drop at about 0.98 marks the perceptible bifurcation between the two populations of object and background. It is assumed that it represents In certain cases, 1 for this half-angle It has been found that it is possible to choose one thousand values; this is probably because This is not always the case. Generally, this half-width is preset (perhaps actually (based on experience) or actively as part of a learning procedure.

Here, regarding the operation of the steps designated as A to A in the flowchart of Figure 2, I will explain it anyway. This flowchart is executed by the processing means 22 in FIG. It shows the main processing steps performed. The first step executed in step A of FIG. The step is to derive a set of component signals from the input signal from the camera 12. Ru. This processing means grasps the input signal in the form of three component color images and writes them down. It is configured to be stored in memory. The next step in B is to provide processing means 22 with resets the row and column counters (corresponds to the row and column of pixels in the input video signal) ) and make these counters specify the top left-hand pixel of each component image. It is to do so.

The center of the flowchart of FIG. 2 in stages F@C to G consists of these rows and columns. At the 0 stage C, which represents the processing of the pixel specified by the counter, each color At step 0, the gray level of the pixel in the component is fetched from memory, These values are defined by α being the vector from the origin to the pixel's position in color space and from the origin Let cos α be the angle between the vector to the position of the average background color in color space. used for calculations. The value of cosα is determined according to the formula set above. This pixel that was tested in step F is determined by the result of the test in step F. It is emphasized as a background or as an object in stage G.

The set of steps H causes the algorithm to progress through this image. The column counter is incremented in the 0 stage H, processing each pixel in turn; The contents of this page are in tier 1! The current row of 0 pixels tested in lI is not yet completed. If not, flow returns to stage C to process the next pixel. This line is completed at stage r. If completed, the flow is sent to stage J where the row counter is incremented and the column counter is will be reset. When the row counter reaches 256, it means that all pixels have been processed. It becomes. This means that the 0 row counter tested in phase K has reached 256. If not, the logic flow returns to step °C. Three color component images with all pixels identified When processed in a stage, the logic flow is sent to a stage, and a stage consists of three components. Terminate the algorithm for a particular set of color images.

The results obtained by the cone algorithm misclassify bright stems as background. It avoids high-temperature illumination and produces good results even under weak illumination, which is unavoidable in practical applications. It's working. Most of the shadows are classified correctly, but the dentalization effect near the origin of the color space Errors can occur in the darkest areas of the picture due to color. 7 gray levels from the origin In the value radius, one gray level at the localization of one pixel in color space. An error in the angle can result in an 8° difference in the angle α in equation (1). In Figure 8 Since the angle between the object and the background group is only about 11, this error causes the pixel to It is sufficient to misclassify from one group to another. This problem is α-g-b-0 does not allow a unique definition of cos α when For example, the normalization of 0 caused by the specificity in equation (1) that makes it Other non-linear definitions of color, such as color, saturation and hue, are characterized by the same specificity.

Therefore, this algorithm requires that the cone is indicated diagrammatically by the dashed line in Figure 7. A modification can be made that is cut at its tip so that it looks like this. Then, the expression ( 1) is used only above a set intensity threshold, e.g. 30 Any pixel below this is classified as a plant. This method results in better results. This improves the results obtained with a full cone.

The cone shown in Figure 7 and defined by the above algorithm is that The imaginary color "cube" extends outward from the origin until it reaches the limit of J. It should be noted that it is defined as extending. Therefore, this cone is a circle Samples of background pixels that do not end in a plate or sphere form a particular shape in color space do.

This group of pixels is near (hereinafter referred to as However, this cone does not extend beyond this sphere. Sometimes this sphere is this group. It is also desirable to slightly reduce the In such cases, certain background pixels may be misclassified. , but more object pixels can be correctly classified.

CO5α.

FIG, 8° Submission of translation of written amendment (Article 184-8 of the Patent Act) November 26, 1991 area

Claims (12)

[Claims] 1. A method for processing signals representing images of objects formed by electromagnetic radiation. There, An image that represents an image of a three-dimensional object placed against a background of substantially uniform color. generating a digital signal; Coordinates for the pixel representing the color of the above pixel by analyzing the color of the pixel of the above image signal and pixel, the coordinates of the pixel are the above over a range of different values of the background illumination, depending on whether it is inside or outside a given area in the color space representing the background color. A method characterized in that the method includes the step of classifying the object as a background or an object. 2. an output containing positional information related to the object and derived from the classified pixels; A method according to claim 1, characterized in that it includes the step of generating a force image signal. 3. represents an image of the object containing information derived from the classified pixels; A method according to claim 1, characterized in that it includes the step of generating an output signal. 4. analyzing the color of a pixel to provide coordinates for the pixel in a three-dimensional color space; as well as the pixel, the color coordinates of the pixel substantially surrounding the area representing the color of the background; Background or objects according to whether they are inside or outside a given volume in a three-dimensional color space to be classified as representing A method according to claim 1, characterized in that it comprises: 5. The predetermined volume is substantially a circle that substantially surrounds the background color area and faces toward the origin. 5. A method according to claim 4, characterized in that the volume is cone-shaped. 6. Regardless of whether such a pixel exists within the above predetermined volume in the color space, The method includes the step of classifying pixels that are closer to a point than a predetermined threshold as objects. 5. A method according to claim 4, characterized in that: 7. An output signal representing the outline of the object using the results of the classification of the pixels A method according to claim 1, characterized in that it comprises generating. 8. Using the results of the above pixel classification, we can determine the localization of an object with certain specified features. 2. A method according to claim 1, characterized in that it comprises deriving intermediate coordinates. 9. The above classification stage is (i) For one surface element, α is the average vector in the three-dimensional color space of the above background color. The angle between Tor and the vector in the color space of the above pixel is the following formula, ▲ Formula, There are chemical formulas, tables, etc.▼ (Here, r, g, and b are the red, green, and blue coordinates of the average value vector of the background color above. where r, g and b are the color coordinates of the above pixel color vector) (ii) for one pixel, cos α is The above conical shape where φ is taken from the area representing the above background color and facing the origin A stage for determining whether or not the value is larger than a predetermined constant cosψ representing a half-angle of a volume. and (iii) the above inequality. cosα＞cosψ The claim further comprises the step of classifying the pixel as background if the pixel is retained. Method according to Section 1. 10. A stage for classifying pixels having color coordinates with values below a predetermined intensity threshold as objects. and the steps defined in claim 10 above the predetermined intensity threshold. 10. A method according to claim 9, characterized in that it comprises the step of using only elements. 11. generating a calibration image signal representing a color image of said background; analyzing a sample of pixels of the scene to provide the color coordinates of the background; the sample color; From the coordinates, the above average vector angle, as well as the origin in the three-dimensional color space to the color space. By joining the volume that approximately surrounds the color coordinates of the sample of the background pixel above, The above-mentioned predetermined area representing the background by calculating the approximate half-angle of the cone formed by 2. A method according to claim 1, characterized in that it comprises a preliminary step of calculating the parameters of . 12. for processing signals representing images of objects formed by electromagnetic radiation. The device provides an image of a three-dimensional object placed against a background of substantially uniform color. means for generating an image signal representative of a pixel of the image signal; and to analyze the color and give the coordinates for the pixel that represents the color of this pixel. The raw coordinates are in a color space that represents the background color through a range of different values of background illumination. Pixels represent background or objects depending on whether they are inside or outside a predetermined area in the An apparatus comprising signal processing means for classifying a pixel as a white pixel.