JPH08272971A - Object recognizing method - Google Patents

Abstract

PURPOSE: To enable an accurate recognition of a linear body which is hardly affected by noise by inputting image data of respective divided slit areas to a neural network and outputting the position of the linear body from the neural network. CONSTITUTION: Input image data stored in a memory 12 for image storage are read in a computer 14, slit by slit. The slits are strip-like areas having the lengths direction almost at a right angle to the direction wherein the linear body 16 included in the input image data extends. The input image data are divided into the slits, and images of the respective slits are supplied to the neural network. The slits are not pixels of 'one line' having one-pixel width, but the striped areas having constant width larger than noise. Consequently, the neural network can easily discriminate between the noise and linear body 16 to be recognized. Consequently, the coordinates of the linear body can be outputted while the influence of the noise is held small.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognition method. In particular, the present invention relates to a recognition method capable of detecting states such as the length of a linear object and the angle at which it is placed.

[0002]

2. Description of the Related Art Automatic mounting inspection of wire bonding,
Automatic wiring of wire harnesses has been a difficult field to realize. As one method for realizing these things, it is considered that the shape of the wire is recognized by image processing and the wire is operated based on this. It is considered that more reliable inspection accuracy can be obtained by recognizing the shape of each wire one by one.

As a conventional technique used for recognizing such a distorted shape of a line object (string, wire harness, etc.), a thinning method which is generally used as a character recognition method can be considered.

For example, Japanese Patent Laid-Open No. 5-248839 discloses a method for recognizing a linear tissue. The method described here is basically a method of extracting a feature amount of a linear tissue by using thinning, which is one of the classical image processing.

When wire objects such as wire harnesses, ropes, rubber tubes, and paper strings are automatically assembled or packed, it is necessary to hook the target object with a hook or grab it with a robot hand. . The essential information for doing such a job is "where to grab". When a line object having low rigidity is placed on the table and left without being applied with sufficient tension, the line object is distorted due to residual stress. Since the shape of such distortion is generally unspecified, it cannot be predicted. That is, the shape of the distortion generated after the line object is placed on the table needs to be recognized tactilely or visually.

In the case of visual recognition, it is necessary that the line object to be recognized has a uniform thickness, that there is no pattern on the surface, and that the light rays hit the surface uniformly. Further, the shape can be recognized only when the foreign matter is never shown in the image. Only when such a condition is satisfied, it is possible to obtain a single line representing a desired line object by subjecting the entire image showing the line object to thinning processing.

However, in reality, it hardly occurs in the ideal case described above. That is, a foreign object is often shown together in an image, or an object to be recognized has a pattern. When the thinning process is performed on such an image, a plurality of lines are generated even if the target line object is only one. Therefore,
In reality, it is almost impossible to recognize a line object by a simple thinning process. Conventionally, it has been recognized that such inconveniences exist, and studies have been made to overcome them, but effective recognition algorithms have not yet been developed.

[0008]

The method of thinning processing has a drawback that even a very small noise appears as a line. Therefore, there is a problem that it is necessary to determine which of the plurality of lines belongs to the target object.

The present invention has been made in view of the above problems, and an object thereof is to provide a method of recognizing a target object capable of accurately recognizing a line object.

[0010]

In order to solve the above problems, a first aspect of the present invention provides a plurality of image data including a line object whose longitudinal direction is substantially perpendicular to the direction in which the line object extends. A dividing step of dividing into slit areas and image data belonging to a plurality of slit areas obtained by the dividing step are sequentially input to a neural network, and the position of the line object in the longitudinal direction of the slit area is sequentially input to the neural network. And a coordinate value output step of outputting the coordinate value, and recognizing the line object by outputting the coordinate value.

In order to solve the above-mentioned problems, a second aspect of the present invention is the object recognition method of the first aspect of the present invention, wherein each of the plurality of slit areas in the dividing step overlaps with an adjacent slit area. The object recognition method is characterized by the following.

In order to solve the above problems, a third aspect of the present invention is the object recognition method according to the first or second aspect of the present invention,
The neural network includes a noise mixing step of mixing noise into image data including a line object serving as teacher data, and the image data mixed with the noise has a longitudinal direction substantially perpendicular to a direction in which the line object extends. An object recognition method comprising: a teacher data dividing step of dividing into a plurality of slit areas; and a neural network learning step of learning using a plurality of divided slit areas as teacher data.

[0013]

The neural network according to the first aspect of the present invention inputs the image data contained in the strip area and outputs the position of the line object in the image data. Therefore, it is possible to output the coordinates of the line object while keeping the influence of noise small.

In the second aspect of the present invention, a plurality of strip regions overlaps adjacent strip regions. Therefore, the neural network can output more accurate coordinates.

The noise mixing step in the third aspect of the present invention is
Noise is mixed into the image data including the recognition target to be the teacher data, and the teacher data dividing step divides the image data into a plurality of strip areas. Then, the neural network performs learning by using this strip area as teacher data.

The teaching data is, of course, data including a curve representing a line object to be recognized,
In the present invention, noise is mixed in the image data including this line object. A neural network resistant to noise can be obtained by learning the neural network by using the data mixed with such noise as the teacher data.

In the first and second aspects of the present invention, since image data divided into strip areas is applied to the neural network, the third aspect of the present invention is also applicable.
Image data created by mixing noise is divided into strip areas and then applied to the neural network.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

As described above, the inventor of the present application considers that conventional thinning, which is a classical image processing, merely complicates the problem and makes it difficult to recognize line objects. , We propose a device to obtain the shape of a line object without going through the thinning procedure.

A line object recognition apparatus according to a preferred embodiment of the present invention, as shown in FIG.
And an image storage memory 12 for storing an input image output from the image input device 10. The image data obtained by the image input device 10 is finally supplied to the computer 14, where the coordinates of the target line object 16 are calculated. In the computer 14, a neural network is configured by software, for example, and outputs the coordinate value of the line object 16 with respect to the image data stored in the image storage memory 12. In this embodiment, the neural network is composed of software in the computer 14, but it is also possible to use dedicated hardware.

The image input device 10 is for obtaining image data of the line object 16 which is a recognition target placed on the test table 18, and its detailed configuration is shown in FIG. As shown in FIG. 2, the image input device 10
Is composed of a camera 10a and an auxiliary mechanism 10b for moving the position of the camera 10a. Although not shown in FIG. 2, the image data output by the camera 10a is supplied to the image storage memory 12 via a predetermined interface.

First, the target line object 16 is photographed by the camera 10a. At this time, if the entire length of the line object 16 is too long to be photographed within the range of the resolution of the camera, the camera may have a structure of rotating in the longitudinal direction of the line object or a structure of performing linear motion. In addition, when the target line object 16 is greatly distorted, it is preferable to add a structure in which the camera makes similar movements to the left and right.

As shown in FIG. 2, the auxiliary mechanism 10b for moving the position of the camera 10a is composed of a vertical camera movement 10b (i) and a horizontal camera movement 10b (ii). In general, only the lateral camera movement 10b (ii) moves. In the following description, it is assumed that the vertical camera movement 10b (i) does not move, and the description will proceed. The vertical camera movement 10b (i) serves to bring the line object back into the visual field if it goes out of the visual field. That is, in the following description, the line object 16
Assumes that it does not protrude from the field of view.

Although the conventional CCD camera or the like can be applied to the camera 10a, any conventional camera can be used as long as it can convert visual information into an electric signal.

The image data taken by the camera 10a in this manner is stored in the image storage memory 12. The image storage memory 12 temporarily stores the image data sent from the camera 10a. For example, various storage devices such as a magnetic disk and an IC card can be applied. Further, the image storage memory 12 is a computer 14
It is also preferable to provide it inside. In the following description, the image data stored in the image storage memory 12 will be referred to as input image data.

FIG. 3 shows a principle diagram showing the principle of the recognition method of the line object 16 which is characteristic in the present embodiment. FIG.
As shown in, the input image data is read by the calculator 14 in units of slits. This slit is a strip-shaped region whose longitudinal direction is substantially perpendicular to the direction in which the line object included in the input image data extends. The input image data is divided into a plurality of slits, and the image of each slit is supplied to the neural network.
A characteristic of this embodiment is that the image data is supplied to the neural network for each slit. It should be noted here that the length of the short side of this slit is set larger than the noise. That is, the slit is not a "one row" of pixels each having a pixel width, but a strip-shaped region having a constant width larger than noise. Therefore, it is expected that the neural network can easily distinguish the noise from the line object to be recognized. That is, the line object 16 always crosses the slit region, but since the noise is smaller than the width of the slit, the noise and the line object 16 can be clearly distinguished.

As shown in FIG. 3, the long side of this slit is equal to the length of one side of the input image. Further, in this embodiment, the input image data is binarized by a predetermined threshold value and used as a binarized image. This is to reduce the information amount of the image data and simplify the configuration of the neural network, but it is of course possible to directly handle the image data having gradation. However, in this case, the scale of the neural network becomes extremely large, so that it is considered almost impossible to realize.

As shown in FIG. 3, the output of this neural network outputs the coordinate value of the line object to be recognized. This coordinate value represents the position of the slit in the long side direction. In the present embodiment, the long side direction of the slit is the Y axis and the short side direction is the X axis. Then, by combining the average value of the X coordinates of the pixels included in the slit image and the coordinate value Y (t) obtained when this slit image is supplied to the neural network, the coordinate value of the line object to be recognized is combined. Is obtained. Coordinate values are obtained for each of the plurality of slits, and finally, a plurality of coordinates of possible positions of the line object are obtained as the recognition result of the line object.

If the position of the slit slides, the output of the neural network also changes. By observing this change, the state of the line object reflected in the input image can be known. That is, by performing time-series signal conversion of the slice pattern of the horizontally long image and processing the converted signal, the shape of the target line object can be obtained as a function without going through the thinning process.

Although the input image data is divided into a plurality of slits (regions) in the present embodiment, each slit is slightly overlapped with the adjacent slits. For example, the size of this slit can be 256 pixels on the long side and 64 pixels on the short side. In this case, it is preferable to overlap, for example, about 8 pixels of the 64 pixels on the short side with the adjacent slits. By overlapping the pixels partially in this way, it is possible to recognize the line object with high accuracy.

The feature of this embodiment is that the input image data is divided into slits having a width wider than noise, and the pixel data belonging to each slit after division is supplied to the neural network. With this configuration, the line object 16 can be recognized without being affected by noise. Furthermore, in the present embodiment, each slit is not completely and independently selected, but if the portions that are overlapped little by little are provided between the adjacent slits, the change between the slits can be made smooth. As a result, it is possible to improve the accuracy of the output result of the neural network.

Since the output value of the neural network is a sigmoid function, it is generally a number between 0 and 1, but by multiplying this by the length of the Y axis of the input image data, The value of the coordinate is obtained.

Here, the value of the sigmoid function does not need to use the entire range (0 to 1). For example, 0.2-0.
When using the range of 8, (coordinate value) = (((neural network output value) -0.
2)) / 0.6) x Y-axis length gives specific coordinate values. Specifically, 0.2-
It is preferred to use the range 0.8.

As described above, the present embodiment is characterized in that the input image data is divided into slit image data and the coordinate values are calculated by the neural network. In this embodiment, D. E. FIG. Rumelhar
t, J .; L. McClellend, and PDP
Research Group: "ParallelD"
distributed Processing ”, MI
It employs a neural network of the type described in T Press (1986). A block diagram of the neural network adopted here is shown in FIG.

The neural network is composed of a plurality of units (neurons). All units have the following functions. First, all units have bidirectional inputs and outputs. That is, it has two types of couplings: (1) a coupling for receiving an input from the outside or an input from a previous unit, and (2) a coupling for an external or output of the next unit. All of these joins have a join weight of any value. The received signal is multiplied by the weight of the signal transmitted by the signal, and the sum of the multiplication results is calculated. This sum is subjected to signal conversion by performing non-linear conversion (for example, conversion by a sigmoid function). The value after signal conversion becomes the output value of the unit.

The unit, as shown in FIG.
It is divided into three types: an input unit 30 that receives an external input, an output unit 40 that sends a signal to the outside, and an intermediate unit 50 that has no contact with the outside. These units are divided into a plurality of groups, each of which is one or more, and are arranged in layers.

Here, when there is one or more layers in which units for performing non-linear conversion are collected, a plurality of intermediate layers are allowed. Also, any connection other than the connection shown in FIG. 4 is allowed in the neural network. Further, the neural network can be selected either by using dedicated hardware or by using software by considering its cost and processing speed.

The line object recognition algorithm according to this embodiment is shown in the flowchart shown in FIG.

First, in step ST5-1, the photographed image is binarized. This is performed in order to emphasize the desired line object 16 and to make the gradation of the pixels uniform. As a method for setting a predetermined threshold value for binarization, any conventionally known method can be used. At this time, the threshold value may be set poorly, and foreign matter may be emphasized and remain in the image after binarization, but the neural network is given a function to ignore small noise by learning. Therefore, some foreign matter or noise may remain in the binarized image.

Hereinafter, for ease of explanation, the image after binarization is referred to as a binary image. According to the recognition principle according to the present embodiment described above, the input image may be divided by slits, that is, scanning by the slits may be performed. However, since the range of the gray level of the image is narrower for the binary image, the neural This has the advantage that the burden on the network can be reduced. Therefore, in the present embodiment, the following description will be made assuming that a binary image is scanned.

In step ST5-2, the slit position is initialized. The initial state of the slit position is
This is when the slit is at the far left (toward) of the binary image. In the following description, the portion that appears in the area inside the starting point slit of the binary image is called a slit image. The resolution of the binary image is N (horizontal) × M (vertical) pixels, the number of pixels in the slit is L × N pixels, and the step width of one scan of the slit is P pixels. At this time, when the step count of scanning by the slit is T, the horizontal position coordinate X of the upper left point of the area currently covered by the slit in the binary image is X = P × (T−1) Become. At this time, 0 ≦ X ≦ MP. The maximum value of T that satisfies these conditions is Tmax
Is defined as the end point of the scanning.

Input of the slit image to the neural network is performed in step ST5-3. The slit image is projected on the input layer of the neural network.
That is, the values of the corner pixels included in the slit image are supplied to the input layer of the neural network. At this time, the resolution of the slit image may be changed. For example, when the number of pixels of the slit is 256 × 64, it is possible to reduce the resolution to, for example, about 12 × 3 pixels by collectively taking an average value of a plurality of predetermined pixels. This is because the greater the number of units in the neural network, the slower the execution speed of software and the harder it is to implement hardware.

The calculation of the output of the neural network is
This is performed in step ST5-4. The output of the neural network represents the position coordinates y (T) of the point passing through the slit central portion when the line object 16 corresponds to the step count T and the line object 16 has the line in the screen. The judgment value is j (T). This y (T)
6 is shown in FIG. As shown in FIG. 8, the central portion of the line object 16 having a constant thickness is y.
It is taken as (T).

The lateral position coordinate is x (T) = P ×
It can be expressed as (T-1 / 2). This y (T) and j
(T) is stored in the image storage memory 12 in the order according to the step count T. For scanning a line object, it is possible to obtain an output value y (T) corresponding to the scanning position x, albeit discretely. That is, in this embodiment, the position of the minute portion of the line object 16 reflected in the image is
The detection is performed in all parts of the line object 16. Further, by obtaining j (T), the line object 16
It is possible to easily detect where both ends of the are located.

A general method for calculating the output of a neural network of the type used in this embodiment is described in D.
E. FIG. Rumelhart, J .; L. McClellen
d, and PDP Research Group:
"Parallel Distributed Pro
cessing ”, MIT Press (1986)
It is described in. Although the method described here is followed as it is in the present embodiment, when an improvement plan such as speedup of learning is made to the above document, the improvement plan is the minimum value search of the error function. If it solves the problem, it does not affect the performance of the object recognition apparatus in this embodiment. Further, even if a method (recursive neural network) of recursively inputting the output value y (T) into one of the input units is used, the same degree of accuracy can be obtained. I. Shown by Jordan (see Figure 13).

It is also necessary to carry out effective learning in order to obtain the output of the neural network with high accuracy.
The learning of the neural network used in this embodiment is also described in D. E. FIG. Rumelhart, J .; L. McCle
llend, and PDP Research Gro
up: "Parallel DistributedP
processing ”, MIT Press (198
You should learn in 6).

In order to obtain inputs and outputs used for learning, a pseudo input image and a teacher function are created by the procedure described below. The pseudo input image is scanned with a slit having the same size as when actually used, and the coordinate value at each step count is calculated from the teacher function and given.

Pseudo-input image (an example is shown in FIG. 7)
Creates a line object to be recognized artificially under the same conditions as the actual input image (line thickness, gray level, resolution), fits it on the screen, and artificially adds noise of a certain scale. It was created by

A method of creating a pseudo input image will be described below.

First, a curve that represents the shape of a line object in a pseudo manner is created. These curves are created as spline functions that do not protrude from the screen of the pseudo image. Assuming that the resolution of the pseudo input image is U × V pixels, the concrete creating means is as follows. First, the number of reference points is determined within the range of 0 to n (the value of n depends on the resolution of the input image and the degree of deformation to be learned). For example, in FIG.
Shows that eight reference points are arranged in N × M images. Then, the reference points set in FIG. 8 are interpolated by an arbitrary spline function. An example interpolated by this spline function is shown in FIG. Next, the thickness of the curve represented by this spline function is set to a predetermined width. It should be noted that this thickness may be set so as to change every time by a random number. This is shown in FIG.
Is shown in. Finally, random noise is added to the screen to create a pseudo input image. This final appearance is shown in FIG.

Creation of Teacher Function The teacher function has a value taken at each slit position by the spline function set when the pseudo input image described above is given as the teacher function. That is, the teacher function is obtained at the same time while the pseudo input image is being created.

The output of the neural network is stored in memory. This memory is step ST5- in FIG.
5 takes place. Then, the output of the neural network is stored in the output storage memory in the order of the slit positions (that is, data may be sequentially written to the next writing position). It should be noted that the format of storage is not limited.

Next, in step ST5-6, T
Is incremented. This has the effect of shifting the slit position by one step.

In step ST5-7, it is determined whether or not the slit position has reached the end point. When the end point is not reached, the above step ST5 is performed at the next slit position.
If -3 or less is performed and the end point is reached, the process of step ST5-8, that is, the signal processing of the neural network is performed.

Signal processing of the neural network is performed in step ST5-8. When the scanning of the line object is completed, that is, when the end point is reached, the stored output value of the neural network is processed.

Output y (T) of neural network
When the stored contents are plotted on the T-Y plane in order to express the above as a graph regarding the scanning position T, for example, the state of the graph is as shown in FIG. As shown in FIG. 8, these points (points representing the output y (T) of the neural network) are a set of discrete points including the peculiar error found in the output of the neural network.

In order to obtain a continuous curve from such discrete points, interpolation by a spline function is performed. Then
The shape of the line object 16 to be obtained is obtained as a smooth function (curve). Since the shape data can be obtained as a function, numerical data can be easily created and can be easily used for automatic control.

[0058]

As described above, according to the first aspect of the present invention, the image data is divided into a plurality of slit areas, the image data of each slit area is input to the neural network, and the position of the line object is determined by the neural network. Since it is output from, it is possible to obtain an object recognition method that is not easily affected by noise and that can accurately recognize a line object.

According to the second aspect of the present invention, since each of the plurality of slit areas overlaps with the adjacent slit areas, the data change between the slits becomes smoother, and as a result, the object of high accuracy can be obtained. A recognition method is obtained.

According to the third aspect of the present invention, the neural network uses the teacher data created by mixing the image data containing the line object with noise and dividing the image data containing noise into slit areas. Have been learned. Therefore, it is possible to obtain an object recognition method that enables effective learning and also enables highly accurate object recognition.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an overall configuration of an object recognition device of this embodiment.

FIG. 2 is a configuration diagram illustrating a detailed configuration of the image input device 10.

FIG. 3 is an explanatory diagram illustrating a principle of recognizing an object according to the present embodiment.

FIG. 4 is a configuration diagram illustrating a configuration of a neural network used in this embodiment.

FIG. 5 is a flowchart showing the flow of operations of the object recognition method in this embodiment.

FIG. 6 is an explanatory diagram illustrating a method of calculating a value of y (T).

FIG. 7 is an explanatory diagram showing an example of a pseudo input image used for learning of the neural network used in this embodiment.

FIG. 8 is an explanatory diagram illustrating creation of a pseudo input image.

FIG. 9 is an explanatory diagram illustrating the creation of a pseudo input image.

FIG. 10 is an explanatory diagram illustrating creation of a pseudo input image.

FIG. 11 is an explanatory diagram illustrating the creation of a pseudo input image.

FIG. 12 is an explanatory diagram showing a case where the output at each slit position of the neural network is plotted.

FIG. 13 is a configuration / schematic diagram of a recurrent neural network.

[Explanation of symbols]

10 image input device, 12 image storage memory, 14
Calculator, 16-line object, 18 test bench.

Claims (3)

[Claims] 1. A dividing step of dividing image data including a line object into a plurality of slit areas having a longitudinal direction substantially perpendicular to a direction in which the line object extends, and a plurality of slits obtained by the dividing step. Image data belonging to the area is sequentially input to the neural network, and the position of the line object in the longitudinal direction of the slit area is sequentially output to the neural network.
A coordinate value output step, and recognizing the line object by outputting the coordinate value. 2. The object recognition method according to claim 1, wherein each of the plurality of slit areas in the dividing step overlaps an adjacent slit area. 3. The object recognition method according to claim 1, wherein the neural network includes a noise mixing step of mixing noise into image data including a line object that is teacher data, and the image data mixed with the noise. Is divided into a plurality of slit areas having a longitudinal direction substantially perpendicular to the direction in which the line object extends, and a neural network learning step of learning the plurality of divided slit areas as teacher data. An object recognition method comprising: