JPS60218164A - Picture processor - Google Patents

Abstract

PURPOSE:To process efficiently through an optimum picture processing process by selectively processing a picture process for an input picture in accordance with the complexity of the input picture. CONSTITUTION:A picture including a specific substance 3 in a working space photographed by TV cameras 11, 12 is inputted to a picture processor body 16 through a TV camera switching circuit 13, an A/D converter 14 and a picture memory 15. The picture processor body 16 calculates the complexity of the input picture, processes the input picture in accordance with a picture processing proces proper to the input picture which is selected out of plural picture processing processes in accordance with the complexity of the input picture to recognize the specific substance 3 to be recognized and applies the recognized result to a robot control part 18 to control the working operation of a robot hand 1.

Description

[Detailed description of the invention] [Technical field of invention] The present invention relates to an image processing device useful for object recognition.

[Technical background of the invention and its problems]

Recognizing objects through image processing is very important, for example, when robots perform certain tasks.

For example, when grasping a specific object with a robot hand, input an image of the environment in which the specific object is placed using a TV camera, etc., and distinguish the specific object from other objects from the input image. Image processing plays an important role in detecting the location of a particular object. For this reason, various image processing apparatuses have been developed.

However, images input to this type of image processing apparatus are not necessarily easy to process.

For example, as shown in FIG. 1(a), when multiple object images in an input image are mutually discrete (separated environment), image processing for this can be performed relatively easily. When a plurality of object images partially overlap (contact environment) as shown in FIG. 2B, image processing becomes somewhat complicated. However, as image processing becomes more complex, it is possible to recognize the target object in the contact environment to some extent. However, as shown in Figure 1 (C), when multiple object images in the input image overlap each other (overlapping environment), it is difficult to reliably and reliably recognize the target object through image processing. is extremely difficult.

Conventionally, various image processing processes have been developed to recognize target objects, including objects in the contact environment.However, when recognizing target objects in the separated environment, the image processing process Due to its complexity, it takes a long time to process and has a problem of poor image processing efficiency.

On the other hand, as shown in FIG. 2, an image 3a of a target object 3 is input as shown in FIG. 3 using a visual TV camera 2 attached to the tip of the robot hand 1, and the characteristics and purpose of the image 3a are Image processing has also been proposed in which the target object 3 is recognized by comparing it with previously known features of the object. However, since this processing is for a local visual image, the movement of the robot hand 1 is controlled by trial and error until only the image of the desired object is obtained from an environment where a plurality of objects exist. There were some problems, such as the need to do so.

[Purpose of the invention]

The present invention has been made in consideration of these circumstances, and its purpose is to provide a practical method that can easily and efficiently recognize target objects from input images of objects existing in a wide range of environmental conditions. The objective is to provide a high quality image processing device.

[Summary of the invention]

The present invention calculates the complexity of the scene of an input image given to an image processing apparatus main body having a plurality of image processing processes for the input image, for example, by calculating the statistics of the distribution of figure lines or connected figures in the input image. The image processing apparatus main body detects the presence or absence of a contradiction in the processing result for the input image by the image processing process set for the image processing apparatus main body, and the image processing apparatus main body adjusts the input image according to the complexity of the scene. A recognition target object in the input image is recognized by selectively controlling a plurality of image processing processes to be executed on the image.

〔Effect of the invention〕

Thus, according to the present invention, objects to be recognized that exist under various environmental conditions can be easily and efficiently image-processed and recognized from an input image of the environment using an image processing process that is optimal according to the above-mentioned environmental conditions. It becomes possible to do so. In other words, it is determined from the complexity of the input image whether the environment in which the object to be recognized exists is, for example, a separate environment, a contact environment, or even an overlapping environment, and image processing appropriate for that environment is performed. Since the input image is processed by selecting a process, the object to be recognized can be efficiently tlffied in a short period of time, and great effects can be achieved by using it for robot control, for example.

[Embodiments of the invention]

Embodiments of the present invention will be described below with reference to the drawings.

5- FIG. 4 is a schematic configuration diagram of the embodiment device. This device is
Select a specific object 3 from among multiple objects in a predetermined workspace.
This supports the work of a robot system that selectively grasps only objects with the robot hand device 1.

In this robot system, a first TV camera 11 is provided at the tip of the robot hand 1 to locally view the predetermined work space, that is, the hand work area of the robot hand 1; A second TV is installed above the predetermined work space and has a view of the entire work space.
A camera 12 is provided. Therefore, the image including the specific object 3 in the work space captured by each of the TV cameras 11 and 12 is transmitted to the TV camera switching circuit 1.
3 and is converted into a digital signal suitable for image processing by the A/D converter 14, and then stored in the image memory 15. The image processing device main body 16 is realized by, for example, a microcomputer or a high-speed arithmetic unit for image processing added to the microcomputer, and processes a series of images corresponding to the input images stored in the image memory 156-. configured to perform processing. This image processing device main body 16 includes:
In order to efficiently perform image processing on the input image, a plurality of image processing processes are prepared, and these image processing processes are selectively applied to the processing on the input image. Further, the object database memory 17 registers various characteristic information regarding a specific object 3 to which a predetermined work is performed in the robot system, such as data regarding its size, shape, etc. The image processing apparatus main body 16 executes a series of image processing using the information regarding the object 3 registered in the object database memory 17 as appropriate.

The main image processing of the image processing device main body 16 generally involves calculating the complexity of the input image, and processing the input image from among the plurality of image processing processes described above depending on the complexity of the input image. Select a suitable image processing process,
Processing the input image according to the selected image processing process to recognize the specific object 3 to be recognized,
As a result of the recognition, for example, the specific object 3 in the work space i
The point is that the working operation of the robot hand 1 is controlled by giving information such as the location of the robot hand 1 to the robot control section 18.

Next, image processing of the image processing apparatus main body 1G will be explained with reference to FIGS. 5 and 6. FIG. 5 is a diagram showing the basic flow of this image processing.

First, the image processing apparatus main body 16 receives a visual image of the entire work space from the second TV camera 12, as shown in FIG. 6(a), for example. This input image is first binarized and given as a binarized image in which, for example, the image data representing the object 3 is 111+ and its background data is 110 II. From such input binary image data, the angle of each point on the outline of the figure corresponding to the object is calculated. This angle corresponds to the curvature of the contour line, and is measured as the inside angle (arrow) of the object figure, as shown in FIG. 6(b). Measurement of the angle with respect to point 0 is at two points P, which are slightly away from point 0.

This is done by determining Q and measuring the angle of two straight lines connecting these points P, O, and Q. Alternatively, the angle may be calculated by considering a square area centered on the point O, and using the intersections of its periphery and the target figure as P' and Q'. After that, the above angle is evaluated. Places where this angle is large, for example exceeding 180', are often areas where multiple objects are in contact with each other or overlap each other. is less than a predetermined value, this is determined to be a separate environment in which the objects do not overlap. If there are many parts with large angles, it is determined whether or not there is a hole X as shown in FIG. 6(a) in the figure of the image. This is determined to be an overlapping environment.

If the hole X is not detected in this process, the outline of the figure is divided at the corners of the figure or at the overlapping or contacting parts of objects, as shown in FIG. 6(C). Then, the average length of each of these divided rings is determined, and if the length is long, the degree of overlap between the objects is considered to be small, and this is determined to be a contact environment. If the above-mentioned length is short, □ this is determined to be the above-mentioned type environment.

This judgment is based on the fact that objects overlap each other more □ and □, which increases the number of separation points between the outlines of each figure, and as a result, each dividing line becomes shorter.

However, if it is determined to be a separated environment, the 200 million image is first labeled, and for each labeled figure, the area, perimeter, and length of each side are added. A feature quantity such as height is measured. The target object is identified by comparing it with feature amounts related to the recognition target object stored in advance in the reproach 17. If the target object is not identified through this process, it is determined that the image figure is deformed due to the overlap between the objects, and the work environment is considered to be the contact environment. In this case, contact points and overlapping portions are identified with respect to the image figure, and the figure is separated at each of these parts as shown in FIG. 6(d). Thereafter, each of these separated figures is labeled, and each label is checked against the database to recognize the target object. This processing is image processing for the contact environment.

On the other hand, if it is determined that the input image is in an overlapping environment, or if the target object cannot be recognized even after image processing for the contact environment, the group of shapes is labeled. , a figure with a large block is detected and a command is issued to the robot control device 18 to break up the block. In response to this command, the robot hand 1 breaks up the mass of objects and separates each object. Thereafter, image input for these objects is executed again and the above-described process is repeated.

Now, when a target object is detected by each of the image processing processes (processing for a separated environment and processing for a contact environment), the robot control device 18
is driven, and its hand portion is moved onto the target object 3. In this state, the target object 3 is detected by the first TV camera 11 installed at the tip of the robot hand 1.
A visual image of the hand is input. Then, it is determined whether the features of this input image match the features of the recognition target object prepared as the database, and if a match is detected,
A predetermined operation is performed on the target object 3. Furthermore, if the hand visual image does not have the characteristics of the target object, it is assumed that there was an error in the object recognition process described above, and the image process is performed again. At this time, the image processing that led to obtaining the hand visual image is checked, and if that processing is from image processing for the separated environment, then image processing for the contact environment is performed.

Furthermore, when the hand visual image is processed by the image processing for the contact environment, the overlapping environment is then processed.

In this way, this device processes the input image while selecting the image processing process for the input image depending on the complexity of the input image. On the other hand, there is no risk of applying an unnecessarily complicated image processing process, and there is no waste, such as repeatedly performing meaningless image processing on input images in the overlapping environment. Then, it becomes possible to effectively perform optimal image processing according to the complexity of the input image.

Therefore, it is possible to improve the processing efficiency for input images, and to shorten the time required for image processing.
Furthermore, great practical effects such as being able to reliably recognize and process a target object can be achieved. Therefore. It can exhibit great effects as a recognition processing means for controlling various robots.

Note that the present invention is not limited to the embodiments described above. For example, it goes without saying that the feature extraction process for graphics in the input image and the types of graphic features to be extracted are not limited to those shown in the embodiments. Further, in the embodiment, processing has been described for processing two-dimensional images, but for example, images may be input from a plurality of different corners 13 of the work space in order to perform stereoscopic processing. Also, an image processing process that changes the image input environment by changing the lighting conditions of the work space, performing selection processing on the color of objects using filters, and even changing the image input angle and lens focus of the TV camera. It is also possible to incorporate as appropriate. Furthermore, for complex input images, it is also possible to employ an image processing process such as a region segmentation method. In short, the type and number of image processing processes used in this device can be determined according to the device specifications, and the criteria for selecting the image processing process and the criteria for determining the complexity of the input image, etc. Various modifications can be made without departing from the gist of the invention.

[Brief explanation of the drawing]

Fig. 1 is a diagram showing examples of input images of different complexity, Fig. 2 is a diagram showing image input means using a TV camera installed at the tip of a robot hand, and Fig. 3 is a diagram showing an example of a visual image of the hand. , FIG. 4 is a schematic configuration diagram of an apparatus according to an embodiment of the present invention, and FIG.
The figure is a diagram showing the flow of image processing in the embodiment apparatus, and FIG. 6 is a diagram showing an example of a processed image in the embodiment apparatus. 1... Robot hand, 3... Target object, 11.
...first TV camera, 12...second TV camera,
13...TV camera switching circuit, 14...A/D converter, 15...image memory, 16...image processing device main body, 17...object database memory, 18.
...Robot control device. Applicant's representative Patent attorney Takehiko Suzue 15- Section 6 (a) (C) (b) (d)

Claims (1)

[Scope of Claims] (1) An image processing device main body that includes a plurality of image processing processes for an input image, means for calculating the complexity of a scene of an input image given to the image processing device main body, and a An image processing apparatus comprising means for selectively controlling an image processing process that the image processing apparatus main body executes on the input image according to the complexity of the input image. (The image processing device according to claim 1, wherein the complexity of the scene of the two-manual image is detected from the statistics of the distribution of figure line segments or connected figures in the input image. (3) Input The image processing device according to claim 1, wherein the complexity of the scene of the image is detected from the presence or absence of inconsistency in the processing results for the input image by the image processing process set for the image processing device main body. .