JPH10160464A - Apparatus and method for recognizing position and posture of object to be recognized - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize or measure three-dimensional position and posture of an object of a specific shape such as a column or the like from a camera image of the object, by obtaining position and posture information on the object from a most fit projected image through a specific process. SOLUTION: The image characteristic is extracted from a photographed image of an object to be recognized of a specific shape by a slit pattern- extracting means 4, a meridian-extracting means 5, a characteristic line-applying means 6, etc. A model projected image-generating means 7 calculates a projected image of a three-dimensional model of the object to be recognized. The model projected image-generating means 7 corrects position and posture of the three dimensional model in accordance with an extracted result by the slit pattern- extracting means 4 and an extracted result by the meridian-extracting means 5, transmits projected images of various positions and postures of the three dimensional model to the applying means 6, and obtains and outputs information of the position and posture of the three-dimensional model judged as best fit. Accordingly, the three-dimensional position and posture of the object can be recognized or measured.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a position / posture recognition device and a position / posture recognition apparatus for controlling a power distribution work robot or an intelligent robot, for example, by setting a work target such as an insulator as a recognition target object and recognizing the position / posture of the object by an image. About the method.

[0002]

2. Description of the Related Art With the development of robot technology, robots have been active in various fields as well as in the industrial field. In recent years, research has been actively conducted to substitute robots for parts replacement and maintenance of equipment in dangerous environments such as high places.

A robot (work robot for a dangerous environment) for performing operations such as replacement of parts of a facility or maintenance in such a dangerous environment is a system in which the operation of the robot is remotely controlled by a human hand. Has been taken,
Since there is a problem in handling such as requiring human labor and skill in operation, the work robot is equipped with a TV camera, and autonomously recognizes a target object using the image and performs work. There is a strong demand for the realization of an autonomous work robot capable of performing the task.

In order to realize an autonomous work robot, a position and orientation recognition device for recognizing a three-dimensional position and orientation of a work target from an input image by image processing is indispensable. If the shape of the target object is known, the position and orientation of the object can be measured even by a monocular camera, and studies on monocular object recognition are being actively conducted.

[0005] In particular, the artificial object has a cylindrical shape or a rotationally symmetric (point-symmetric) shape, but is not merely a cylindrical shape, but a shape formed by stacking cylinders or truncated cones in the axial direction. There are many objects having a shape (for example, a gourd shape) having a swelling or depression in the axial direction. Such a shape is hereinafter referred to as a “generalized cylinder”.

[0006] Recognition of the position and orientation of these "generalized cylinders" is an important factor for realizing an autonomous work robot. As an example of the use of the autonomous work robot, for example, consider a robot performing distribution line construction on a utility pole, and the shape of an object to be recognized by the robot in order to perform work is limited.

[0007] Among them, the pillars and insulators considered to be the most important recognition objects are generalized cylindrical objects.
As a method for recognizing the three-dimensional position and orientation of an object based on image information about a known object such as an insulator on a telephone pole, which is captured by a monocular camera, a projected image is represented as a graphic. There is a method of applying a graphic pattern matching method.

In this case, the position and orientation of the object are recognized by comparing the shape of the projected image of the object with the previously stored pattern of the appearance of the object. However, when an object having a “generalized cylinder” shape can freely change its position and orientation three-dimensionally, there are the following five types of changing factors in the appearance of the object. That is, there are a total of five types of change factors: “size”, “inclination on the screen”, “inclination in the depth direction”, “position in the center axis direction”, and “position in the direction perpendicular to the center axis”.

[0009] When all the changing factors are considered, the pattern of appearance becomes infinite. Therefore, when a graphic pattern matching method is used, “fix the size” or “rotate in the depth direction” It was necessary to limit the patterns to be prepared by a method such as "fixing".

[0010] Therefore, as another method for recognizing the three-dimensional position and orientation of an object, a two-dimensional image feature point indicating the shape characteristic of the target object is extracted from the image, and the target is extracted using a combination method. A method of correlating with a feature point of an object and recognizing the three-dimensional position and orientation of the object is often used.

However, when an object having a “generalized cylinder” shape is to be recognized by extracting feature points of the shape, the object having the “generalized cylinder” shape is to be recognized. Since the surface has a smooth shape, there are no protrusions or corner points that can be feature points, and therefore, there is a problem that it is difficult to extract feature points from an image.

There is also a method in which a characteristic straight line or a characteristic curve is extracted instead of extracting feature points, and recognition is performed based on correspondence with a recognition target object. In this method, for example, when the target object has a cylindrical shape, an elliptical shape or a parallel line is extracted, and the object is recognized from the shape.

In this case, in order to perform accurate measurement, it is desirable that the contour of the elliptical shape be extracted along the entire circumference. However, when an object is in contact with the top and bottom of the cylinder, the entire circumferential contour cannot be extracted, and is often extracted as part of an elliptical arc. Becomes difficult. In the case of "cylinder" or "generalized cylinder", the contours on both sides are not contours appearing as ridges peculiar to the object, but are contours called occlusion contours. If the difference is not clear, it is difficult to extract even if image processing for extracting an elliptical contour line is used as it is.

For this purpose, for example, Japanese Patent Publication No. 6-5628
In Japanese Patent Publication No. 7 (Goto: position detection device for a cylindrical object), an object to be recognized is limited to a columnar object, and active illumination is projected from an obliquely upward direction with respect to the cylinder axis, and upward. A technique is employed in which a projection image of a cylindrical object is photographed from an attached camera to actively extract a shielding contour as a shadow. Then, a three-dimensional position is detected from the shape of the projected image.

However, in this method, the positional relationship between the object and the camera is extremely limited. Therefore, when the object and the camera are in a distant positional relationship, it is impossible to apply the method. For example, when trying to use image processing technology for object recognition for an autonomous work robot performing distribution line construction on a telephone pole, the positional relationship between the object to be recognized and the camera is unknown at first, and This is because the position of the camera with respect to the object is not in the upper direction of the object, but in the horizontal direction, and it is considered that the camera is often taken from a location distant from the recognition target object with a horizontal field of view.

[0016]

One of the objects that must be recognized in order to realize a robot for use in performing distribution line construction on a power pole is, for example, an insulator. When capturing an object with the shape of a `` generalized cylinder '' using a camera and recognizing it from the image, a method using a graphic pattern matching method, a method using extraction and combination of feature points and feature lines of the image, or Application of an image recognition technology such as a method of irradiating active light can be considered.

However, if the conventional graphic pattern matching method is used, the number of patterns to be prepared becomes enormous, and if the method of extracting and combining feature points is used, the recognition target is "generalized." It is difficult to extract feature points and feature lines from images in the case of an object with a “cylindrical” shape, and the method of irradiating active light requires that the camera position be above the recognition target. There was a problem of being limited.

Therefore, in order to realize an autonomous work robot for use in performing distribution line work on utility poles, there is an urgent need to establish an insulator recognition technique that must be able to be recognized. Therefore, the purpose of the present invention is to use a "column"
It is an object of the present invention to provide a position and orientation recognition apparatus and a position and orientation recognition method that enable the position and orientation of a shape and a “generalized cylinder” shape object to be measured from a camera image.

[0019]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides an image input means for inputting an image of a specific shape of a recognition target object, and a slit for actively inputting the specific shape of the recognition target object. A slit light irradiation device that irradiates light, a slit pattern extraction unit that extracts a projection pattern of slit light by image processing from an image obtained from the image input unit, an image obtained from the image input unit, and a slit pattern extraction unit Meridian extraction means for extracting a contour intersecting the projection pattern of the slit light of the object to be recognized from the information obtained from the object;
A projection for holding a three-dimensional model, calculating a projection image of the three-dimensional model, correcting the three-dimensional position and orientation of the three-dimensional model based on information indicating the degree of fitting, and comparing the three-dimensional model with the recognition target object. A model projection image generating means for obtaining an image, and comparing the image characteristics of the recognition target object with the projection image obtained by the model projection image generating means from the image obtained from the image input means to determine the degree of fitting. Discriminating and correcting the position and orientation of the recognition target model to fit a characteristic line generated from the shape of the recognition target object; and feature / position information of the best three-dimensional model projection image of the fitting degree. And a result output means for obtaining and outputting the result from the model projection image generating means.

In the present invention, the image feature of the image obtained from the image pickup means for picking up the recognition target is compared with the projection image of the prepared three-dimensional model of the recognition target, and the degree of fitting is verified. The projection image obtained by sequentially correcting the position and orientation of the model according to the result is again compared with the image feature to verify the degree of fitting, the best fitting projection image is obtained, and the best fitting projection image obtained is obtained. , The information on the position and orientation of the recognition target is obtained to recognize the three-dimensional position and orientation of the recognition target object.

More specifically, the image feature of the object to be recognized is extracted from an image of the object to be recognized having a specific shape by slit pattern extracting means, meridian extracting means, characteristic line fitting means, etc., and model projection is performed. The image generating means calculates a projection image of the three-dimensional model using the held three-dimensional model of the recognition target object having the specific shape,
The three-dimensional position and orientation of the three-dimensional model are corrected based on the degree of fit determined by the fit determining means to obtain a projection image for comparison and collation with the recognition target object.

In addition, a contour line that is difficult to extract is irradiated with slit light, and the position of the contour line that intersects the slit pattern is extracted from the shape of the slit pattern. Is extracted.

The model projection image generating means corrects the position and orientation of the three-dimensional model according to the extraction result of the slit pattern extracting means, and corrects the position and attitude of the three-dimensional model according to the extraction result of the meridian extracting means. The projection images of the various positions and postures of the three-dimensional model are transmitted to the feature line fitting means, and information on the position and posture of the three-dimensional model determined to have the best fit is obtained and output.

As described above, image information obtained by capturing an image of a recognition target is obtained, image characteristics of an image obtained from the image information are extracted, and the extracted image characteristics are used to project the prepared three-dimensional model of the recognition target. In comparison with the image, the degree of the fit is verified, the position and orientation of the model are sequentially corrected, the best fit projection image is obtained, and the position and orientation are obtained from the obtained best fit projection image. Thus, the three-dimensional position and orientation of the recognition target object are recognized.

Therefore, the position and orientation of the object to be recognized having a "cylindrical" shape or a "generalized cylindrical" shape can be measured from the image, opening the way to the realization of an autonomous work robot.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In the present invention, image characteristics of an image obtained from an image pickup means for imaging a recognition target are compared with a projection image of a prepared three-dimensional model of the recognition target, and the degree of the fitting is verified. Then, the projected image obtained by sequentially correcting the position and orientation of the model according to the verification result is again compared with the image feature to verify the degree of fit, the best fit projected image is obtained and the obtained best image is obtained. By obtaining information on the position and orientation of the recognition target from a projection image that fits well, the three-dimensional position and orientation of the recognition target object are recognized.

More specifically, the object to be recognized has a specific shape, and the image feature of the object to be recognized is extracted from an image of the object to be recognized by slit pattern extracting means, meridian extracting means, feature line fitting means and the like. Is extracted. The model projection image generation means calculates a projection image of the three-dimensional model using the held three-dimensional model of the recognition target object having the specific shape, and based on the degree of fit determined by the fit determination means. 3D model 3
A dimensional position and orientation are corrected to obtain a projection image for comparison and collation with the recognition target object.

In addition, a contour line that is difficult to extract is irradiated with slit light, and the position of the contour line that intersects the slit pattern is extracted from the shape of the slit pattern. Is extracted.

The model projection image generating means corrects the position and orientation of the three-dimensional model according to the extraction result of the slit pattern extracting means, and corrects the position and attitude of the three-dimensional model according to the extraction result of the meridian extracting means. The projection images of the various positions and postures of the three-dimensional model are transmitted to the feature line fitting means, and information on the position and posture of the three-dimensional model determined to have the best fit is obtained and output.

As described above, image information obtained by capturing an image of a recognition target is obtained, image characteristics of an image obtained from the image information are extracted, and the extracted image characteristics are projected onto the prepared three-dimensional model of the recognition target. Compared with the image, verifying the degree of the fit, sequentially correcting the position and orientation of the model, finding the best fit projection image, and finding the position and orientation from the found best fit projection image. By doing so, the three-dimensional position and orientation of the recognition target object are recognized.

Therefore, the position and orientation of an object to be recognized having a "cylindrical" shape or a "generalized cylindrical" shape can be measured from an image, opening the way to the realization of an autonomous work robot. Hereinafter, specific examples of the present invention,
This will be described with reference to the drawings.

FIG. 1 is a schematic block diagram of a columnar object recognition apparatus to which the position and orientation recognition apparatus of the present invention is applied. First, the configuration of the present apparatus shown in FIG. 1 will be described.
The columnar object recognition device includes a TV camera 1, a slit light irradiating unit 2, an image input unit 3, a slit pattern extracting unit 4, a meridian extracting unit 5, a characteristic line fitting unit 6, a model projection image generating unit 7, and a result outputting unit 8. Consists of

The TV camera 1 is an image pickup device such as a video camera, and is installed so that a columnar object to be recognized is input. The TV camera 1 photographs a region near the target object and converts the photographed image. It is output as an image signal.

The slit light irradiating means 2 is a device for projecting cross-shaped light on the surface of the object to be recognized, and is provided with a slit device so that the slit light irradiation pattern becomes cross-shaped. It is arranged in the vicinity of the camera so as to irradiate cross-shaped light in a pattern shape across the contour of the target object that is difficult to extract by image processing alone. Of course, the positional relationship between the TV camera 1 and the slit light irradiating means 2 is such that the image of the light of the cross-shaped pattern projected on the surface of the recognition target object by the slit light irradiating means 2 has a positional relationship that can be imaged by the TV camera 1. Shall be kept.

The slit light irradiating means 2 may be formed by using a relatively powerful illuminating device such as an incandescent lamp or a mercury lamp. In this embodiment, the description will be made assuming that a laser slit light irradiating device is used.

The image input section 3 is for digitizing and inputting an image signal transmitted from the TV camera 1, and controls the irradiation switch of the slit light irradiation means 2 to irradiate the slit light. It has a function of selecting and inputting an image at the time or an image at the time when the slit light is not irradiated.

The slit pattern extracting means 4 processes the digital image transmitted from the image input unit 3 when the slit light is irradiated and the digital image obtained when the slit light is not irradiated (processing method). Will be described later in detail), a slit pattern in the image is extracted, the position information is received from the model projection image generating means 7, and a digital image at the time when the slit light is not irradiated is received from the image input unit 3. Has a function of extracting a contour line intersecting the slit pattern by performing image processing on the image using the position information, and in response to a request from the model projection image generation means 7 which holds the extraction result in a memory. It has a function of transmitting data to the server.

The feature line fitting means 6 extracts the image features of the digital image at the time when the slit light transmitted from the image input unit 3 is not irradiated, and extracts the recognition target object transmitted from the model projection image generating means 7. It has a function of comparing the projected position of the corresponding feature line of the three-dimensional model with the image feature (the processing method will be described later in detail), and calculating an optimal fitting position.

The model projection image generation means 7 holds the structure data of the three-dimensional model of the object to be recognized and the data of the position and orientation, and generates a projection image of the model. Further, the three-dimensional position of the model is calculated back from the data indicating the two-dimensional existence position of the model transmitted from the slit pattern extracting means 4 and the meridian extracting means 5, and the position and orientation data of the three-dimensional model of the object to be recognized are obtained. It also has the function of correcting

The result output means 8 has a function of outputting a final recognition result. For example, it outputs a three-dimensional position and orientation of the recognition object as numerical values, or outputs the recognition result on an input image. For example, the model projection image is overwritten and displayed based on the model.

The operation of the present invention having the above configuration will be described. As an operating environment of an autonomous work robot equipped with this system as a robot visual system, distribution line construction on a utility pole will be described as an example. In this operating environment, the autonomous work robot needs to recognize and position the work object. On a utility pole, for example, there are a number of "generalized cylinder" -shaped objects such as insulators, and their positions are specified.

If the telephone pole is standing perpendicular to the ground,
In an image obtained by shooting the image input TV camera 1 while keeping it horizontal, the insulator attached vertically to the ground is shot vertically. When a columnar object such as an insulator is a recognition target, it is difficult to extract the axial contour of the cylinder (vertical contours on both sides) by using only image processing.

At this time, the slit irradiating device (slit light irradiating means 2) is installed so that the slit pattern crosses the outline on both sides of the cylinder. The autonomous work robot is mounted on an aerial work platform of a work vehicle. An aerial work platform has an arm that can be bent and stretched.The base of the arm is held so that it can be turned on a work vehicle, and a basket on which a worker rides is attached to the free end of the arm. However, when the work robot is mounted on the tip of the arm to work, the basket may be removed without any problem.

The operation of the aerial work platform is performed by an operator. First, the operator places the work vehicle near a utility pole, and raises the free end of the work platform at a high position toward the utility pole. Then, the position of the arm tip is adjusted so that the work robot comes to an appropriate position while considering the movable range of the robot and the influence of obstacles. Since the surrounding environment varies from place to place, it is impossible to install a robot at exactly the same position every time work is performed, and the arrangement on the utility pole also varies from place to place.

However, since the work to be performed is predetermined, it is possible to determine in advance what work object needs to be recognized, and to make the object fit on the screen,
The position and posture of the robot can be set. At this time, it is also possible to set the position of the robot so that the slit is irradiated on the recognition target.

In such a case, the object to be recognized by the present recognition system can be assumed to be within the screen, and the spatial range in which the object can exist is also assumed. can do.

Because, as described above, the positional relationship between the working robot and the object is related to the movable range of the robot.
This is because the operator sets the work robot so as to be within a certain range.

FIG. 2 is an overall processing procedure of the position and orientation recognition apparatus for insulator recognition constructed according to the present invention. FIGS. 3 to 7 are schematic diagrams showing the state of the image at each time showing the transition of the process for explaining the flow of the process.

The position / posture recognizing device for insulator recognition includes an image input step (step in FIG. 2) in which the slit light irradiating means 2 irradiates the insulator with the slit light and irradiates each image with the slit light from the TV camera 1. S21), slit pattern extraction processing by the slit pattern extraction means 4 for performing slit pattern extraction processing on the slit irradiation image (step S22 in FIG. 2), and by finding the end points of the pattern, the shielding contour of the insulator on the image is determined. The end point position fitting processing (step S23 in FIG. 2) in the model projection image generating means 7 which is the finding processing, and the meridian extraction processing in the meridian extracting means 5 for extracting the meridian based on the obtained end point positions (FIG. 2) In step S24), the three-dimensional position and orientation of the recognition target model are applied to the meridians using the extracted meridians. Model projection image generation means 7 for modifying the
2 (step S25 in FIG. 2), which is a process for determining two types of parameters, "position of the cylindrical object in the direction of the central axis" and "rotation angle in the depth direction". The characteristic line fitting processing by means 6 (step S26 in FIG. 2), the results obtained through these processings (“size”, “screen tilt”, “depth tilt”, “center axial direction” , And a position in the vertical direction with respect to the central axis), which is output processing (step S27 in FIG. 2) which is processing by the result output means 8 for outputting a total of five types of variable element determination information. , Get the final result. The details of the processing will be described step by step.

[Image Input (Step S21)] First, the TV camera 1 is pointed at the insulator so that the insulator to be recognized is photographed, and the insulator is imaged by the TV camera 1 at this position, and three types of the insulator are obtained. (Image input; step S21). The three types of images here are:
One is an image captured as it is without using the slit light irradiating means 2, and the other is an image captured in a state where the slit light of the cross-shaped pattern is irradiated on the object using the slit light irradiating means 2. . Thereby, of the three types of images,
In the two types, images in a state where the slit light is applied to the object can be obtained.

It is assumed that the two images in the slit light irradiation state have different positions of the slit light of the cross-shaped pattern with respect to the object. For this purpose, upon imaging, the slit light irradiating means 2 irradiates the object with slit light from two different positions. For example,
From the position of the TV camera 1 toward the target from the position 30 cm to the lower right and 30 cm to the lower left from the camera.
It is said that irradiation is performed from two points from a position separated by a distance of 1 cm.

Therefore, in order to achieve this, for example, two slit light irradiating means 2 are used, and their installation positions may be one each on the left and right with respect to the camera optical axis (slit light irradiating means 2). Fixed support method). Of course, if a configuration is provided in which a left-right moving mechanism is provided so that slit light can be projected onto the object from different positions on the left and right with respect to the camera optical axis, even one slit light irradiating means 2 can be used (slit light irradiating means moving method) .

Although any structure may be used, the point is that the viewpoint of the camera is not changed, and the slit light can be irradiated separately from two different positions at different times. The reason why the slit light must be irradiated from these two different positions is that the vertical contour green (right meridian) on the right side of the object to be recognized (here, the insulator) and the vertical contour line on the left (left meridian) ) Can be individually extracted.

For this purpose, the two slit light irradiating means 2 alternately irradiate the object to be recognized with a cross-shaped pattern at a time interval, and an image of each cross-shaped pattern projected state is obtained by the TV camera 1. Control the system to take an image.

As described above, since the operator installs the robot so that the positional relationship between the robot and the object is within a certain range in relation to the movable range of the robot and the like, the laser slit light is applied to the object. It is quite possible for the system to be set up so that the object is illuminated. Also,
Even if the slit light does not hit, the operator only needs to correct the position of the robot or the TV camera.

An image signal obtained by capturing each of such images with the TV camera 1 is input to the image input unit 3.
Captures and converts each into a digital image. That is, the image input unit 3 outputs an image photographed without irradiating slit light (hereinafter, referred to as an “image without slit”) and an image irradiated with slit light from the left side (hereinafter, “left slit irradiated image”). ) And an image illuminated with slit light from the right side (hereinafter referred to as “right slit illuminated image”), and these are converted into digital images.

During the input of the above three types of images, the image input TV camera 1 must not move spatially. However, if it is necessary to move the TV camera 1 mechanically,
What is necessary is just to comprise so that correspondence of the pixel position between three types of images can be calculated using a movement parameter. Also, even when the TV camera 1 does not move spatially, if there is a difference in the time to shoot three types of images, a difference may appear in the background depending on the background, but a difference of about several seconds (usually a digital A background difference due to a time sufficient for capturing a plurality of images) can be dealt with by a slit pattern extraction process described later, and the difference can be absorbed.

FIG. 3 is a schematic diagram of an image without a slit, and FIG. 4 is a schematic diagram of an image irradiated with a slit. When a laser projector is used as the slit light irradiating device 2, measures such as mounting an interference filter having a laser wavelength transmission region on the front surface of the lens of the TV camera 1 so that an irradiation pattern is clearly photographed. May be taken.

[Slit pattern extraction processing (step S
22)] Next, slit pattern extraction processing is performed on the left and right slit irradiation images (step S22 in FIG. 2).

FIG. 8 shows the details of the processing procedure of the slit pattern extraction processing. In the slit irradiation image, the luminance value of the pixel of the portion irradiated with the slit pattern is brighter than the image without the slit. Therefore, if the difference between the luminance values of the corresponding surface elements of the image without slits is calculated from the luminance value of the slit irradiation image, only the slit pattern portion appears. An image (hereinafter, referred to as a difference image) in which the difference between the luminance values of the pixels is obtained is created, and an end point position of the recognition target object is extracted by extracting a slit pattern portion.

Therefore, first, a difference image between the left and right slit irradiation images and the non-slit image is created, and the difference image is used in the subsequent processing (steps S81a and S81b in FIG. 8). The difference calculation performed here is not the absolute value of the difference between the images but is the following calculation.

When the surface element value of the slit irradiation image of the pixel at the coordinates (x, y) is P (x, y) and the surface element value of the corresponding surface element of the image without slit is P ′ (x, y) , The pixel value of the difference image is obtained by the following calculation.

P (x, y) −P ′ (x, y) When P (x, y)> = P ′ (x, y) 0 P ′ (x, y)> P (x, y) Next, the difference image is binarized. The binarization threshold is determined, for example, by the Otsu discriminant analysis method (eg, “Image Analysis Handbook”, University of Tokyo Press, 1991, Mikio Takagi
(See, for example, supervision by Y. Shimoda) (step S82 in FIG. 8).
a, Step S82b). Next, a labeling process (a process of dividing a binarized pixel into a continuous area) is performed by eight concatenation, and the label is divided into labels (steps S83a and S83b in FIG. 8).

The label extracted from the difference image may include a change in the background. For example, it is a swing of a tree leaf or a flicker of a screen. From these labels, it is necessary to determine the cross pattern made by the slit light. For this purpose, for example, the cross pattern is identified by the following method.

[Cross-Cross Pattern Discrimination] (i) Removal of Noise Label with Small Area Since the label generated by the slit light has a sufficiently large number of pixels (area), the number of surface elements of the extracted label is set to two types, large and small, with a certain threshold. Is determined, and the label determined to be a small label as a result is removed. The determination of the discrimination threshold may be performed using Otsu's discriminant analysis method, similarly to the difference processing.

(Ii) Removal of Noise Label with Large Label Density The value obtained by dividing the number of pixels included in the label by the area of the polygon circumscribing the label is defined as the label density. small. On the other hand, since a cluster of circles appearing as noise or a rectangular noise label has a high label density, it can be identified as a cross shape by judging from the magnitude of the label density. Therefore, labels having a high label density are removed.

(Iii) Extract a label with the largest area of the circumscribing polygon If there are a plurality of candidates that have passed the above processes (i) and (ii), the area of the circumscribing polygon is Extract the largest label.

The cross pattern is extracted by the above processing (steps S84a and S84b in FIG. 8). [End Point Position Fitting Process (Step S23 in FIG. 8)] After the slit pattern extraction process is completed, the process proceeds to the end point position fitting process. The end point position fitting is a process of finding a shielding contour of an insulator on an image by finding an end point of a cross pattern.

In the cross pattern extraction processing as described above, the simplest figure effective as a polygon for calculating the label density is a circumscribed rectangle. FIG.
Shows a schematic diagram when calculating the label density with a circumscribed rectangle. The polygon circumscribing this label is not limited to a rectangle, and may be, for example, a convex polygon circumscribing the extracted label. FIG. 18 is a schematic diagram when the extracted labels are circumscribed convex polygons.

The left end position and the right end position of the label extracted by the above processing are measured. When processing the difference image on the left side, the left end position of this region is extracted, and when processing the difference image on the right side, the right end position of this region is extracted (steps S85a and S85b in FIG. 8).

The above processing is performed on the left and right difference images. The following two values are obtained from the extracted left and right end point positions. (1) The width of the left and right end points of the slit light is calculated, and the depth position of the recognition target object is calculated (step S8 in FIG. 8).
6).

(2) The center positions of the left and right end points are calculated and regarded as the position of the center of gravity of the object to be recognized. The three-dimensional position of the recognition target object is calculated from the position of the center of gravity (step S in FIG. 8).
87).

The object to be recognized in this example is an insulator on a telephone pole as shown in FIG. 3, and the meridian of this insulator is called a shielding contour, and is like a ridgeline or a corner line of the object. Since it is not a line held by the object itself but a line that appears when the target is projected on the image, it is a line that is difficult to extract by image processing alone depending on the background and lighting conditions.

However, extracting the slit pattern from the difference image between the image irradiated with the slit light and the image not irradiated as described above and extracting the end point position of the extracted slit pattern has conventionally been difficult. It is possible to accurately determine the shielding contour. Note that the two values obtained here are provisional values for performing subsequent processing,
It is corrected to an accurate value in the subsequent processing.

Note that step S86 and step S86 in FIG.
87 includes a procedure for calculating the three-dimensional position and orientation of the recognition target object from the information on the position in the image. Also in the subsequent processing, there are many procedures for obtaining a projection image of the recognition target object and calculating the three-dimensional position and orientation of the recognition target object from information on the projection position. Therefore, these calculation methods will be collectively described in detail here. Note that these calculations are processed inside the model projection image generating means 7.

[Calculation Method] FIG. 9 is a diagram showing the internal configuration of the model projection image generating means 7. As shown in the figure, the model projection image generation means 7 includes a model position calculation means 91, a model position and orientation parameter storage means 92, a model structure parameter storage means 93, a model three-dimensional position calculation means 94,
It comprises a camera parameter storage means 95 and a model projection position calculation means 96.

Of these, the model position calculating means 91
Receives data on the position in the image to be recognized from the slit pattern extracting means 4 and the meridian extracting means 5,
Using these and the camera parameters stored in the camera parameter storage means 95, the three-dimensional position and orientation of the model are calculated.

The model position and orientation parameter storage unit 92 holds the three-dimensional position and orientation parameters of the model calculated by the model position calculation unit 91. The model structural parameter storage unit 93 stores and holds the position information of the model M as the three-dimensional coordinates of each vertex. The model three-dimensional position calculation unit 94 stores the model M at the arbitrary position and orientation in the model. This is for calculating the three-dimensional position of each constituent point (each vertex in the example of FIG. 10).

The parameters of the three-dimensional position and orientation of the model can be represented by a three-dimensional transformation matrix (matrix of 4 rows and 4 columns). 3 for each component point at any position and orientation of the model
It utilizes the ability to calculate dimensional position.

The camera parameter storage means 95 stores the T
The camera parameters of a camera input system of the present system including a V camera 1 and an image input unit 3 are stored.
The model projection position calculation means 96 includes the camera parameters of the camera input system stored in the camera parameter storage means 95 and the respective configurations at arbitrary positions and orientations of the model obtained by the model three-dimensional position calculation means 94. The projection position of the model is calculated from the three-dimensional position of the point.

In this specific example, the model projection image generation unit 7
Holds a three-dimensional model of the object to be recognized, for example, as a wireframe model as indicated by reference numeral M in FIG. The example shown in FIG. 10 indicates that when the recognition target object is an insulator of a power distribution structure, the model of the recognition target object is approximated as a combination of regular 30-sided truncated pyramids.

The position information of the wire frame model M is held in the model structure parameter storage means 93 as three-dimensional coordinates of each vertex. Further, camera parameters of the camera input system including the TV camera 1 and the image input unit 3 are measured in advance and stored in the camera parameter storage unit 95.

The camera parameters are parameters representing the projection relationship from the coordinates (X, YZ) in the three-dimensional space to the projection position (x, y) on the screen. For example, the following Expressions (1) and (2) represent twelve constants {pi} i = 1, 2,
.., 12 represent the projection relationship.

[0084]

(Equation 1)

These parameters {pi} i = 1,2,2
……, 12 are determined by calculating points whose three-dimensional coordinates are known
It is sufficient to take images of points or more, measure the projection position, and consider the equations (1) and (2) as linear equations and obtain them by the least square method. Alternatively, a camera position vector C (here, the camera position indicates the position of the camera focal point),
Given the focal length F, the vector A in the optical axis direction of the camera, and the image center position (xo, yo), the parameters {pi} i = 1, 2,..., 12 can be obtained from the following equation. .

First, from the optical axis direction vector A of the camera,
An X direction vector Uo and a Y direction vector Vo of the camera light receiving element surface are determined. These may be determined so that the vector Uo, the vector A, and the vector Vo form a right-handed rectangular coordinate system. At this time, the parameters are calculated by the following equations. In the following equations, the vector elements are represented by adding the vector signs of arrows on the element codes.

[0087]

(Equation 2)

Here, (•, •) indicates the inner product of the vectors. It is easy to convert these 12 parameters into physical camera parameters. Camera position vector C, focal length F, camera optical axis direction vector A,
The image center position (xo, yo) can be obtained by the following equation.

[0089]

(Equation 3)

In this specific example, the camera parameters will be described below based on expressions (1) and (2). When data relating to the position in the image to be recognized is transmitted from the slit pattern extracting means 4 and the meridian extracting means 5 to the model projection image generating means 7, the model projection image generating means 7 converts the data into a model position calculating means. Give to 91. The model position calculation means 91 calculates the three-dimensional position and orientation of the model using the data on the position of the recognition target in the image and the camera parameters stored in the camera parameter storage means 95. This calculation method will be described later in detail.

The three-dimensional position and orientation parameters of the model calculated by the model position calculating means 91 are stored in the model position and attitude parameter storage means 92.
Since the parameters of the three-dimensional position and orientation of the model can be expressed by a three-dimensional transformation matrix (matrix of 4 rows and 4 columns), multiplication of the transformation matrix for each component point of the model makes it possible to obtain an arbitrary model of the model. The three-dimensional position of each constituent point (each vertex in the example of FIG. 10) at the time of the position and posture can be calculated. This process is performed by the model three-dimensional position calculating means 94.

At this time, the model projection image is calculated according to the following procedure. First, for each of the constituent points {Pi} i = 1, 2,... N calculated by the model three-dimensional position calculating means 94, the projected position of each of the constituent points is calculated based on Equations (1) and (2). p
i｝ i = 1, 2,... N are obtained.

Next, a projected image of each side is obtained by approximating a straight line in the image between the projected positions of the respective constituent points. From the projection image of the recognition target model generated in this manner (all lines are drawn at this time), the outermost line on the image is selected, and an outer contour line is generated.

In order to identify the insulator by image recognition and to make it easy to identify its posture and position, in the case of a wiring facility in which an autonomous work robot is to work, as shown in FIG. In some cases, a surface in which a band-shaped ring-shaped coating color band is formed in advance is used. That is, in this specific example, when the recognition target is an insulator as shown in FIG.
Information on the position of the paint band plays an important role in recognition of such insulators.

Therefore, if there is a line that plays an important role in recognition on the surface of such a recognition target object, in the present invention, the line is also extracted and used as an outer contour line. FIG. 11 shows a projection image of the recognition target model generated in this manner. In this specific example, since the insulator as shown in FIG. 3 is to be recognized, a large number of lines are also used in the circumferential direction to express the shape of the constriction on the upper part of the insulator.

However, if the object to be recognized has a simple shape close to a cylindrical shape, the shape can be expressed by a small number of lines (that is, a small number of constituent points) in the circumferential direction. It can be expressed by regular polygonal prism. This has significant implications. That is, an extremely large amount of calculation is required to strictly determine the outer contour of an object having a complicated shape according to the projections of Expressions (1) and (2). However, by approximating the model with the component points and the linear approximation in this way, an approximate contour projection image of the recognition target object can be generated with a small amount of calculation. That is, an approximate contour projected image of the recognition target object can be generated easily and in a short time.

The model projection image generation section 7 holds the model as three-dimensional coordinate information. Therefore, it is easy to spatially move or rotate the model, and five kinds of change factors of the appearance of the model (FIG. 12).
(See Reference.) A projection image corresponding to each can be easily generated.

For example, in order to give the model a rotation corresponding to the tilt θ on the screen, the rotation from the viewpoint position obtained by the equation (9) to the center position of the current object to be recognized is performed.
What is necessary is just to find the dimensional vector J and give a rotation of θ about its axis.

If the number of model points is increased to approach an infinite number, the distance between vertices on the screen becomes less than one pixel. In this case, the fitting on the screen is equivalent to performing the fitting with the projected image of the strict model constituted by the curve.

Therefore, as the number of points constituting the model increases, the approximation accuracy increases. However, on the other hand, the amount of calculation for processing increases. The accuracy of the approximation will decrease, but the amount of calculation will decrease.

Therefore, an appropriate value will be adopted in consideration of these conditions. In the present embodiment, the optimum value is approximated by a 30-sided polygon. However, it should be noted that the present invention is not limited to this.

As described above, the model projection image generation unit 7
In order to hold a model having a three-dimensional position and orientation and to generate a projection image according to the camera parameters of the input system, if the model projection image generation unit 7 generates a contour projection image and a recognition target of the input image, If the contours of the image match, the actual position and orientation of the recognition target are the position and orientation of the model of the model projection image generation unit 7 (the model projection image generation unit 7 holds the model position in three dimensions. ), And 3
Dimensional measurement becomes possible.

Based on such a concept, the model projection image generation means 7 generates a projection image of the model of the recognition target object. Now, when the cylindrical shape described as an example is projected, it is assumed that it is a projection of a cylindrical object to be recognized, and the cylindrical object is recognized as a camera when viewed from the direction of its axis. FIG. 13 shows the positional relationship with the object.
It becomes like.

Here, the radius R of the circle in FIG. 13 is the radius R of the columnar shape. Position A of vertical lines on both sides in cylindrical image
If 'and B' are known, the focal point F and the direction of the optical axis of the camera are determined by Expressions (9) to (12).
The central angle 2θ can be determined correctly. Triangle ACF
And the triangle BCF are congruent, so that the distance d from the camera to the object can be obtained by equation (13).

[0105]

(Equation 4)

By using the above method, the depth position of the recognition target object can be calculated from the width of the left and right end points of the columnar shape. A method of calculating the three-dimensional position of the object when the position in the image and the depth position of the object are known will be described below.

Now, as shown in FIG. 14, a plane Q perpendicular to the camera optical axis vector A is located at a position away from the camera focal position vector F by d in the optical axis direction of the camera optical axis vector A.
Is assumed. The element of the camera focal position vector F is represented by F =
Assuming that (Fx, Fy, FZ) and the element of the camera optical axis vector A are A = (Ax, Ay, Az), the position vector H = (Hx, Hy, H
z) is represented by the following equation. (Hx, Hy, Hz) = (Fx + Axd, Fy + Ayd, FZ + AZd) (14) The equation of a plane Q passing through the point H and perpendicular to the optical axis vector A is given by the following equation.

(X−Hx) Ax + (Y−Hy) Ay + (Z−Hz) AZ = 0 (15) On the other hand, the center position of the recognition target object on the surface is C (xc, y
Suppose that the distance from the camera to the object to be recognized is d, given by c). At this time, the center position (X, Y, Z) of the recognition target object in the space is expressed by Expressions (1) and (2).
It is necessary to satisfy Expressions (14) and (15) in order to satisfy the projection equation and to be on the above-described plane Q. Therefore, combining these equations leads to the following simultaneous equations.

[0109]

(Equation 5)

By solving the simultaneous equations of the equation (16), the three-dimensional position (X, y) is set so that the distance from the camera to the object is d and the projection position on the surface is (xc, yc).
Y, Z) are required. Also, it will be necessary for later processing,
A method for correcting the "tilt on the screen" of the model will now be described.

[Method of Correcting "Inclination on Model" of Model] First, the inclination of the projected image of the model on the screen is calculated. This is, for example, the projection position of the center position of the upper end face of the recognition target model,
What is necessary is just to check the inclination on the screen of the line connecting the projection position of the lower end face center position.

Next, the difference between the inclination on the screen of the projection image of the current model and the inclination on the surface to be changed is calculated. Equation (1
If the recognition target model is rotated by this angle using the optical axis obtained in step 0) as the rotation axis, the inclination on the screen matches the angle of the change destination. In this way, the "tilt on the screen" of the model can be corrected.

As described above, the model projection image generating means 7 has a function of calculating and correcting the three-dimensional position of the model based on the position information on the screen as described above. Based on the above results, the model projection image generation is performed such that the position of the vertical contour line of the projection image of the model matches the end position of the cross slit extracted by the slit pattern extraction processing (step S22 in FIG. 2). The position parameter of the model of the means 7 is corrected (step S23 in FIG. 2). Thus, the end point position fitting process, which is the process in step S23 in FIG. 2, is completed. Then, when the end point position fitting processing is completed, the meridian extraction processing (step S2 in FIG. 2)
Enter 4). This is the following processing.

[Meridian Extraction Process (Step S24)] The meridian extraction process (step S24 in FIG. 2) will be described.
FIG. 15 shows a processing procedure of the meridian extraction processing. The “meridians” used in the present invention are vertical contour lines on both sides that appear when a cylindrical model to be recognized in this specific example is projected on a screen.

Since this contour line appears parallel to the axis of the columnar shape, the direction based on the meridian is hereinafter referred to as “axial direction”.
Described as "direction perpendicular to the axis." Now, since the approximate three-dimensional position of the recognition target object has been obtained by the processing up to this point, it is possible to know at what position the contour of the recognition target object is projected.

A meridian extraction area al as shown in FIG. 5 is set so as to include the estimated position. The size of this area al may be determined in advance, for example, as high as 200 pixels and width as 100 pixels, but may be set each time according to the size of the projected image of the recognition target model. Is also good. For example, the following is performed. On the basis of the length of the projected image of the meridian of the model generated by the model projection image generating means 7, the length is 1.0 times the width, and the length is 1.
The rectangle is set so that the projection position of the meridian is at the center with the height set to 5 times.

It is not known where the end point position obtained by extracting the slit pattern corresponds to the vertical position of the recognition target, and the recognition target may have an inclination of less than about 10 degrees on the screen.

However, as shown in FIG. 5, by setting a wide meridian extraction area so as to include the meridian, it is possible to recognize even if there is a deviation from the end point position or a deviation in the angle.

This area is set for the left and right meridians, and a meridian extraction process as shown in FIG. 15 is performed. First, an edge extraction process having directivity is performed on edges in the axial direction (vertical direction on the screen) by image processing on each pixel in the region.
Since the model projection image is generated first, the “direction” of the meridian of the recognition target object is also estimated at the same time. Therefore, an edge extraction process having directivity in the direction can be performed here. As a result, the value of the edge strength in the vertical direction is calculated for each surface element in the region (step S15 in FIG. 15).
1).

Next, one point is selected from each of the upper end and the lower end of the partial area from which the edge has been extracted, and one of the two selected points is set as a start point position (step S15 in FIG. 15).
2). The other is set as the end point position (step S153 in FIG. 15).

The start point position and the end point position are connected by a straight line,
The edge intensities of the pixels on the straight line are integrated (step S154 in FIG. 15). At the upper end and the lower end of the partial area, there are points for the number of constituent pixels at the end positions. Therefore, similar processing is performed for all possible combinations of the upper and lower points.

For analyzing the integrated value, first, assuming that the number of pixels at the upper end used for the integration is M and the number of surface primes at the lower end is N, M × M
N storage areas are secured. In this storage area, each combination connecting the positions of the upper end and the lower end of the partial area corresponds to one specific array element.
In the × N storage area, an operation of adding the value of the edge strength of the surface element on the straight line to the position of the array element corresponding to each straight line and placing the data is performed (step S1 in FIG. 15).
55).

That is, the value of the edge strength of the pixel passing through the straight line is voted at the position of the array element corresponding to each straight line. When the voting value is written in the M × N storage area in this manner, a histogram is obtained in a two-dimensional array. In the case of an edge surface element, the pixel at that surface element position has a high edge intensity, and therefore, it is highly possible that, of the voting results, a straight line corresponding to a pixel indicating a large vote value is an edge.

Therefore, in the M × N area, a predetermined number of two-dimensional local maximal positions are selected in descending order of the integrated value and are set as meridian candidates (step S1 in FIG. 15).
58). Then, the position of the meridian candidate is obtained by back calculation of the start point position and the end point position of the meridian (step S in FIG. 15).
159).

A similar process is performed on both the left and right meridians of the image, meridian candidates on both sides are obtained, and their positions are known. By selecting the meridian candidate having the largest integrated value from these meridian candidates, passing through the end point position obtained by the slit pattern extraction process and satisfying the fact that "the left and right meridians are parallel", The position is determined (step S1510 in FIG. 15).

In the line extraction method described in this embodiment, the line to be extracted is a digital straight line (when a straight line is expressed by an equation, the position of the line is defined in a real space.
In the case of a digital straight line, the position of the line is a position quantized to the pixel size.) And since it is a digital straight line, the process of selecting a line “passing through the end point position obtained by the slit pattern extraction process” is actually
"The distance between the extracted digital straight line and the calculated end point position is a predetermined surface prime number (for example, determined to be within 2 pixels)
Select a smaller line. "

[Meridian Fitting Process (Step S25)]
When the meridians of the model are obtained by the above-described maneuver, the three-dimensional position and orientation of the recognition target model are applied to the meridians using the method described in the end point position fitting process in step S23 in FIG. (Meridian fitting process; step S25).

By the processing so far, three change elements out of the five kinds of change elements of the appearance of the model are determined. The remaining parameters are "position of the center in the axial direction",
There are two types of “rotation angle in the depth direction” (for the five types of appearance change factors, see the explanatory diagram of FIG. 12).

[Characteristic Line Fitting Process (Step S2)
6)] Next, FIG. 2 showing processing by the characteristic line fitting means 6
The process proceeds to the feature line fitting process in step S26. FIG.
6 shows a processing procedure of the characteristic line fitting processing.

In this specific example, the insulator is described as an object to be recognized. In the case of the insulator, the characteristic lines are lines above and below the colored band. Since the painted band is painted around the body of the insulator, it is projected in an elliptical shape on the surface. By this feature line fitting process, a process of fitting upper and lower elliptical contour lines of the model projection image to an edge extraction result in the image and a matching process are performed. The fitting in step S26 is processing for determining two types of parameters left in the processing up to the previous step.

That is, the remaining two parameters (two parameters that have not been determined) are “the position of the columnar object in the direction of the central axis” and “the rotation angle in the depth direction”. The two types of parameters can be determined by generating various types of model projection images according to the types of parameters and performing the verification of the fit.

The specific contents of the processing are as follows. The recognition target model changes the model position and orientation according to the two parameters. The first is an axial translation of a cylindrical object. The axial direction of the object is obtained by obtaining a plane Q 'passing through the center of the object and perpendicular to the camera optical axis and projecting one axis of the object to the plane in the same manner as described in the end point position fitting process. Can be Second
Is the rotation in the depth direction, and this rotation axis is set on the plane Q ′ described above and passes through the center of the object and is set as an axis perpendicular to the translation direction.

By changing these two parameters two-dimensionally, the fit between the model projection image and the image feature is verified, and the optimum fit position and angle are obtained. The parameter change range and the number of divisions are set as follows. The parameter change range is the entire area where the projected image of the recognition target object can exist. In practice, it is often possible to assume the posture and position where the object exists within a certain range. For example, if the camera never shoots an object from above or below,
The existence region can be limited to a rotation angle of 180 degrees.

The camera is "looking up" at an object.
If it is possible to distinguish between "looking down" and "looking substantially from the side", it is sufficient to set the change area within a range of at most 90 degrees.

On the other hand, it is not necessary to change the position of the object in the axial direction to an area protruding from the screen, and the change range can be limited by the size of the screen. As mentioned above,
These two parameters can limit the range of change. The number of divisions of the parameter may be set so that the change in the position of the projected image in the image is less than one pixel. Finer changes in position and orientation are not related to the fit in the image. Therefore, the number of divisions can be limited to a finite number.

As described above, since the parameter change range and the number of divisions are determined, a model projection image is generated for all combinations of these two parameters, and fitting verification is performed. The fitting verification is performed as follows. A criterion for judgment is required for the fitting verification. Here, the following criterion is adopted.

Now, it is assumed that the pixel positions on the outer contour line of the model projection image are {Ci (cix, ciy)} i = 1, 2,..., M. Further, it is assumed that the edge intensity at the pixel position (cix, clx) is {E (cix, ciy)} i = 1, 2,. At this time, the fitting scale is defined by the following equation.

[0138]

(Equation 6)

That is, the values of the edge intensities are integrated as they are, and the higher the value is, the better the judgment is made. This integration of the fitting scale may be performed on the pixels of the entire outer contour line of the model transmitted from the model projection image generating means 7. However, in this step S26, only the elliptical contour line is used. Therefore, it is also possible to perform the integration of the fitting scale only on the elliptical outline.

Further, as shown in equation (17), the surface element for integrating the edge strength is determined not only by the pixel position of the outer contour line of the model projection image but also by a pixel within one pixel from the pixel position of the outer contour line. Neighboring region (that is, four pixels in an oblique position with respect to the four pixels at the top, bottom, left and right of the pixel,
The edge integration value of the pixel area may be integrated.

As described above, when the values of the neighboring areas are also integrated,
It is possible to deal with slight differences in the size and shape of the model held by the apparatus and the actual recognition target, and blurring of the image.
The above procedure will be described with reference to the flowchart of FIG. 16, which is a processing procedure of the characteristic line fitting processing. First, the range and the number of divisions of the two parameters to be subjected to the fitting are determined. When a projection image of the recognition target model is generated in this range, a region where a characteristic line (a band in this specific example) appears in the image is calculated. In this case, projection images are respectively formed by combinations of the maximum value and the minimum value of the parameter, and a large area that includes the projection images may be set (step S1).
61).

Next, edge extraction processing is performed on the inside of the area. In the edge extraction in this case, unlike step S151 of the meridian extraction process described with reference to FIG. 15, an edge extraction process having no directionality is performed. For example, a Laplacian-Gaussian filter may be used (step 162).

Next, one of the two parameters to be changed, “the rotation angle in the depth direction” (tilt angle), is set as one value in the change area (step S163). Further, the other parameter "position in the direction of the central axis of the cylindrical object" (position in the axial direction) is set (step S164).

A contour projection image of the recognition target model is generated based on the set parameters (step S165). Based on the equation (17), the edge strength on the characteristic line (of the contour line of the elliptical portion in the present embodiment) is integrated (step S1).
66). This fitting measure is stored in a two-dimensional array in the same manner as in the meridian extraction processing (step S16).
7).

When the calculation of the fitting scale for the entire change range of the two parameters is completed, the position of the two-dimensional local maximum is selected from the integrated value of the edge strength stored two-dimensionally (step S1610). .

Then, the tilt angle of the local maximum value and the position in the axial direction are calculated backward, and this is set as the tilt angle and the axial position best applied (step S1611). With the above procedure, the three-dimensional position and orientation of the recognition target model have been obtained, and the processing in step S26 in FIG. 2 ends.

[Output Processing (Step S27)] As described above, when the selection of the optimum position and orientation by the characteristic line fitting processing of step S26 in FIG. 2 is completed,
Next, by performing the process of step S27, which is an output process, the result output unit 8 outputs the final recognition result,
The whole process ends.

The details of the present invention have been described above. In short, the present invention compares the image characteristics of the image obtained from the image pickup means for picking up the recognition target with the projection image of the prepared three-dimensional model of the recognition target. The degree of fit is verified, and the projected image obtained by sequentially correcting the position and orientation of the model according to the verification result is again compared with the image feature to verify the degree of fit, and the best fit projection is obtained. The three-dimensional position and orientation of the recognition target object are recognized by obtaining information on the position and posture of the recognition target from the projected image that best fits the obtained image.

More specifically, the image feature of the object to be recognized is extracted from an image of the object to be recognized having a specific shape by slit pattern extracting means, meridian extracting means, characteristic line fitting means, and the like. The image generating means calculates a projection image of the three-dimensional model using the held three-dimensional model of the recognition target object having the specific shape,
The three-dimensional position and orientation of the three-dimensional model are corrected based on the degree of fit determined by the fit determining means to obtain a projection image for comparison and collation with the recognition target object.

Further, for a contour line that is difficult to extract, a slit light is irradiated, and the position of the contour line that intersects the slit pattern is extracted from the shape of the slit pattern. Is extracted.

The model projection image generation means corrects the position and orientation of the three-dimensional model according to the extraction result of the slit pattern extraction means, and corrects the position and orientation of the three-dimensional model according to the extraction result of the meridian extraction means. The projection images of the various positions and postures of the three-dimensional model are transmitted to the feature line fitting means, and information on the position and posture of the three-dimensional model determined to have the best fit is obtained and output.

As described above, the image information obtained by capturing the recognition target is obtained, the image features of the image obtained from the image information are extracted, and the extracted image features are projected onto the projection of the prepared three-dimensional model of the recognition target. In comparison with the image, the degree of the fit is verified, the position and orientation of the model are successively corrected, the best fit projection image is obtained, and the position and orientation are obtained from the best fit projection image obtained. Thus, the three-dimensional position and orientation of the recognition target object are recognized.

Therefore, the position and orientation of the object to be recognized having the “cylindrical” shape or the “generalized cylindrical” shape can be measured from the image, opening the way to the realization of an autonomous work robot.

It should be noted that the present invention is not limited to the specific examples described above, but can be implemented with various modifications. In addition, the method described in the embodiments of the present invention can be executed on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), a semiconductor memory, or the like, as a program that can be executed by a computer. It can also be stored and distributed.

[0155]

As described above, according to the present invention,
Specific shapes, such as "cylinder" shapes and "generalized cylinders"
It is possible to recognize or measure the three-dimensional position and orientation of an object from the image of the object without recognizing the shape of the object without any marker. Its practical effects are enormous, such as being able to carry on and enabling the realization of autonomous robots for operations such as power distribution facilities.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining the present invention, and is a block diagram showing an entire configuration of a specific example of the present invention.

FIG. 2 is a diagram for explaining the present invention, and is a flowchart showing an operation example of the system of the present invention.

FIG. 3 is a diagram for explaining a transition of processing in the system of the present invention, and is a schematic diagram showing a state of a captured image without a slit.

FIG. 4 is a diagram for explaining a transition of processing in the system of the present invention, and is a schematic diagram showing a state of a captured slit irradiation image.

FIG. 5 is a diagram for explaining a transition of processing in the system of the present invention, and is a schematic diagram in a state where a meridian extraction area set for meridian extraction processing is added.

FIG. 6 is a diagram for explaining a transition of processing in the system of the present invention, and is a schematic diagram including a meridian extracted by meridian extraction processing.

FIG. 7 is a diagram for explaining a transition of processing in the system of the present invention, and is a schematic diagram of a final fitting result.

FIG. 8 is a diagram for explaining the present invention, and is a flowchart showing a processing procedure of laser pattern extraction processing.

FIG. 9 is a diagram for explaining the present invention, and is a configuration diagram of a model projection image generating unit in the system of the present invention.

FIG. 10 is a diagram for explaining the present invention, showing an example of a model in the system of the present invention.

FIG. 11 is a diagram for explaining the present invention, showing an example of a generated model projection image.

FIG. 12 is a diagram for explaining the present invention, and is a diagram for explaining five types of changing factors of how a cylindrical model looks.

FIG. 13 is a diagram for explaining the present invention, and is an explanatory diagram of a method for calculating the depth of a cylindrical model.

FIG. 14 is a diagram for explaining the present invention, and is a diagram for explaining conversion from a two-dimensional position to a three-dimensional position.

FIG. 15 is a diagram for explaining the present invention, and for explaining a procedure of meridian extraction processing in the system of the present invention.

FIG. 16 is a diagram for explaining the present invention, and is a diagram for explaining a procedure of a characteristic line fitting process in the system of the present invention.

FIG. 17 is a diagram for explaining the present invention, and is a schematic diagram for explaining label density calculation in the system of the present invention.

FIG. 18 is a diagram for explaining the present invention, and is an explanatory diagram of a method of performing label density calculation using a convex polygon circumscribing a label.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... TV camera 2 ... Slit light irradiation means 3 ... Image input part 4 ... Slit pattern extraction means 5 ... Meridian extraction means 6 ... Feature line fitting means 7 ... Model projection image generation part 8 ... Result output means 91 ... Model position calculation means 92: Model position and orientation parameter storage means 93: Model structure parameter storage means 94: Model three-dimensional position calculation means 95: Camera parameter storage means

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Kyoichi Okamoto, Inventor 1-1-1 Oyodonaka, Kita-ku, Osaka-shi, Osaka

Claims (5)

[Claims] 1. A slit light irradiating means for irradiating slit light to a specific shape recognition target object, an image input means for inputting an image of the specific shape recognition target object, and an image from an image obtained from the image input means A slit pattern extracting means for extracting a projection pattern of the slit light by processing; an image obtained from the image input means; and information obtained from the slit pattern extracting means, the contour of the object to be recognized intersecting the projection pattern of the slit light. Holding a meridian extracting means for extracting a line, and a three-dimensional model of the recognition target object having the specific shape;
A model that calculates a projection image of the three-dimensional model and corrects a three-dimensional position and orientation of the three-dimensional model based on information indicating the degree of fitting to obtain a projection image for comparison and collation with the recognition target object. Projection image generation means, From the image obtained from the image input means, the image features of the recognition target object, the projected image obtained by the model projection image generation means is compared and compared to determine the degree of fitting, A feature line fitting means for correcting the position and orientation of the recognition target model to fit a feature line generated from the shape of the recognition target object; and a position and a best three-dimensional model projection image of the fitting degree.
And a result output unit that obtains and outputs the posture information from the model projection image generation unit and outputs the position information. 2. An image input means for inputting an image of an object to be recognized having a cylindrical shape or a shape formed by stacking cylinders or truncated cones in the axial direction thereof; Slit light irradiating means for irradiating slit light so as to intersect with a contour extending in the direction along, a slit pattern extracting means for extracting a projection pattern of slit light by image processing from an image obtained from the image input means, Meridian extraction means for extracting a contour line of the recognition target object intersecting a projection pattern of slit light from an image obtained from the image input means and information obtained from the slit pattern extraction means; and a recognition target object having the specific shape. And a projected image of the three-dimensional model is calculated.
Based on the information indicating the degree of fit, the 3D model 3
A model projection image generating unit that obtains a projection image for correcting a three-dimensional position and orientation to compare and match with the recognition target object, an image feature of the recognition target object from an image obtained from the image input unit, and the model A feature line fitting unit that determines the degree of fitting by comparing and comparing the projection images obtained by the projection image generating unit, corrects the recognition target model position and orientation, and fits a feature line resulting from the shape of the recognition target object, The position of the best three-dimensional model projection image of the fitting degree
And a result output unit that obtains and outputs the posture information from the model projection image generation unit and outputs the position information. 3. The apparatus for recognizing the position and orientation of a recognition target object according to claim 1, wherein said slit light irradiating means is configured to irradiate slit light having a shape constituted by at least two intersecting lines. The slit pattern extracting unit divides a surface element that is a candidate for a pixel corresponding to the projection pattern of the slit light as a connected region, and a ratio of the number of pixels included in the connected region to an area of a shape circumscribing each connected region is determined. A recognition target characterized in that a projection pattern of slit light is extracted by selecting a connection area having a maximum circumscribed shape area from connection areas having values smaller than a value for determination. Object position and orientation recognition device. 4. The recognition target object position / posture recognition apparatus according to claim 3, wherein a shape circumscribing the connection region in the slit pattern extracting means is a rectangle circumscribing the connection region. Position and orientation recognition device. 5. A slit light irradiation pattern is extracted from image information of an image of a recognition target formed by irradiating slit light and information of the image of the recognition target not irradiated with slit light. The resulting feature points or feature lines, extract the information of the position intersecting the slit light irradiation pattern, correct the position and orientation of the recognition target prepared based on the information, and extract the image features extracted from the input image information. The degree of fitting is verified by comparing with the projected image of the three-dimensional model to be recognized, and the position and orientation of the model are sequentially corrected to obtain the best fitting projected image. From the obtained best fitting projected image, A method for recognizing a position and orientation of a recognition target object, characterized by recognizing a three-dimensional position and posture of the recognition target object by obtaining the position and posture.