JP2002296010A - Own position identifying method using image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the accuracy of a technique for determining an object or identifying an own position by comparing a camera-inputted image with a template image. SOLUTION: An image is inputted from an imaging means such as a camera, and the photographed image is compared with a teaching image (template image) photographed with a wider visual field while the resolution of images is varied. This is repeated to search for the object of teaching related to the teaching image (template image). If the object has absolute positional information in a room, the own position of the imaging means can be determined by determining the position of the object from at least two photographed images. Based on this, a movable body can be navigated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology for finding a line of sight of a camera or a specific target object from a surrounding environment image.

[0002]

2. Description of the Related Art In a navigation technique of a robot, it is essential to specify a target object from a camera image which is also an eye of the robot and to identify a position of the robot.

[0003] For this purpose, an image serving as a template is prepared in advance in the robot itself, and the template image is compared with an actual camera image of the robot to grasp an object or identify its own position. Had been

At this time, in order to reliably grasp the target, a template image with high image accuracy (high resolution) is prepared, and an operation for comparing the template image with a camera image is performed.

However, when such a high-resolution template image is used, the amount of calculation becomes large for comparison with the camera image, and an error occurs when a plurality of objects similar to the template image exist in the camera image. There was a possibility that detection would be performed. Further, in order to determine whether or not the template image is included in the camera image at that time, it is necessary to specify information on the line of sight of the camera at that time.

[0006] On the other hand, the camera image does not always exactly match the template image, and in many cases, information that is the basis of object search or self-position identification cannot be specified.

For example, if the actual camera image is closer to the object than the photographing position of the template image, the object of the camera image will be photographed larger than that of the template image.

As described above, when the camera position at the time of template image acquisition and the camera position at the time of actual camera image input are slightly deviated, a certain portion of the image is elongated and a certain portion is contracted. This makes it difficult to confirm the similarity as it is.

[0009] Such a problem is not limited to the robot technology, and is recognized as a similar problem in the field of car navigation and the like.

[0010]

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and has been made in consideration of the accuracy of a technique for identifying an object and identifying its own position by comparing a camera input image with a template image. It is an object of the present invention to provide a technique capable of reliably controlling a target object and navigating a moving object.

[0011]

According to the present invention, an image is inputted from an image pickup means such as a camera, and the captured image is compared with a teaching image (template image) taken in a wider field of view than the image. Change while comparing. By repeating this, the teaching target (object) associated with the teaching image (template image) is searched.
If the teaching object has absolute position information in the room, the position of the teaching object is specified at least two times.
The self-position of the imaging unit can be specified by performing the above-described captured image, and the navigation of the moving object can be performed based on this.

[0012]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is an explanatory diagram showing the concept of a process according to an embodiment of the present invention, and FIG. 2 is a block diagram showing a process of the process.

The processing of this embodiment is realized by a general-purpose computer system. The functions described below are executed by a central processing unit (CPU) of the computer system reading and executing a program provided to the computer system. Will be

The computer system includes a RAM,
It has a storage device such as a ROM or a hard disk device (HD), and the program is installed in these storage media. In these storage media, the teaching image shown in FIG. 1 and the teaching target part image are stored. Here, the teaching image is an image obtained by photographing the room in advance, and in FIG. 1, it is an image in which a gas range unit, a sink unit, and a refrigerator of a system kitchen (kitchen) are arranged. As the teaching target component image, an image of the switch, grill, and ignition port of the gas range unit is prepared. These teaching target part images are
It has absolute position information in the room in association with the teaching image, and by acquiring the positions of these teaching target part images, the camera position in the room can be calculated. ing.

A camera (TVC) is attached to a mobile object that can move in a room, for example, a robot, and arbitrarily shoots images of the room sequentially. The shot images are taken by a central processing unit (CPU) of the computer system. The digital image is stored in the storage medium as a camera image together with the teaching image and the teaching target part image (S
1).

The central processing unit (CPU) reads the camera image and the teaching image stored in the hard disk drive (HD) by the image processing program and performs low resolution processing (S2). In the low-resolution processing, for example, the resolution of the teaching image can be reduced to compare the similarity with the camera image with a small amount of data (S3).

FIG. 3 shows an example of the determination method at this time. That is, the order in which the luminance changes appear on a line crossing the center of the low-resolution image, that is, the luminance change sequence,
Each teaching image and the camera input image are used as a character sequence. The comparison of the brightness change sequence is performed by (1) matching the number of elements of the sequence by interpolation, and (2) quantifying the degree of matching by the maximum value of the normalized correlation value shown in the following equation.

(Equation 1) Here, s represents an element of the sequence of brightness changes of the input image, and t represents an element of the sequence of brightness changes of the template image.

This will be specifically described with reference to FIG. First, from the teaching image in the kitchen room as shown in FIG.
A low-resolution full-circle panoramic image (for example, about 24 × 138 pixels) is acquired, luminance characteristics are calculated from this image, and an intersection point with an arbitrarily selected a0-a3 equal dividing line is acquired as a luminance sequence. This sequence is represented by a1, a2, a
3, a3, a2, a1, a0, a0, a0, a0,.
Becomes

On the other hand, a low-resolution image corresponding to the teaching image is also obtained from the camera image, a brightness characteristic from the camera image is calculated, and a brightness sequence at the intersection with the same dividing line as in the teaching image processing is obtained. I do. By the way, when there is a difference between the camera position at the time of acquiring the teaching image and the camera position at the time of inputting the actual camera image, a difference appears as an image. This will be described with reference to FIG.

For example, when an image obtained by photographing C at point A in the figure is a teaching image and an actual camera is mounted on the robot and photographing C from point B, Since the camera position is different between the screen of C and the screen of C on the camera image, a difference appears in the image. That is, the image B (the teaching image) photographed at the point A is smaller than the image B (the teaching image).
The image of C (camera image) photographed at the point is a magnified image. On the other hand, when an all-around panoramic image is obtained at point B, a position 180 degrees opposite to C (D in the figure)
The camera image at the point ()) appears as an image reduced from the teaching image.

Therefore, since the actual camera image is a partially enlarged / reduced image than the teaching image, no coincidence point can be found even when the teaching image and the camera image are compared in pixel units. . As described above, when a low-resolution full-circle panoramic image is acquired as a television image, a partial enlargement or reduction appears in a change in the luminance value due to a shift in the acquisition position between the teaching image and the panoramic image (television image). become.

However, even if such a change in the brightness value is expanded or contracted, the brightness change sequence itself is not affected. Therefore, it is easy to find a matching point between the teaching image and the television image, and it is possible to detect a matching point between the television image in the teaching image.

In the detection of the coincidence point of the television image in the teaching image, a coincidence point with the television image is searched while gradually increasing the resolution by using the low-resolution teaching image as an initial search target screen. At this time, at the stage of confirming the similarity between the teaching image and the camera image, the position (the position of the target component) may be searched while searching for the teaching target component image position in the camera image (S4).

If the similarity with the teaching image can be predicted from the low-resolution image, the resolution of the teaching image and the camera image is increased by a predetermined amount, and the processing of S3 to S5 is repeated again. In this way, by repeatedly comparing the camera image with the teaching image (taught target component image) while sequentially changing the resolution of the image from a low-resolution image to a high-resolution image, the teaching image (target component) in the camera image can be compared with the teaching image (target component). The similarity prediction accuracy is increased, and the position can be specified.

At this time, the camera position in the room can be specified from the positional relationship between the detected two or more teaching target components (S7). Further, the camera position can be specified from the position of the teaching target component obtained from the first camera position and the position of the same teaching target component obtained from the second camera position.

The first camera position and the second camera position may be the positions before and after the camera has been moved by the moving body, or may be the positions where two cameras are mounted on the moving body. Alternatively, the positions of the cameras that have been photographed at the same time may be used. By specifying the camera position in the room in this way, it is possible to control the moving object in the room (S8).

Note that, in the example shown in FIG.
For 0 to a3, for example, the range between the maximum value and the minimum value of the luminance is equally divided into five and four threshold values are determined, but the present invention is not limited to this. In addition, the intervals between the thresholds need not be equal.

As described above, according to the present embodiment, it is possible to extract a stable image characteristic which is not influenced by a slight enlargement / reduction of a portion caused by a deviation of the camera position of the input camera image from the teaching image. Using this as an index, it becomes easy to detect the similarity between the input camera image and the teaching image.

It should be noted that the similarity between the teaching image and the camera input image may be compared using the sum of similarities between the teaching image portion at a position corresponding to a predetermined portion of the camera input image. Good. In this case, processing such as enlargement / same magnification / reduction is performed on the fragment image obtained by dividing the camera input image, and the best value of the correlation with the fragment image at a position substantially corresponding to the teaching image is calculated. Then, using the sum of the correlation values calculated for all the fragment images as an index, it is possible to grasp the similarity between the input camera image and the teaching image. More specifically, a fragment image obtained by dividing a camera input image into strips in the horizontal direction is created, and an image obtained by performing enlargement / same magnification / reduction processing on each fragment image is temporarily stored in a storage medium. .

Next, the teaching image is read out, the correlation between the teaching image and the fragment image at a position corresponding to the teaching image is calculated for the fragment image of each size, and the correlation value when the correlation is the best is recorded. I do. This process is repeated for all the fragment images, and the similarity between the input camera image and the teaching image is obtained using the best sum of the correlation values as an index.

Next, the direction of the camera position at the time of acquiring the camera input image as viewed from the teaching image can be determined based on the slight positional shift between the teaching image held in the storage medium and the camera input image. Specifically, as shown in FIG. 4, when the camera input image acquisition position and the camera position at the time of acquiring the teaching image are shifted, when the two images are superimposed, the camera input image is compared with the teaching image at a certain place. The minute area of the image is shifted to the right, and in some places in the opposite direction. Based on the direction in which this deviation is maximum and minimum,
It is possible to grasp in which direction the camera position at the time of acquiring the camera input image is different from the camera position at the time of acquiring the teaching image.

Further, the displacement of the camera in the room can be grasped based on the change in the size of the image between the teaching image and the camera input image. That is, as shown in FIG. 4, when the camera input image acquisition position and the camera position at the time of acquiring the teaching image are shifted, when the two images are superimposed, the camera input image is slightly smaller than the teaching image at a certain place. The area is expanding and in some places shrinking. When the room is circular and the image acquisition position slightly moves near the center of the room, the image in the moving direction (approaching direction) tends to be the largest, and the image in the opposite direction (separating direction) tends to be the smallest. . Therefore, based on the direction in which the degree of enlargement / reduction of the minute area image is maximized, it is roughly estimated in which direction the camera position at the time of acquiring the camera input image is deviated from the camera position at the time of acquiring the teaching image. can do.

As described above, by associating the teaching image with the camera position at the time of acquiring the teaching image, the direction of the camera in the room can be determined based on the positional shift and the difference in size between the camera input image and the teaching image. And the position can be specified.

As described above, the position of the camera mounted on the moving body can be determined based on the similarity between the teaching image and the camera input image. Such teaching images are acquired and accumulated intermittently on a storage medium at a predetermined spatial distance, and by successively repeating similarity checking processing between these teaching images and the camera input image, The self-position can be navigated along the movement path of the camera that has acquired the teaching image.

FIG. 5 shows a group of teaching images for navigation taken at 45 cm intervals. As shown in the figure, by continuously reading the teaching image and continuously checking the similarity with the camera input image, it is assured that the moving object equipped with the camera is moving on the correct route, and the moving object is You can navigate along the exact route.

Further, the self-moving direction from the angle of the latest camera input image with respect to the most suitable teaching image can be grasped, and the posture with respect to the teaching image can be corrected.
In this case, the teaching image does not necessarily need to be a full-circle panoramic image, and it is sufficient that a range around the traveling direction is secured.

Further, in order to strictly determine the arrival at the target point, an image obtained by enlarging / reducing a central portion of the camera input image is compared with the last teaching image, and a camera input image which matches an image which is not enlarged or reduced is obtained. May be determined as the final arrival position.

Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, but includes the following supplementary notes. That is,

(Supplementary Note 1) An image is input from the imaging means,
The input image is compared with a teaching image captured in a wider field of view than the input captured image, and the comparison is performed by repeating the comparison between the images while changing the resolution of the captured image or the teaching image. An object search method for searching for a teaching object associated with an image.

(Supplementary Note 2) An image is input from the imaging means,
Compare the input image with the teaching image captured in a wider field of view than the input captured image, the comparison is performed by repeatedly comparing the similarity between the images while changing the resolution of the captured image or the teaching image By searching for a teaching target associated with the teaching image, specifying the position of the teaching target in the captured image, and specifying the position of the teaching target from at least two or more captured images. A self-position identification method for identifying a self-position of an imaging unit.

(Supplementary note 3) The self-position identification method according to supplementary note 2, wherein the imaging means is provided on a movable body that is movable in a predetermined environment.

(Supplementary Note 4) The self-position identification method according to Supplementary Note 2, wherein the comparison of the similarity between the images is performed by comparing the respective sequences of luminance changes in the horizontal direction between the captured image and the teaching image.

(Supplementary Note 5) The self-position identification method according to Supplementary Note 2, wherein the comparison of the similarity between the images is performed by summing the similarity between the partial image of the captured image and the partial image of the teaching image.

(Supplementary Note 6) A supplementary note that specifies the camera position at the time of acquiring the photographed image as viewed from the camera position at the time of acquiring the teach image, based on a minute change in the position of the photographed image from the teaching image at a predetermined portion. 3. The self-position identification method according to 1 or 2.

(Supplementary Note 7) The camera position at the time of acquiring the photographed image as viewed from the camera position at the time of acquiring the teach image, based on a change in the size of a minute target image with respect to the teach image at a predetermined portion of the photographed image. 3. The self-position identification method according to claim 1 or 2, wherein

(Supplementary Note 8) In addition to the above, teaching images are intermittently accumulated at predetermined spatial intervals, and similarity detection between the teaching images and the photographed images is sequentially repeated for each of the teaching images. A navigation method using the self-position identification method according to Supplementary note 1 or 2, wherein the self-position is navigated along the movement path of the camera position from which the teaching image was acquired.

(Supplementary Note 9) An image is input from the imaging means,
The input image is compared with a teaching image captured in a wider field of view than the input captured image, and the comparison is performed by repeating the comparison between the images while changing the resolution of the captured image or the teaching image. A computer-executable program for searching for a teaching object associated with an image.

(Supplementary Note 10) An image is input from an imaging means, and the input captured image is compared with a teaching image captured in a wider field of view than the input captured image. Performed by repeatedly comparing the similarity between the images while changing, searching for the teaching target associated with the teaching image, specifying the position of the teaching target in the captured image, A computer-executable program for identifying a position of an imaging unit by identifying a position from at least two or more captured images.

According to the present invention, the accuracy of the technique for identifying an object and identifying its own position by comparing a camera input image with a template image is improved, and further, the position of the camera is easily grasped, and navigation is performed. Can be performed reliably.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram showing an embodiment.

FIG. 2 is a block diagram illustrating processing according to the embodiment;

FIG. 3 is an explanatory diagram showing a technique of determining image similarity by comparing a luminance change number sequence according to the embodiment;

FIG. 4 is a diagram for explaining a relationship between a camera position and an image shift (enlargement / reduction) according to the embodiment.

FIG. 5 is an example of an intermittent teaching screen group for navigation in the embodiment.

[Explanation of symbols]

 HD hard disk drive

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 H04N 7/18 Z 5L096 F term (Reference) 2F065 AA00 AA03 BB05 DD03 FF04 JJ03 JJ26 MM21 PP01 QQ13 QQ24 QQ29 QQ38 QQ42 RR02 RR03 3C007 KT11 MT04 5B057 AA05 AA16 DA08 DA12 DB02 DB09 DC34 5C054 AA01 FC04 FC14 FF01 HA28 5H301 AA02 AA10 BB14 CC03 CC06 DD02 GG09 5L096 AA06 BA05 CA03 EA03 GA03 JA03 EA03 GA03

Claims (5)

[Claims] 1. An image is inputted from an image pickup means, and the input image is compared with a teaching image photographed in a wider field of view than the input photographed image. The comparison changes the resolution of the photographed image or the teaching image. By repeatedly comparing the similarity between the images while searching, searching for the teaching target associated with the teaching image, specifying the position of the teaching target in the captured image, A self-position identification method for identifying a self-position of an imaging unit by performing identification from at least two or more captured images. 2. The self-position identification method according to claim 1, wherein the comparison of the similarity between the images is performed by comparing respective sequences of luminance change numbers in the horizontal direction between the captured image and the teaching image. 3. The self-position identification method according to claim 1, wherein the comparison of the similarity between the images is performed based on the sum of the similarities between the partial image of the captured image and the partial image of the teaching image. 4. A camera position at the time of acquiring the photographed image as viewed from the camera position at the time of acquiring the teach image, based on a minute change in the position of the photographed image with respect to the teach image at a predetermined portion. Self-locating method as described. 5. A camera position at the time of acquisition of the photographed image as viewed from the camera position at the time of acquisition of the teach image, based on a change in the size of a minute target image with respect to the teach image at a predetermined portion of the photographed image. The self-position identification method according to claim 1.