JPH1083446A - Obstacle detection device/method and moving device/ method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely detect an obstacle with simple constitution by detecting the obstacle based on pictures when a light source is put out, when it is lighted and when it is put out, which are sequentially obtained. SOLUTION: An arithmetic unit 5 stores luminance picture data Ya-Yc corresponding to the pictures when LED 6 is put out, when it is lighted and when it is put out, which are sequentially obtained from a CCD camera 4, color difference picture data Ra-Rc and Ba-Bc in a main storage circuit. The arithmetic unit 5 detects picture elements whose absolute value of the difference of luminance levels is not more than a threshold and whose absolute value of the difference of color levels is not more than the threshold by using luminance picture data Ya and Yc and color difference picture data Ra and Rc. Then, the presence or absence of the reflected light of LED 6 in the respective picture elements is detected by using color difference picture data Rb and judges it to be a part of the obstacle when the picture element received reflected light.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[Table of Contents] The present invention will be described in the following order. Technical Field to which the Invention pertains Related Art Problems to be Solved by the Invention Means for Solving the Problems Embodiments of the Invention (FIGS. 1 to 5) Effects of the Invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an obstacle detecting device, an obstacle detecting method, a moving device, and a moving method. For example, a moving device such as a robot device detects an obstacle to avoid collision with the obstacle. And an obstacle detection method, an obstacle detection method, a moving device, and a moving method.

[0003]

2. Description of the Related Art Conventionally, as this kind of obstacle detection device,
In some cases, a distance to an obstacle is measured by projecting an ultrasonic wave to an obstacle and measuring a time until a reflected wave from the obstacle is detected by an ultrasonic detection unit. In addition, as this type of obstacle detection device, the size of the shadow of the object generated by irradiating the object, the size of the image of the object photographed by the photographing object, the tip of the image of the object Calculating the distance between the image and the tip of the shadow image and the angle between the shadow and the image of the object, and calculating the horizontal distance between the object and the object from at least two of these calculation results By
A device for measuring the distance to an obstacle has been proposed (Japanese Patent Application No. 1-245264 (JP-A-3-109612)).

As an obstacle detection device of this type, a spot of an object to be measured projected from a laser beam
There has been proposed an apparatus that calculates the position of an obstacle by continuously performing processing for obtaining the position of a spot on a measured object by photographing with a CD (Charge Coupled Device) camera (that is, obtaining the difference between images). (Japanese Patent Application No. Hei 4-291150 (Japanese Patent Application Laid-Open No. Hei 6-137828)). Further, as this type of obstacle detection device, there has been proposed an obstacle detection device which detects a position of an obstacle by detecting a difference between patterns by a laser in a case where an obstacle is present and a case in which no obstacle is present (Japanese Patent Application No. 2002-110630). 6-42
No. 436 (JP-A-7-253319).

[0005]

However, in the method using the reflected wave of the ultrasonic wave, the reflectivity of the ultrasonic wave is different depending on the material of the obstacle and the surface shape of the obstacle, so that the obstacle can be accurately detected. When such an obstacle detection device is applied to a mobile device such as a robot device, there is a risk that the mobile device collides with an obstacle. In addition, since information such as the shape and color of an obstacle cannot be obtained, for example, when playing a game with a robot device or the like, when it is necessary to identify an opponent by color or shape, the present invention should be applied to this type of robot device. Could not be done.

In the method of using a shadow of an object by irradiating the object with light, an obstacle (for example, a wall) where the shadow cannot be observed
Could not be detected, and it was still insufficient for practical use as an obstacle detection device. In the method of photographing the spot of the measured object with two CCD cameras and obtaining the position of the spot of the measured object, two CCD cameras are required.
If the obstacle detection device becomes large and this main obstacle detection device is applied to the above-mentioned robot device, there is a possibility that the robot device becomes large and the movement becomes slow. Further, in a method using a pattern difference by a laser, there is a possibility that a positional relationship with an obstacle cannot be accurately detected.

The present invention has been made in consideration of the above points, and has an obstacle detecting apparatus, an obstacle detecting method, and a moving apparatus to which an obstacle detecting apparatus can be accurately detected with a simple configuration. An object and a moving method to which the obstacle detecting method is applied are proposed.

[0008]

According to the present invention, a first image obtained when the light source is turned off and a second image obtained when the light source is turned on are sequentially obtained from the image pickup means.
By comparing the first image and the third image among the image of the third image and the third image when the light source is turned off, the change in the luminance level is equal to or less than the first luminance threshold value, and / or Detecting an area in which the change in color level of the color is equal to or less than the first color threshold value, and based on the second image, in the area, the change in luminance level exceeds the second luminance threshold value; And / or detecting a region where the color level of the predetermined color exceeds the second color threshold as an obstacle, calculating the area of the detected region, and detecting the positional relationship with the obstacle based on the calculation result. The driving means of the moving device is controlled based on the positional relationship with the obstacle. If there are obstacles,
The obstacle is detected as an area where the change in the luminance level and / or the color level is equal to or less than the first luminance threshold and / or the first color threshold, respectively, in comparison with the first image and the third image. Then, of the detected areas, the change in the luminance level and / or the color level is reflected as an area exceeding the second luminance threshold and / or the second color threshold, respectively, based on the second image. You. Further, since the positional relationship with the obstacle is reflected in the number of pixels receiving the reflected light of the light source light, the positional relationship with the obstacle is represented by a change in the luminance level and / or the color level, respectively, of the first luminance threshold. And / or the area of a region which is equal to or less than the second color threshold value and exceeds the second luminance threshold value and / or the second color threshold value.

In the present invention, the integrated data for each row or each column is generated by integrating the brightness data of each pixel in the image obtained by the image pickup means for each row or each column, and based on the respective integrated data. When the image is divided into two parts, the row or column in which the difference in pixel average in each area of the divided image is the largest is detected as the boundary with the obstacle, and the positional relationship with the boundary is determined based on the detection result. It detects and controls the driving means of the moving device based on the positional relationship with the boundary. If there is an obstacle, the boundary with the obstacle is defined as the row or column where the difference in pixel average in each region of the divided image is the largest when the image is divided into two based on the integration data for each row or column. Is detected as

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

In FIG. 1, reference numeral 1 denotes a bogie type moving device to which the obstacle detecting device of the present invention is applied as a whole. In this moving device 1, a bogie 3 is driven under the control of an obstacle detecting device 2. As a result, it is possible to prevent the vehicle from moving, thereby preventing collision with an obstacle or falling from a table.

The obstacle detection device 2 takes in a video image picked up by a CCD camera 4 provided on a carriage 3 as an image pickup means into a computing device 5 as an image pickup signal S1. The arithmetic unit 5
A traveling direction control signal S2 for controlling the traveling direction of the truck 3 is transmitted to a motor 3B for driving wheels 3A of the truck 3 based on the imaging signal S1. The carriage 3 controls the direction of each wheel 3A by a motor 3B in a direction corresponding to the traveling direction control signal S2, thereby controlling the traveling direction of the moving device 1 in a direction corresponding to the traveling direction control signal S2.

Further, the arithmetic unit 5 transmits a photographing direction control signal S3 for controlling the photographing direction in the vertical direction of the CCD camera 4 to the tilter 5 provided in the CCD camera 4 as photographing direction control means, based on the photographing signal S1. Send out. The tilter 5 determines a shooting direction of the CCD camera 4 by a shooting direction control signal S.
Control is performed in a direction corresponding to 3. The CCD camera 4 includes an LED (Light Emitti) provided on the CCD camera 4 as a light source.
An image of the subject in the direction illuminated by the red light emitted from the ng diode 6 is taken. The LED 6 emits red light intermittently based on a control signal S4 from the arithmetic unit 4.

Here, in practice, as shown in FIG. 2, the arithmetic unit 5 receives the picked-up image signal S1 supplied from the CCD camera 4 in the Y / C separation circuit 7, and in the Y / C separation circuit 7, A luminance signal Y, a color difference signal Y-R (hereinafter, this is referred to as a color difference signal R), and a color difference signal BY (hereinafter, this is referred to as a color difference signal B) corresponding to an image of one frame from the imaging signal S1. Then, the luminance signal Y is converted to an analog digital (A /
D) Sending out as luminance data DY via the conversion circuit 8 and sending out the color difference signals R and B as color difference data DR and color difference data DB via the A / D conversion circuit 9.

The luminance data DY is stored in a memory 11 as a storage means based on a control signal S5 sent from a microcomputer 10 (hereinafter referred to as a microcomputer). It is stored in a storage area of the memory 12 as storage means based on the transmitted control signal S5.

The microcomputer 10 sends the address signal S6 to the memories 11 and 12 so that the memory 1
After the bus between the microcomputers 1 and 12 and the microcomputer 10 is released, the control signal S5 is sent to the memories 11 and 12 so that the luminance data D
Y and the color difference data DR and DB are read.

The microcomputer 10 assigns the label "a", the label "b" or the label "c" to the luminance data DY and the color difference data DR and DB, respectively, and then stores the address signal S6 as a storage means. The control signal S5 is sent to the storage circuit 13 to release the bus between the microcomputer 10 and the main storage circuit 13, and the control signal S5 is sent to the main storage circuit 1.
3, the main storage circuit 13 stores the luminance image data Ya, Yb or Yc, the color difference image data Ra, Rb or Rc, and the color difference image data Ba, Bb or Bc.

The microcomputer 10 also includes luminance data Ya, Yb, and Yc corresponding to three frames of images labeled "a", "b", and "c", and color difference data Ra, Rb, and Rc. And color difference data Ba, Bb and B
c in the main storage circuit 13, the luminance data Ya, Yb and Yc for these three frames and the color difference data R
Based on a, Rb, and Rc and the color difference data Ba, Bb, and Bc, as described later, for example, a process of detecting the presence or absence of an obstacle and the boundary between the obstacle and the floor or the boundary between the table and the floor (hereinafter, this process is referred to as Is called an obstacle detection process).

Thereafter, the microcomputer 10 outputs the address signal S
6 to the input / output (I / O) control circuit 14 to release the bus between the microcomputer 10 and the I / O control circuit 14, and the traveling direction control signal S2 corresponding to the result of the obstacle detection processing. And an imaging direction control signal S3 to the I / O control circuit 1
The moving direction of the carriage 3 and the photographing direction of the CCD camera 4 are controlled by sending the signals to the motor 3B and the tilter 5 via the control unit 4 respectively. Further, the microcomputer 10 controls the control signal S4
Is transmitted to the LED 6 via the I / O control circuit 14 to control blinking of red light emitted from the LED 6.

When the microcomputer 10 completes the control of the motor 3B and the tilter 5 by performing the obstacle detecting process, the microcomputer 10 labels the labels of the luminance data Yc, the color difference data Rc and the color difference data Bc stored in the main storage circuit 13. In addition to "a", luminance data DY, chrominance image data DR, and chrominance image data DB corresponding to two frames of images are newly obtained, and the labels "b" and "c" are respectively arranged in time order.
Are stored in the main storage circuit 13 as luminance image data Yb, color difference image data Rb and color difference image data Bb, and luminance image data Yc, color difference image data Rc and color difference image data Bc. That is, the microcomputer 10 determines the traveling direction of the carriage 3 and the CCD camera based on the luminance data DY, the color difference data DR, and the color difference data DB for three frames to which the labels “a”, “b”, and “c” are respectively attached. The photographing direction of No. 4 is controlled.

Here, an obstacle detecting procedure and a moving procedure in the moving apparatus 1 will be described with reference to flowcharts shown in FIGS. First, the microcomputer 10 starts the obstacle detection procedure and the movement procedure from step SP1, and
In step 2, the control signal S4 is sent to the LED 6 to turn off the LED 6.

Subsequently, in step SP3, the microcomputer 10 stores, as the first image, the luminance data DY corresponding to one frame of the image when the LED 6 is turned off, the color difference data DR and the color difference data DB in the memories 11 and 12, respectively. Wait until stored in. The microcomputer 10
The luminance data DY corresponding to the image of one frame and the color difference data DR and DB are stored in the memory 11 and the memory 1 respectively.
When it is confirmed that the data has been stored in step S2, in step SP4, the luminance data DY and the color difference data DR and DB are fetched from the memories 11 and 12, and the luminance data DY, the color difference data DR and DB are labeled "a". After that, the luminance image data Ya, the color difference image data Ra and B
It is stored in the main storage circuit 13 as a.

Next, in step SP5, the microcomputer 10 turns on the LED 6 by sending a control signal S4 to the LED 6, and in step SP6, as the second image, brightness data corresponding to one frame image when the LED 6 is turned on. It waits until DY and the color difference data DR and DB are stored in the memory 11 and the memory 12, respectively. When the microcomputer 10 confirms that the luminance data DY and the chrominance data DR and DB corresponding to the image for one frame have been stored in the memories 11 and 12, respectively, in step SP7, the microcomputer 11 reads out the luminance data from the memories 11 and 12. DY and chrominance data DR and DB are fetched, and the luminance data DY and chrominance data DR and DB are labeled "b", and then stored as the main memory circuit 1 as luminance image data Yb, chrominance image data Rb and chrominance image data Bb.
3 is stored.

Next, the microcomputer 10 sends a control signal S4 to the LED 6 in step SP8 to make L
The ED 6 is turned off, and in step SP9, the process waits until the luminance data DY corresponding to one frame of the image when the LED 6 is turned off and the color difference data DR and DB are stored in the memories 11 and 12, respectively, as the third image. .
When the microcomputer 10 confirms that the luminance data DY corresponding to the image for one frame and the chrominance data DR and DB are stored in the memories 11 and 12, respectively, the microcomputer 11 reads these luminance data from the memories 11 and 12 in step SP10. Import DY, color difference data DR and DB,
After attaching the label “c” to the luminance data DY and the color difference data DR and DB, the luminance data DY and the color difference image data Rc and Bc are stored in the main storage circuit 13.

Subsequently, the microcomputer 10 proceeds to step SP11.
, The luminance image data Ya, Yb and Yc, the color difference image data Ra, Rb and Rc, and the color difference image data B
The effect of noise is reduced by performing filter processing on a, Bb, and Bc using a smoothing filter. Thereafter, the microcomputer 10 calculates the change in the luminance level of each pixel using the luminance image data Ya and the luminance image data Yc as the area detecting means, and calculates each pixel using the chrominance image data Ra and Rc. , The change in the red color level corresponding to the color of the light of the LED 6 is calculated, and the absolute value of the difference between the luminance levels is equal to or less than a threshold value T1 set in advance as a first luminance threshold value, and The pixel P whose absolute value of the difference is equal to or smaller than a threshold value T2 set in advance as a first color threshold value is detected.

Here, in this step SP11,
Since the two pixels captured at the time of turning off the LED 6 are used to detect the pixels P having the first luminance threshold value T1 or less and the first color threshold value T2 or less, a failure occurs. A stopped object such as an object and a moving object can be accurately distinguished.

Subsequently, the microcomputer 10 proceeds to step SP12 as an obstacle detecting means, and detects the presence or absence of the reflected light of the LED 6 in each pixel P by using the color difference image data Rb taken when the LED 6 is turned on. That is, the pixel values of the color difference image data Ra, Rb, and Rc are set to ra, r, respectively.
In terms of b and rc, the microcomputer 10 calculates, for each pixel P, the following equation (1) in which the pixel value of each pixel P is set as the second color threshold value

(Equation 1) By calculating whether or not the pixel Q is satisfied, a pixel P (hereinafter, referred to as a pixel Q) satisfying the expression (1) is calculated, and the number N of the pixels Q is counted as a calculating means. That is, the pixel P
If the pixel value of satisfies the expression (1), it is determined that the pixel P is receiving the light reflected by the obstacle, it is determined that the pixel P is a part of the obstacle, and the number N of the pixels Q is obtained. . Accordingly, the moving device 1 detects an obstacle in the traveling direction by performing the processing in steps SP11 and SP12.

Subsequently, the microcomputer 10 integrates the luminance data of each pixel of the image for each row (that is, in the horizontal direction of the image) at step SP13 as the integral data generating means based on the luminance image data Yc. The integral data (one-dimensional data) I for each row is created. That is, for example, when the image is composed of k pixels in the horizontal direction and l pixels in the vertical direction, the microcomputer 10 integrates k pixels for row j to generate integrated data I (j). .

Next, as a boundary detecting means, the microcomputer 10 divides the image into two in the horizontal direction, that is, divides the image into two parts in step SP14 based on the integral data I for each row generated in step SP13. At this time, the position X of the row in which the difference between the pixel averages in each of the divided regions is larger than the preset threshold value T3 and is the maximum is defined as the boundary between the obstacle and the table (or floor) or the boundary between the table and floor. To detect.

That is, the microcomputer 10 calculates the evaluation value E (m) at a certain row m in the image by the following equation (2).

(Equation 2) In the evaluation value E (m) for l obtained by using the equation (2), the position X of the row that is larger than the threshold value T3 and is the maximum is defined as an obstacle and a table (or floor). Is detected as a boundary between the table and the table and a floor.

Here, it is conceivable that the boundary detected in step SP14 is a table or floor pattern.
However, if the boundary detected in step SP14 is a table or floor pattern, the image captured by the CCD camera 4 includes pixels that have received the reflected light of the LED 6, whereas the detected boundary is the table and floor. In the case of the boundary of, since the reflected light of the LED 6 is not received,
By detecting the presence or absence of a pixel that has received the reflected light of the LED 6 based on the image captured when the LED 6 is turned on obtained in the next phase, it is possible to prevent a table or floor pattern from being detected as a boundary between the table and the floor. be able to.

Subsequently, in step SP15, the microcomputer 10 executes step SP12 as an obstacle position detecting means.
It is determined whether or not the number N of the pixels Q obtained in the above is greater than a preset threshold value T4, and if a positive result is obtained in step SP15, the microcomputer 10 controls the carriage 3 straight as it is as control means. Then, it is determined that the mobile device 1 collides with an obstacle, and the process proceeds to step SP16. That is, the microcomputer 10 detects the positional relationship between the obstacle and the moving device 1 as a change in the number N of the images Q, that is, the change in the area of the region receiving the reflected light of the LED 6 in this step SP12, and
4 to prevent the mobile device 1 from colliding with an obstacle.

The microcomputer 10 controls the traveling direction control signal S to the motor 3B in step SP16 as control means.
2 to change the direction of the carriage 3 in the direction corresponding to the traveling direction control signal S2, and then to step SP
Proceed to 17 to move the carriage 3 straight. Thus, in the moving device 1, even when an obstacle is encountered in the traveling direction of the moving device 1, it is possible to prevent the collision with the obstacle by changing the traveling direction of the bogie 3 before colliding with the obstacle. It is made to be able to do.

On the other hand, the microcomputer 10 determines as an obstacle position detecting means in step SP15 a negative result, that is, there is an obstacle in the traveling direction, but there is still no risk of collision with the obstacle even if the vehicle proceeds straight. In this case, the process proceeds to step SP18, and as the boundary position detecting means,
It is determined whether the position X obtained in step SP14 is above or below a preset position Y in the horizontal direction on the screen (for example, the center position in the horizontal direction on the screen). That is, when the position X is higher than the position Y in the screen as the boundary position detecting means, the microcomputer 10 determines that there is no risk of colliding with an obstacle or falling from the table, and proceeds to step SP17 to carry the truck as it is. Go 3 straight.

On the other hand, when the position X is lower than the position Y in the screen in step SP18, the microcomputer 10 as a boundary position detecting means, if the mobile device 1 goes straight ahead, collides with the obstacle, It is determined that there is a risk of falling from step SP19, and the process proceeds to step SP19. That is, in this step SP18, the microcomputer 10 determines the positional relationship between the moving device 1 and the boundary of the obstacle or the moving device 1 based on the position X in the screen.
By detecting the positional relationship between the table and the floor boundary, the moving device 1 is prevented from colliding with an obstacle or falling from the table.

Subsequently, the microcomputer 10 as a photographing direction control means determines in step SP19 that the photographing angle of the CCD camera 4 in the vertical direction is set to a predetermined threshold T5.
Is determined, and if a negative result is obtained, it is determined that there is a risk that the moving device 1 may collide with an obstacle or fall from the table when the trolley 3 is moved straight ahead, and the process proceeds to step SP20. move on. In step SP20, the microcomputer 10 sends the traveling direction control signal S2 to the motor 3B.
To change the direction of the carriage 3 in the direction corresponding to the traveling direction control signal S2, and then to step SP1.
At 7, the carriage 3 is moved straight.

On the other hand, if the microcomputer 10 obtains a positive result in step SP19 as the photographing direction control means, it proceeds to step SP21 and sends the photographing direction control signal S3 to the tilter 5 so that the position X is located at the center of the screen. Thus, after the shooting direction of the CCD camera 4 is set so that the position X is located at the center of the screen, the process proceeds to step SP17 to move the carriage 3 straight.

That is, the microcomputer 10 executes step SP1
8, even if it is determined that the moving device 1 may collide with an obstacle or fall from the table because the position X is below the position Y in the screen, the CCD camera 4 Is the threshold T5
, The vehicle 3 is turned upward without any risk of colliding with an obstacle or falling off the table even if the moving device 1 is moved straight ahead. I try to avoid it. Thereby, it is possible to prevent the moving range of the moving device 1 from being unnecessarily limited.

Thus, in this mobile device 1, the mobile device 1
Even if there is an obstacle or a boundary between the table and the floor in the direction of travel, before detecting the boundary between the obstacle and the table (or floor) and the boundary between the table and the floor, before colliding with the obstacle and before falling from the table By changing the traveling direction of the carriage 3, the moving device 1 can be prevented from colliding with an obstacle or falling from a table.

Subsequently, the microcomputer 10 proceeds to step SP22.
In FIG. 7, the labels of the luminance image data Yc and the chrominance image data Rc and Bc stored in the main storage circuit 13 are changed to the label "a" to obtain the luminance image data Ya and the chrominance image data Ra and Ba. Proceed to SP23. Next, the microcomputer 10 determines whether or not a switch provided in the mobile device 1 is turned off to end the operation, for example, and returns a step SP5 when a negative result is obtained.

Thereafter, the microcomputer 10 proceeds to step SP6.
By executing the processing from to step SP10, the luminance image data Yb and the color difference image data Rb and Rc are obtained as the fourth image, and the luminance image data Yc and the color difference image data Rc and Bc are obtained as the fifth image. These image data are stored in the main storage circuit 13 in place of the image data to which the above-mentioned labels “a” and “b” are attached, and are used as a third image in which the label “c” is replaced with the label “a”. Luminance image data Ya, color difference image data Ra and B
a, luminance image data Yb and chrominance image data Rb and Bb as the fourth image newly obtained in steps SP7 and SP10, and luminance image data Yc and chrominance image data Rc and Bc as the fifth image. Is used to repeat the processing from step SP11 to step SP23. Hereinafter, the above operation is repeated until the switch is turned off. If the microcomputer 10 obtains a positive result in step SP23, it proceeds to step SP24 and ends the obstacle detection procedure and the moving procedure.

In the above configuration, this moving device 1
After obtaining the luminance image data Ya and Yc and the color difference image data Ra, Rc and Ba and Bc when the LED 6 is turned off, and the luminance image data Yb and the color difference image data Rb and Bb when the LED 6 is turned on, the luminance image data Ya and Yc are obtained. And the color difference image data Ra and Rc, the change in the brightness level is the first.
And the change in the red color level is equal to or less than the first color threshold value T2, and the color difference image data Rb is used to detect the pixel P (1). )), The number N of the pixels Q receiving the reflected light of the LED 6 is counted.

When an obstacle is present, the luminance level and the red color level of the pixel capturing a part of the obstacle hardly change over time. Therefore, a part of the object captured by the pixel P can be determined to be a part of the stopped object, and thus there is a possibility that a part of the obstacle is captured. However, since a part of the object captured by the pixel P is not always an obstacle, if the pixel P is a pixel capturing a part of the obstacle, the pixel P receives reflected light when the LED 6 is turned on. Pixels that capture a part of the obstacle become reddish. Therefore, the obstacle is detected as the pixel Q of which the red color is increasing among the pixels P.

Further, as the moving device 1 approaches the obstacle, the number of pixels receiving the reflected light of the LED 6 increases, and the positional relationship with the obstacle is determined by the number N of the pixels Q receiving the reflected light.
, That is, a change in the area of the region receiving the reflected light of the LED 6.

Further, the moving device 1 controls the brightness image data Y
c, the integrated data I for each row is generated, and the evaluation value E is calculated for each of the integrated data I. The position X of the calculated evaluation value E which is larger than the threshold value T3 and which is the maximum is determined. To detect. Here, when there is an obstacle in the traveling direction of the moving device 1 or when there is a boundary between the table and the floor, the boundary between the obstacle and the table (or floor) or the boundary between the table and the floor is determined for each row. When the image is divided into two based on the integral data I, it is detected as the position X of the row at which the difference between the pixel averages of the respective regions at the luminance level is maximum. As the moving device 1 approaches the obstacle or the boundary between the table and the floor, the position X moves to the lower side in the screen. Therefore, the positional relationship between the obstacle and the table (or floor) boundary or the table and floor boundary is detected based on the position X in the screen.

Therefore, in the moving device 1, the obstacle can be detected as the pixel Q which has received the reflected light of the LED 6 among the pixels P, and the positional relationship with the obstacle can be detected by the LED 6.
Can be detected as a change in the area of the region receiving the reflected light (the number N of the pixels Q), so that the obstacle and the positional relationship with the obstacle can be accurately detected.

Further, in the moving device 1, the boundary between the obstacle and the table (or floor) or the boundary between the table and floor is detected as the position X of the line that is larger than the threshold value T3 and is the maximum in the evaluation value E. And the positional relationship between the obstacle and the table and the boundary between the floor and the table can be detected based on the position of the position X in the screen. It is possible to accurately detect the floor boundary and the positional relationship between the obstacle and the table and the floor boundary.

According to the above configuration, the absolute value of the change in the luminance level is equal to or less than the first luminance threshold T1 using the luminance image data Ya and Yc and the color difference image data Ra and Rc obtained when the LED 6 is turned off. And the absolute value of the change in the red color level is equal to or less than the first color threshold value T2, and the color difference image data Rb obtained when the LED 6 is turned on is used to detect the red color of the pixel P. The increased number of pixels Q is detected and the number N of the pixels Q is counted. When the number N of the pixels Q exceeds the threshold value T4, the bogie 3 is turned, so that obstacles and Since the positional relationship with the obstacle can be accurately detected, the collision with the obstacle can be reliably prevented. Thus, a moving device that can realize an obstacle detecting device and an obstacle detecting method that can accurately detect an obstacle with a simple configuration, and that can reliably prevent collision with an obstacle with a simple configuration, and A moving method can be realized.

Further, according to the above configuration, the integrated data I for each row of the luminance image data Yc obtained when the LED 6 is turned off is generated, and the evaluation value E is calculated based on the 1 integrated data I obtained for each row. Of the calculated evaluation value E, the position X of the row that exceeds the threshold value T3 and is the maximum is detected as the boundary between the obstacle and the boundary between the table and the floor. Is below position Y and CC
When the shooting direction of the D camera 4 does not exceed the threshold value T5, the direction of the cart 3 is changed.
It is possible to accurately detect the boundary between the obstacle and the floor, the boundary between the table and the floor, the positional relationship between the obstacle, and the positional relationship between the table and the floor. It can be reliably prevented. Thus, it is possible to realize an obstacle detection device and an obstacle detection method capable of accurately detecting an obstacle with a simple configuration, and reliably prevent a collision with the obstacle and drop from the table with a simple configuration. The moving device and the moving method that can be performed can be realized.

Further, according to the above configuration, the positional relationship with the obstacle can be detected as the number N of the pixels Q that have received the reflected light of the LED 6, so that the number of the pixels Q that have received the reflected light of the LED 6 can be detected. If the cart 3 is controlled so as to keep N constant, the moving object can be tracked while maintaining a constant distance from other moving objects. For example, in a game or the like that needs to track a moving object. Can be applied. Further, according to the above-described configuration, the shape of the obstacle can be recognized based on the pixel Q that receives the reflected light of the LED 6, and the color of the obstacle can be recognized based on the color difference image data. Therefore, for example, when playing a game using a robot device or the like, the present invention can be applied to a case where it is necessary to identify an opponent by color or shape.

In the above embodiment, the case where the pixel P is detected by using the luminance image data Ya and Yc and the color difference image data Ra and Rc in step SP11 of FIG. 3 has been described. Not limited to this, using the luminance image data Ya and Yc, a pixel whose luminance level is equal to or less than the first luminance threshold T1 may be detected as the pixel P, and using the color difference image data Ra and Rc,
A pixel whose color level is equal to or smaller than the first color threshold value T2 may be detected as the pixel P.

In the above-described embodiment, a case has been described in which the pixel Q of the pixel P which has received the reflected light of the LED 6 is calculated as a change in the red color level using the color difference image data Ra, Rb and Rc. However, the present invention is not limited to this, and may be calculated as a change in the luminance level using the luminance image data Ya, Yb, and Yc. In this case, a second luminance threshold value is set in advance, and the absolute value of the change in the luminance level is calculated based on the second luminance threshold value.
Is calculated as a pixel Q.
The following operation is the same as in the above-described embodiment.

The color difference image data Ra, Rb and Rc
And the luminance image data Ya, Yb and Yc, the absolute value of the change in the luminance level of the pixel P exceeds the second luminance threshold value and the absolute value of the change in the color level is the second color. A pixel exceeding the threshold may be calculated as the pixel Q.

In the above-described embodiment, the case where the LED 6 emitting red light is used as the light source has been described. However, the present invention is not limited to this, and a light source such as a light source emitting light other than red light may be used as the light source. Other various light sources may be used.
In addition, if a plurality of light sources that emit light of different colors are used as light sources, an obstacle can be detected with higher accuracy.

Further, in the above-described embodiment, the integrated data I for each row is calculated using the luminance image data Yc, and the evaluation value E calculated for each row exceeds the threshold value T3.
Further, the case where the position X of the maximum row is detected as the boundary between the obstacle and the table (or the floor) or the boundary between the table and the floor has been described. However, the present invention is not limited to this. Is used to calculate the integral data for each column (vertical direction of the image), and among the evaluation values calculated for each column, the position of the row that exceeds the preset threshold value T6 and is the maximum is regarded as an obstacle. It may be detected as a boundary between a table (or a floor) and a boundary between a table and a floor. In this case, it is particularly effective when the moving device 1 is moving along the wall, and it is possible to prevent the moving device 1 from colliding with the wall.

Further, in the above-described embodiment, after three frames of image data are first fetched into the main storage circuit 13, two frames of image data are newly fetched.
Although the case where the obstacle detection process is performed based on the image data of the two frames and the image data replaced with the label “a” (original image data with the label “c”) has been described, The present invention is not limited to this.
If a negative result is obtained in step 23, return to step SP2,
Three new frames of image data are fetched and these three
Obstacle detection processing may be performed based on image data for a frame.

Further, in the above-described embodiment, when the image is divided into two based on the integral data I for each row obtained by integrating the luminance data of each pixel in the image for each row,
The row where the difference between the pixel averages of the divided areas is larger than the threshold value T3 and is the maximum is defined as an obstacle and a table (or floor).
However, the present invention is not limited to this, and a row in which the difference between the pixel averages of the divided regions is the largest is defined as an obstacle and a table (or (A floor) or a boundary between a table and a floor.

Further, in the above embodiment, the tilter 5
Although the case where the photographing direction of the CCD camera 4 is controlled by using the above is described, the present invention is not limited to this, and the photographing direction of the CCD camera 4 is controlled by using various other photographing direction control means such as a pan tilter. You may.

Further, in the above-described embodiment, the case where the obstacle detecting device 2 according to the present invention is applied to the bogie type moving device 1 has been described. However, the present invention is not limited to this, and the present invention is applied to a portion corresponding to FIG. As shown in FIG.
The present invention can be applied to various other moving devices as long as the device is configured to be movable, such as a robot device 20 that can walk, stand or sit on four legs.

[0060]

As described above, according to the present invention, the first image when the light source is turned off, the second image when the light source is turned on, and the third image when the light source is turned off are sequentially obtained from the imaging means. By comparing the first image and the third image, the change in the brightness level is equal to or less than the first brightness threshold value and / or the change in the color level of the predetermined color is the first image.
And detecting, based on the second image, a change in the brightness level of the detected area that exceeds the second brightness threshold value and / or a color level of a predetermined color. Detects an area exceeding the second color threshold as an obstacle, calculates an area of the detected area, detects a positional relationship with the obstacle based on the calculation result, and determines a positional relationship with the obstacle. By controlling the driving means of the moving device on the basis of, the change in the luminance level and / or the color level of the predetermined color is less than or equal to the first luminance threshold and / or the first color threshold, respectively. In addition, it can be detected as an area that exceeds the second luminance threshold and / or the second color threshold, respectively, and the positional relationship with the obstacle can be detected as the area of the detected area. Thus, an obstacle detecting device, an obstacle detecting method, a moving device, and a moving method that can accurately detect an obstacle with a simple configuration can be realized.

According to the present invention, the integrated data for each row or each column is generated by integrating the brightness data of each pixel in the image obtained by the imaging means for each row or each column, and the integrated data for each row or each column is generated. When the image is divided into two based on the integral data for each, the row or column in which the difference in pixel average in each region of the divided image is the largest is detected as the boundary with the obstacle, and the positional relationship of the boundary is determined. , And controlling the driving means of the moving device based on the positional relationship with the boundary, the boundary between the obstacle and the row or column where the difference of the average of the pixels in each region of the divided image is maximized. Can be detected as Thus, an obstacle detecting device, an obstacle detecting method, a moving device, and a moving method that can accurately detect an obstacle with a simple configuration can be realized.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an embodiment of a moving device to which an obstacle detection device according to the present invention is applied.

FIG. 2 is a block diagram for explaining the configuration of the arithmetic unit.

FIG. 3 is a flowchart for explaining an obstacle detection procedure and a moving procedure.

FIG. 4 is a flowchart for explaining an obstacle detection procedure and a movement procedure.

FIG. 5 is a schematic diagram showing a robot device to which the obstacle detection device according to the present invention is applied.

[Explanation of symbols]

1, 20... Moving device, 2... Obstacle detection device, 3.
Dolly, 4 ... CCD camera, 5 ... Tilter, 6 ... LE
D, 7 ... Y / C separation circuit, 8, 9 ... A / D conversion circuit, 10 ... Microcomputer, 11, 12 ... Memory, 13 ... Main storage circuit, 14 ... I / O control circuit .

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code Agency reference number FI Technical indication // G01V 8/10 G01V 9/04 S

Claims (26)

[Claims] A light source which emits light intermittently; an image pickup means for picking up an image of a subject in a direction illuminated by the light emitted from the light source; Storage means for storing the first image, the second image when the light source is turned on, and the third image when the light source is turned off; the first image and the third image stored in the storage means , The change in luminance level is the first
Area detection means for detecting an area in which a change in the color level of a predetermined color is equal to or less than a first color threshold, and / or Among them, an area in which the change in the luminance level exceeds the second luminance threshold value and / or an area in which the color level of the predetermined color exceeds the second color threshold value is detected, and the area is detected as an obstacle. An obstacle detection device comprising: an object detection unit. A calculating means for calculating an area of the area detected by the obstacle detecting means; and an obstacle position detecting means for detecting a positional relationship with the obstacle based on a calculation result of the calculating means. The obstacle detection device according to claim 1, further comprising: 3. The storage means stores a fourth image when the light source is turned on and a fifth image when the light source is turned off, instead of the first image and the second image, The area detection means compares the third image and the fifth image stored in the storage means, whereby a change in the brightness level is equal to or less than the first brightness threshold value, and Alternatively, an area in which a change in the color level of the predetermined color is equal to or less than the first color threshold value is detected, and the obstacle detection unit determines that the change in the luminance level is based on the fourth image. 1 in the area where the change in the color level of the predetermined color is equal to or less than the first luminance threshold, and / or the change in the luminance level is equal to or less than the second luminance threshold. And / or if the color level of the predetermined color is the second color threshold Detecting a region exceeding the value, the obstacle detection apparatus according to claim 1, characterized in that detecting the area as an obstacle. 4. An integrated data generator for generating and outputting integrated data for each row or column by integrating the luminance data of each pixel in the image obtained by the imaging means for each row or column of the image. Means for converting the image into two based on the output of the integration data generating means.
A boundary detecting unit that detects the row or column in which the difference in pixel average in each region of the divided image is maximum when the division is performed, and detects the row or column as a boundary with an obstacle. An obstacle detection device, characterized in that: 5. The obstacle detecting device according to claim 4, further comprising a boundary position detecting means for detecting a positional relationship with the boundary based on a detection result of the boundary detecting means. 6. A boundary position detecting means for detecting a positional relationship with the boundary based on a detection result of the boundary detecting means, and a photographing direction of the photographing means is controlled based on a detection result of the boundary position detecting means. 5. The obstacle detecting device according to claim 4, further comprising a photographing direction control unit. 7. The boundary detecting means detects the row or column in which the difference between the pixel averages in each area of the divided image exceeds a predetermined threshold value and the difference between the pixel averages is maximum. The obstacle detection device according to claim 4, wherein the row or the column is detected as a boundary with the obstacle. 8. An imaging step for imaging an object in a direction illuminated by light emitted from a light source intermittently emitting light, and a first image sequentially obtained in the imaging step when the light source is turned off, The second when the light source is turned on
And a storage unit for storing the third image when the light source is turned off, and comparing the first image and the third image stored in the storage unit to determine a change in the luminance level. First
An area detection step for detecting an area in which a change in color level of a predetermined color is equal to or less than a first color threshold value, and / or an area detection step based on the second image. Among them, an area in which the change in the luminance level exceeds the second luminance threshold value and / or an area in which the color level of the predetermined color exceeds the second color threshold value is detected, and the area is detected as an obstacle. An obstacle detection method, comprising: an obstacle detection step. 9. An operation step for calculating an area of the area detected by the obstacle detection step, and an obstacle position detection for detecting a positional relationship with the obstacle based on a calculation result of the operation step. 9. The obstacle detecting method according to claim 8, comprising a step. 10. The storage step stores, in place of the first image and the second image, a fourth image when the light source is turned on and a fifth image when the light source is turned off. The area detecting step compares the stored third image and the fifth image to determine that the change in the luminance level is equal to or less than the first luminance threshold value and / or the predetermined luminance value. An area in which a change in color level of a color is equal to or less than the first color threshold value is detected, and the obstacle detection step includes a step in which the change in the luminance level is based on the fourth image. Wherein the change in the color level of the predetermined color is equal to or less than a threshold value and / or the change in the color level of the predetermined color is equal to or less than the first color threshold value; And / or the color level of the predetermined color is a second color Obstacle detection method according to claim 8, characterized in that detecting a region exceeding the threshold, to detect the area as an obstacle. 11. An integrated data generation step for integrating the luminance data of each pixel in the image obtained by the imaging means for each row or column of the image to generate integrated data for each row or column. When the image is divided into two based on the integration data for each row or each column generated in the integration data generation step, the row or the row where the difference of the average of the pixels in each area of the divided image is the maximum. An obstacle detection method, comprising: detecting a column and detecting the row or column as a boundary with the obstacle. 12. The obstacle detecting method according to claim 11, further comprising a boundary position detecting step for detecting a positional relationship with the boundary based on a detection result of the boundary detecting step. 13. A boundary position detection step for detecting a positional relationship with the boundary based on the detection result of the boundary detection step, and a photographing direction of the photographing means is controlled based on the detection result of the boundary position detection step. The obstacle detecting method according to claim 11, further comprising a photographing direction control step. 14. The boundary detecting step detects the row or column in which the difference between the pixel averages in each area of the divided image exceeds a predetermined threshold value and the difference between the pixel averages is maximum. The obstacle detection method according to claim 11, wherein the row or column is detected as a boundary with the obstacle. 15. A moving device that moves based on an output of a driving means, wherein: a light source intermittently emits light; an imaging means for imaging a subject in a direction irradiated by light emitted from the light source; Storage means for storing a first image when the light source is turned off, a second image when the light source is turned on, and a third image when the light source is turned off, sequentially obtained from an imaging means; By comparing the obtained first image and the third image, the change in the luminance level is equal to the first image.
Area detection means for detecting an area in which a change in the color level of a predetermined color is equal to or less than a first color threshold, and / or Among them, an area in which the change in the luminance level exceeds the second luminance threshold value and / or an area in which the color level of the predetermined color exceeds the second color threshold value is detected, and the area is detected as an obstacle. Object detecting means, calculating means for calculating the area of the region detected by the obstacle detecting means, and obstacle position detecting for detecting a positional relationship with the obstacle based on the calculation result of the calculating means. And a control means for controlling the driving means based on the detection result of the obstacle position detecting means. 16. The apparatus according to claim 15, wherein said control means controls said driving means so as to keep a distance to said obstacle constant based on a result of detection by said obstacle position detecting means. A mobile device as described. 17. The storage means stores a fourth image when the light source is turned on and a fifth image when the light source is turned off, instead of the first image and the second image, The area detection means compares the third image and the fifth image stored in the storage means, so that the change in the brightness level is equal to or less than a first brightness threshold value, and / or An area in which a change in the color level of the predetermined color is equal to or less than a first color threshold value is detected, and the obstacle detection unit determines that the change in the brightness level is based on the first image based on the fourth image. In the region where the change in the color level of the predetermined color is equal to or less than the luminance threshold and / or the change in the color level of the predetermined color is equal to or less than the first color threshold, the change in the luminance level exceeds the second luminance threshold. And / or the color level of the predetermined color sets a second color threshold value. Detects obtain area, the mobile apparatus according to claim 15, characterized in that detecting the area as an obstacle. 18. A moving device which moves based on an output of a driving means, wherein the luminance data of each pixel in an image obtained by the imaging means is integrated for each row or each column, whereby the integration for each row or each column is performed. Integration data generating means for generating and outputting data; and outputting the image based on the output of the integration data generating means.
A boundary detecting means for detecting the row or column in which the difference between the pixel averages in each region of the divided image is maximum when dividing, and detecting the row or column as a boundary with an obstacle; Boundary position detection means for detecting the positional relationship with the boundary based on the detection result of the detection means, and control means for controlling the driving means based on the detection result of the boundary position detection means, Moving device. 19. The boundary detecting means detects the row or column in which the difference between the pixel averages in each region of the divided image exceeds a predetermined threshold value and the difference between the pixel averages is maximum. The mobile device according to claim 18, wherein the row or the column is detected as a boundary with an obstacle. 20. The moving apparatus according to claim 18, further comprising a photographing direction control means for controlling a photographing direction of said photographing means based on a detection result of said boundary position detecting means. 21. A moving method for controlling a moving direction of a moving device which moves based on an output of a driving means, comprising: A first image when the light source is turned off, and a second image when the light source is turned on, which are sequentially obtained in the imaging step.
By comparing the stored first image and the third image with the storage step for storing the third image when the light source is turned off and the third image when the light source is turned off, the change in the luminance level can be changed to the first image. An area detection step for detecting an area where the change in the color level of the predetermined color is equal to or less than the luminance threshold and / or a change in the color level of the predetermined color equal to or less than the first color threshold; An area in which the change in the luminance level exceeds a second luminance threshold value and / or an area in which the color level of the predetermined color exceeds the second color threshold value, and the area is detected as an obstacle; A detection step, a calculation step for calculating the area of the region detected in the obstacle detection step, an obstacle position detection step for detecting a positional relationship with the obstacle based on a calculation result of the calculation step, Movement method characterized by comprising a control step for controlling said drive means based on a detection result of the obstacle position detection step. 22. The apparatus according to claim 21, wherein said control step controls said driving means so as to maintain a constant distance from said obstacle based on a detection result of said obstacle position detection step. Movement method described. 23. The storage step stores, in place of the first image and the second image, a fourth image when the light source is turned on and a fifth image when the light source is turned off, The area detecting step compares the third image and the fifth image stored in the storage means, and determines that the change in the brightness level is equal to or less than the first brightness threshold value, and Alternatively, an area in which a change in the color level of the predetermined color is equal to or less than the first color threshold value is detected, and the obstacle detection step determines that the change in the brightness level is based on the first image based on the fourth image. And the change in the color level of the predetermined color is equal to or less than the first color threshold. Over and / or the color level of the predetermined color Movement method of claim 21 in which the detecting areas exceeding the color threshold, and detects the area as an obstacle. 24. A moving method for controlling a moving direction of a moving device that moves based on an output of a driving means, comprising: integrating luminance data of each pixel in an image obtained by an imaging means for each row or each column; An integration data generation step for generating integration data for each row or column; and dividing the image into two parts based on the integration data for each row or column generated in the integration data generation step. A boundary detection step of detecting the row or column in which the difference in pixel average in each region of the image is the largest, and detecting the row or column as a boundary with an obstacle; and a detection result of the boundary detection step. A boundary position detecting step for detecting a positional relationship with the boundary, and a control step for controlling the driving means based on a detection result of the boundary position detecting step. A moving method characterized by comprising steps. 25. The boundary detecting step detects the row or column in which the difference between the pixel averages in each area of the divided image exceeds a predetermined threshold value and the difference between the pixel averages is maximum. The moving method according to claim 24, wherein the row or the column is detected as a boundary with the obstacle. 26. A moving method according to claim 24, further comprising a photographing direction control step for controlling a photographing direction of said photographing means based on a detection result of said boundary position detecting step.