JPH01188911A - Detecting method for abnormal traveling route of image type unmanned vehicle - Google Patents

Abstract

PURPOSE:To easily decide the normal or abnormal drive of an unmanned vehicle by storing previously the sections where a drive instruction mark should be picked up by an image pickup device mounted on the vehicle and other sections where no mark is picked up and comparing these stored sections with the actual image pickup data. CONSTITUTION:A CPU 11 performs the control of an unmanned vehicle via the processing means for image pickup, recognition, arithmetic and drive respectively and at the same time sets previously the cycle of the image pickup processing action. When a CCD camera 3 starts its image pickup action with a control signal received from the CPU 11, the image is stored in a work memory 13. At the same time, a pattern is recognized and a mark 6 is decided. In this case, the position of the mark 6 in the image is calculated together with a range including completely the mark 6 and decided previously. Based on these calculation data, the CPU 11 decides a traveling route L of the vehicle and also sets the next image pickup timing to start the drive of the vehicle. However the abnormal drive is decided in case the mark 6 is soiled or mislocated since the difference is produced from the prescribed data.

Description

[Detailed Description of the Invention] [Industrial Application Field] This invention captures an image of a sign placed on the road surface to instruct the driving of an unmanned vehicle using an imaging device installed in the unmanned vehicle, and Image processing of pixel signals to recognize signs,
The present invention relates to a detection method for detecting abnormalities in the driving route of an image-based unmanned vehicle that determines the driving route of the unmanned vehicle.

[Prior Art] Conventionally, this type of image-based unmanned vehicle images a continuous driving line as a sign placed on the road surface, and processes the pixel signals in the captured image to determine the driving direction. A device that runs on a road surface along a route is known. In this image-based unmanned vehicle, the steering angle is determined from the deviation of the forward running line in the captured image.

Therefore, in this image-based unmanned vehicle, there may be a missing part in the middle of the driving line, the unmanned vehicle may deviate from the driving route, or there may be unnecessary dirt or obstacles near the driving line. In such cases, countermeasures have been taken, such as determining that there is an abnormality in the travel route and performing stop control.

However, in the image-based unmanned vehicle, there is a problem in that the running line must be provided continuously without interruption, and it takes time and effort to install it. Therefore, an image-based unmanned vehicle has been proposed that captures images of discretely arranged marks as signs that can reduce the time and effort required to install the signs, and processes the captured images to determine a driving route.

[Problems to be Solved by the Invention] Therefore, even in image-based unmanned vehicles that employ conventional discretely arranged marks, problems occur when marks are missing, the unmanned vehicle deviates from its driving route, or there is unnecessary dirt or dirt on the road surface. When there is an obstacle, it is necessary to take measures such as determining that the driving route is abnormal and controlling the unmanned vehicle to stop.

This invention has been made in view of the above-mentioned circumstances, and its purpose is to image discretely arranged marks, process the pixel signals in the captured image, and recognize the signs, thereby allowing unmanned vehicles to move. In an image-based unmanned vehicle that determines the route, the traveling route is abnormal if a mark is missing, the unmanned vehicle deviates from the traveling route, or if there is unnecessary dirt or obstacles on the road surface between each mark. An object of the present invention is to provide an image-based method for detecting an abnormality in a driving route in an unmanned vehicle, which can easily detect abnormalities.

[Means for Solving the Problems] In order to achieve the above object, the present invention provides an imaging device in which an unmanned vehicle is equipped with marks that are discretely arranged on the road surface at predetermined intervals to instruct the unmanned vehicle to travel. In an image-based unmanned vehicle, the driving route is determined by sequentially capturing images as the unmanned vehicle moves, and by processing the pixel signals in the captured images to recognize marks and determining the driving route of the unmanned vehicle. When doing so, the driving section where the next mark should be imaged and the driving section where the next mark should not be imaged are set in advance, and the imaging device is operated to take images from time to time while the unmanned vehicle is traveling, and the image is taken. If the operating point is a driving section where a mark should be imaged but the mark is not imaged, or if the imaging operation point is a driving section where a mark is not supposed to be imaged and some object is imaged, the actual It is determined that the travel route is abnormal.

[Effect] Therefore, if a part of the discretely arranged mark is missing or if the unmanned vehicle deviates from the travel route determined by image processing, the imaging operation point of the imaging device may be affected by the mark. Marks are no longer imaged even in the driving section that should be imaged, and the actual driving route of the unmanned vehicle is determined to be abnormal. In addition, if there is dirt or an obstacle on the road surface between the marks that could be mistaken for a mark, the image capturing operation point of the imaging device is in a driving section where the mark should not be imaged. Also, some object will be imaged, and it will be determined that the actual travel route of the unmanned vehicle is abnormal.

[Embodiment] Hereinafter, an embodiment embodying the present invention will be described in detail based on the drawings.

FIG. 2 shows a side view of the image-based unmanned vehicle 1. A CCD (charge cd) serving as an imaging device is installed at the upper center of the support frame 2, which is erected at the upper center of the front side of the unmanned vehicle 1.
A camera 3 is provided. The CCD camera 3 is mounted on the support frame 2 so as to image a trapezoidal area E (see FIG. 3) on the road surface 4 in front of the unmanned vehicle 1, which is set in advance. In this embodiment, the trapezoidal area E imaged by the CCD camera 3 is captured as a rectangular image 5 as shown in FIG.

In addition, in this embodiment, the area E imaged by the CCD camera 3
Image 5 is composed of 256×256 pixels.

As shown in FIG. 2.3, marks 6 for instructing the driving of the unmanned vehicle 1 are discretely arranged on the road surface 4 at predetermined equal intervals (in this case, 5 meter intervals).

As shown in FIG. 4, in this embodiment, the mark 6 has a circular shape, and a pair of pointed corners 6a are formed extending from opposite sides thereof.

The extending direction of the line β connecting the pair of corner portions 6a indicates the traveling direction of the unmanned vehicle 1 when passing the mark 6, and also indicates the direction in which the next mark 6 is located.

As the unmanned vehicle 1 travels, the discretely arranged marks 6 are sequentially imaged by the CCD camera 3. Further, in this embodiment, the mark 6 is bright white, and the road surface 4 is dark colored. Therefore, the signal from the CCD camera 3 that captured the mark 6 (hereinafter referred to as "
The level of the pixel signal (referred to as "pixel signal") is high (on the contrary, the level of the pixel signal from the CCD camera 3 that captures the image of the road surface 4 is low).

In this embodiment, the pixel signal of the captured image 5 is converted into predetermined pixel data, and then the image is processed to recognize the instruction information of the mark 6, so that the driving path L of the unmanned vehicle 1 (see FIG. 3) is determined. is determined, and the driving of the unmanned vehicle 1 is controlled.

Next, the electrical configuration of the travel control device mounted on the unmanned vehicle 1 will be explained with reference to FIG.

The microcomputer 10 is a central processing unit (rC
PUJ) 11, a program memory 12 consisting of a read-only memory (ROM) that stores control programs, and a readable and rewritable memory that temporarily stores CPU II arithmetic processing results, pixel data, etc.
It consists of a working memory 13 (RAM), a timer 14, etc. Then, the CPU II controls the CCD according to the control program stored in the program memory 12.
The camera 3 is operated, and based on the captured image information, a travel route for the unmanned vehicle 1 is determined, and various arithmetic processing operations for controlling travel and steering are executed.

The A/D converter 16 A/D converts the pixel signals from the CCD camera 3 from analog values to digital values, and at the same time determines whether each pixel signal is greater than or equal to a predetermined set value during the A/D conversion. If the pixel signal is greater than the set value, it is set as "1" corresponding to the pixel in the white mark 6 part, and on the other hand, if the pixel signal is less than the set value, it is set as "1" for the pixel in the dark road surface 4 part. Each pixel signal that is sequentially input is binarized into pixel data so that rOJ corresponds to . Therefore, the image 5 captured by the CCD camera 3 is stored in the working memory 13 as a group of 256 x 256 binarized pixel data.

In this embodiment, when a new image pixel data group is input to the working memory 13, the previous pixel data group is erased from a predetermined storage area, and the new image pixel data group is stored in the same storage area. is memorized.

Further, in this embodiment, for convenience of explanation, a scanning method is adopted in which the image 5 captured by the CCD camera 3 is scanned in the horizontal direction, and the scanning moves from the top to the bottom of the image 5. Of course, the scanning method may also be used.

The binarization level controller 18 outputs setting value data for the A/D converter 16 to binarize based on the control signal from the CPU II to the A/D converter 16.

The drive controller 19 controls the driving motor 20 and the steering mechanism 21 based on the same control signal (from CPULI).The rotational speed of the driving motor 20 is controlled based on the control signal, and the steering mechanism 21 is controlled based on the control signal. The steering angle is controlled based on the control signal.

The vehicle speed detector 22 detects the rotation speed of the drive system of the unmanned vehicle 1 and outputs the detection signal to the CPUI 1 via the input/output interface 15. CPU II is vehicle speed detector 22
The traveling speed of the unmanned vehicle 1 is calculated based on the detection signal from the unmanned vehicle 1, and the traveling distance of the unmanned vehicle 1 is calculated by integrating the traveling speed and the time measured by the timer 14.

゛ Furthermore, after the previous mark 6 is first imaged by the CCD camera 3 and the unmanned vehicle 1 starts traveling based on the instruction information of the mark 6, the CPU II selects the next adjacent mark 6.
The image capturing timing is set until the image is first captured by the CCD camera 3.

That is, in this embodiment, the imaging timing of the CCD camera 3 is set at equal distance intervals in accordance with the travel of the unmanned vehicle 1. For example, in FIG. 6(a), the unmanned vehicle 1 captures an image of the first area E1 at the first position P1 as the imaging operation point indicated by the solid line, and captures the mark 6 first. The imaging timing is set so that an image of the second area E2 is captured at a second position P2 as an imaging operation point indicated by a two-dot chain line that is a distance forward from P1, and further forward by a distance. a third position P3 and a fourth position P as imaging operation points which are separated by
4. The third area E3, the fourth area E4, and the fifth area E5 at the fifth position P5 and the sixth position P6, respectively.
Then, the imaging timing is set to capture an image of the sixth area E6.

Further, the CPU II presets in advance a traveling section in which the mark 6 should be imaged next and a traveling section in which the mark 6 should not be imaged next, corresponding to each of the imaging timings.

That is, as shown in FIGS. 6(a) and 6(b), at the imaging timing of each of the areas E2 to E6, each area E2. E3.
The traveling section S1 from the first position P1 to the fourth position P4 and the traveling section S3 from the fifth position P5 to the sixth position P6 corresponding to the imaging timing of E6 are the traveling sections in which the mark 6 should be imaged. Set as an interval. Also, the fourth
The traveling section S2 from the fourth position P4 to the fifth position P5 corresponding to the imaging timing of the area E4 and the fifth area E5 is set as the traveling section in which the mark 6 should not be imaged. These travel sections 81 to S3 are set in the first
The distance is set based on the distance from the position P1 of .

Then, the traveling section 31 in which the mark 6 is to be imaged.
33, each area E2. E3. Each position P2 corresponding to the imaging timing of E6. If the mark 6 is not imaged at P3°P6, or if the mark 6 is not to be imaged in the traveling section S2, each area E4. When some object (for example, dirt on the road surface 4, an obstacle, etc. that may be mistaken for Nimark 6) is imaged at the fourth position P4 and the fifth position P5 corresponding to the imaging timing of E5. , respectively, and determine that the actual traveling route of the unmanned vehicle 1 is abnormal, and based on the determination result, the traveling motor 20 and the like are controlled to stop, thereby causing the unmanned vehicle 1 to stop traveling.

Then, the CPU 1 indicates that the unmanned vehicle 1 is running in each traveling section 31 to S.
3 is determined by calculating the distance traveled from the first position P1 based on the operation of the vehicle speed detector 22 and timer 14.

Next, the processing operation of the CPUI 1 will be specifically explained.

The basic operations of the CPU II are an imaging processing operation that activates the CCD camera 3, a recognition processing operation that performs image processing to recognize the mark 6 provided on the road surface 4 based on the image 5 taken by the camera 3, and the recognition. A calculation processing operation for calculating the traveling route of the unmanned vehicle 1 based on the result, and controlling the driving motor 20 and the steering mechanism 21 based on the calculation result to control the unmanned vehicle 1.
It consists of a traveling processing operation that causes the vehicle to travel. Then, the CPU II performs control in the order of imaging processing operation - recognition processing operation - arithmetic processing operation - running processing operation, and not only sets the cycle of the imaging processing operation in advance, but also sets the time of each processing operation in advance. There is. Then, by sequentially repeating a series of processing operations, the unmanned vehicle 1 travels along the travel route. Note that C
The PU 11 controls the driving of the unmanned vehicle 1 through a driving processing operation based on the above calculation processing operation.

Now, mark 6 shown by broken lines in FIGS. 6(a) and (b)
When the driving route is determined based on the instruction information and the unmanned vehicle 1 is under driving control, when the CCD camera 3 is operated based on the control signal from the CPU II, the CC
The D camera 3 images and scans a first area E1 in front indicated by a two-dot chain line in the figure into an image 5 as shown in FIG. 7(a).

The pixel signal corresponding to each pixel from the CCD camera 3 is A/D converted by the A/D converter 16 and binarized, and is stored as pixel data in a predetermined storage area of the work memo 1J130.

In the pixel data stored in the working memory 13, the intensity of the part corresponding to the mark 6 becomes large, the intensity of the part corresponding to the road surface 4 becomes small, and the part corresponding to the mark 6 and the part corresponding to the road surface 4 are separated. will be determined respectively.

Then, in order to process the pixel data of this image 5 at high speed, the CPU 1 sets a processing area 7 in a predetermined range that completely includes the mark 6 in the image 5, and also determines the position of the processing area 7 at the subsequent <imaging timing. Prediction settings are made in image 5.

That is, in which position of the image 5 captured by the CCD camera 3 while the unmanned vehicle 1 is traveling, the mark 6 is present, and where is the mark 6?
Determine what is the predetermined range that completely includes.

In this case, the processing area 7 is set based on the traveling route that has already been determined based on the instruction information of the mark 6, and the preset interval of the imaging timing of the CCD camera 3, but the details thereof will be omitted.

Then, based on the pixel data of the set processing area 7, the CPU 1 executes a predetermined recognition processing operation and arithmetic processing operation to determine the driving route revealed by the instruction information of the mark 6 (in this case, the CPU 1 determines the driving route that is revealed by the instruction information of the mark 6) The driving route is determined), the next series of imaging timings are set, and the driving section corresponding to the imaging timing at which mark 6 is to be imaged is set, and the next image of mark 6 is to be imaged. After setting the prohibited travel section, a travel processing operation is executed to cause the unmanned vehicle 1 to travel.

That is, in FIGS. 6(a) and (b), each area E2 to E
6 is set, and then each area E2.6 corresponding to the imaging timing at which the mark 6 should be imaged is set. E3.
Travel section St including E6.

S3 is set, and a travel section S2 is set that honors each area E4 to E5 corresponding to the ↑most image timing at which the mark 6 should not be imaged next. That is, the first position P1
Each traveling section 81 to S3 is set based on the distance from the first position P1 to the sixth position P6 (distance five times the distance).

Therefore, when the unmanned vehicle 1 actually starts traveling according to the travel route determined at the first position P1 in FIG. and at the third position P3, the CCD camera 3 is operated to take an image at the imaging timing set as described above,
The second area E2 is captured and scanned as an image 5 as shown in FIG. 7(b), and the third area E3 is scanned as shown in FIG. 7(b).
An image 5 as shown in C) is captured and scanned.

Then, the CPU 1 controls each area E2. Regarding the image 5 of E3, a predetermined recognition processing operation and arithmetic processing operation are performed based on the pixel data of a preset processing area 7 including the mark 6, and the driving route is corrected and determined based on the instruction information of the mark 6. The travel processing operation is further executed. As a result, the unmanned vehicle 1 travels further straight along the determined travel route.

Subsequently, when the unmanned vehicle 1 travels, the CPU 11 causes the CCD camera 3 to perform an image capturing operation at the fourth position P4 and the fifth position P5 at the image capturing timing set as described above, and captures images in the fourth area. E4 and the fifth position P5 are imaged and scanned. In this case, since no object is imaged on the road surface 4 in the fourth area E4 and the fifth area E5,
The CPU 11 continues the travel processing operation and causes the unmanned vehicle 1 to travel along the travel route.

Furthermore, if there is dirt or obstacles on the road surface 4 that could be mistaken for mark 6 in the fourth area E4 and fifth area E5, they will be imaged by the CCD camera 3. Therefore, the CPU 1 determines that the actual traveling route is abnormal, and based on the determination result, controls the driving motor 20 and the like to stop, thereby causing the unmanned vehicle 1 to stop traveling.

Furthermore, when the unmanned vehicle 1 travels, the CPU II causes the CCD camera 3 to take an image at the imaging timing set as described above at the sixth position P6, and images and scans the sixth area E6.

As a result, the CPU II, based on the pixel signal of the image captured in the sixth area E6 (equivalent to image 5 in FIG. 7(a)),
A predetermined processing area 7 including the mark 6 is set. and,
Based on the pixel data in the processing area, a predetermined recognition processing operation and arithmetic processing operation are executed to determine the driving route according to the instruction information of the mark 6, and each imaging timing until the next mark 6 is imaged is determined. At the same time, after setting the traveling section where the mark 6 should be imaged next and setting the traveling section where the mark 6 should not be imaged, the unmanned vehicle 1 runs by executing the traveling processing operation. let

If the mark 6 to be imaged in the sixth area E6 is missing, the mark 6 will not be imaged by the CCD camera 3, and the CPU II will determine that the actual travel route is abnormal. Based on the determination result, the driving motor 20 and the like are controlled to stop, thereby causing the unmanned vehicle 1 to stop traveling.

Thereafter, a series of processing operations are similarly executed, and the unmanned vehicle 1 is continuously driven.

On the other hand, for example, the traveling route of the unmanned vehicle 1 is determined at the first position P1 in FIGS. If, for some reason, the unmanned vehicle 1 deviates from the determined travel route and travels in the direction of arrow Z after the travel sections S2 in which the mark 6 is not to be imaged are set, When the travel zone SL, S3 corresponding to the imaging timing set at the first position P1 is reached, the mark 6 is no longer imaged by the CCD camera 3. As a result, cputt determines that the actual travel route of the unmanned vehicle 1 is abnormal, and based on the determination result, stops the traveling motor 20 and the like to stop the unmanned vehicle 1 from traveling.

As described above, in this embodiment, when determining the travel route, the imaging timing is set at a predetermined interval, and then the mark 6
The driving sections SL and S3 where the mark 6 is to be imaged are set, and the driving section S2 where the mark 6 is not to be imaged is set.
The CD camera 3 is operated to take an image, and the image taking operation points, that is, each position P2, P3, . P6 is mark 6 is the traveling section SL
, S3 and the mark 6 is not imaged, or the imaging operation point, that is, each position P4. When P5 is a travel section S2 in which the mark 6 should not be imaged and some object is imaged, it is determined that the actual travel route of the unmanned vehicle 1 is abnormal. For this reason,
Even in the image-based unmanned vehicle 1 that employs the discretely arranged marks 6, if the marks 6 are missing, the unmanned vehicle 1 deviates from the driving route, or there is a risk of the road surface 4 being mistaken for the marks 6.
If there is dirt or an obstacle on the vehicle, it is possible to easily detect that the travel route is abnormal and to stop the unmanned vehicle 1.

Note that this invention is not limited to the above embodiments,
The present invention can be implemented as follows by changing a part of the structure as appropriate without departing from the spirit of the invention.

(1) In the embodiment described above, processing of the image 5 was explained when the front-to-back width of the area E imaged by the CCD camera 3 was relatively smaller than the arrangement interval of the marks 6. However, as shown in FIG. Area E imaged by camera 3
If the longitudinal width of the mark 6 is relatively larger than the arrangement interval of the marks 6, as shown in FIG. The image capturing timing at which the mark 6 should be imaged may be set for an image captured in a region closer to the unmanned vehicle 1 than the optical axis point Q.

In this case, even if at least one mark 6 is always imaged in the area E with a large longitudinal width according to each imaging timing, the mark 6 is not imaged in the area closer to the unmanned vehicle 1 than the optical axis point Q. Therefore, it is possible to set the traveling section where the mark 6 should be imaged next or the traveling section where the mark 6 should not be imaged next.

Furthermore, since the image that is captured in a large size as the mark 6 approaches the unmanned vehicle 1 is processed, the instruction information of the mark 6 can be processed easily and with high reliability.

(2) In the above embodiment, the CCD is used intermittently at predetermined intervals.
Although the imaging timing is set so that the imaging operation of the camera 3 is performed, it is also possible to configure the CCD camera 3 to perform the imaging operation continuously at all times without setting the imaging timing.

(3) In the embodiment described above, the CCD camera 3 was used as the imaging device, but other imaging devices may be used.

(4) In the above embodiment, the pixel configuration (resolution) of the image in the CCD camera 3 was set to 256 x 256 pixels, but it is not limited to this, and may be changed as appropriate, such as 512 x 512 pixels, 1024 x 1024 pixels, etc. You may.

[Effects of the Invention] As detailed above, according to the present invention, images of discretely arranged marks are captured, pixel signals in the captured images are processed to recognize the marks, and the driving route of an unmanned vehicle is determined. For image-based unmanned vehicles, if a mark is missing or the unmanned vehicle deviates from the driving route, or if there is dirt or obstacles on the road between each mark that could be mistaken for a mark, the driving route may be changed. It has the excellent effect of being able to easily detect that something is abnormal.

[Brief explanation of the drawing]

Fig. 1 is an electrical block circuit diagram of a travel control device showing one embodiment embodying the present invention, Fig. 2 is the same (side view of an unmanned vehicle, etc., Fig. 3 is a plan view of an unmanned vehicle, etc.), and Fig. 4 is a side view of an unmanned vehicle etc. The figure is a plan view of the mark, Figure 5 is an explanatory diagram to explain the image taken by the CCD camera, and Figures 6 (a) and 6 (b) are the areas imaged as the unmanned vehicle travels. 7(a) to 7(c) are explanatory diagrams for explaining images captured by the CCD camera corresponding to a series of imaging timings. Fig. 8 is a side view of an unmanned vehicle etc. showing another embodiment, and Fig. 9 is a plan view of the unmanned vehicle etc. An unmanned vehicle 1, a CCD camera 3 as an imaging device, a road surface 4,
Image 5, mark 6, driving route L1 driving section S1 to S3,
First position P1 to sixth position P6 as imaging operation points
゜

Claims (1)

[Scope of Claims] 1. Marks discretely placed on the road surface at predetermined intervals to instruct the unmanned vehicle to travel are sequentially imaged by an imaging device equipped on the unmanned vehicle as the unmanned vehicle travels, and the captured images are captured. In an image-based unmanned vehicle that processes pixel signals inside the vehicle to recognize marks and determine the driving route of the unmanned vehicle, when determining the driving route, the driving section where the mark is to be imaged next is determined. and a traveling section in which the mark should not be imaged, and the imaging device is operated to take an image from time to time while the unmanned vehicle is traveling, and the imaging operation point is in the traveling section where the mark is to be imaged. The actual driving route is determined to be abnormal if the mark is not imaged, or if the imaging operation point is in a driving section where the mark should not be imaged and some object is imaged. A method for detecting abnormalities in driving routes in image-based unmanned vehicles.