JPH01232401A - Image processor for moving vehicle - Google Patents

Abstract

PURPOSE:To accurately decide a travel path on a travel path surface at a high speed by providing a judging means which judges a normal travel path according to color information on an edge part. CONSTITUTION:An input means 100 reads the front path of an automobile by a color image input device such as a camera and a CCD scanner and inputs an image of the travel path surface by scanning lines with a color signal. An edge detecting means 102 detects the edge part of the image according to specific color information among pieces of input color information. The judging means 103 inputs pieces of color information on the primary colors R, G, and B which are normalized by a normalizing means 101, finds the upper-limit value and lower-limit value of each piece of color information from the mean value of normalized picture elements, and decides the normal travel path unless the signal level of the normalized color information on the edge part is within the range set determined by those upper-limit and lower-limit values. Consequently, the travel path such a s a line showing the travel path is accurately detected at a high speed without being affected by the reflection of light, etc., on the travel path.

Description

Detailed Description of the Invention (Industrial Application Field) The present invention relates to image processing of a moving vehicle that determines the direction of travel based on a travel route marked on a floor or road surface, such as a road edge or line. It is related to the device.

(Prior Art) With the recent progress in FA (factory automation) in factories, etc., automatic guided vehicles for transporting parts, assembled parts, etc. in the manufacturing process are being adopted in many factories. These automatic guided vehicles are controlled to optically read lines indicating travel routes marked on the floor of the factory, and to travel on the lines or between the lines.

(Problem to be solved by the invention) However, in conventional automatic guided vehicles, lines indicating the traveling route etc. are input as black and white image information, and they are binarized based on appropriate values. The driving route was determined by determining the edge parts such as lines based on the image data of the part (for example, Japanese Patent Laid-Open No. 60-15761
(See Publication No. 1). Therefore, in addition to the lines that indicate the running path on the floor, for example, there are areas where the surface of metal embedded in the floor is exposed, or where light from fluorescent lights, etc. from the ceiling is reflected on the floor. Because changes in brightness occur in the read image signal, the vehicle may mistakenly judge these metal parts, reflective surfaces, etc. as lines indicating the driving path, causing the vehicle to move in a direction different from the actual driving path. There was a problem.

The present invention has been made in view of the above-mentioned conventional example, in which image information indicating a running route is input as color information, edge portions of the image information are determined based on predetermined color information of the color information, and edge portions of the image information are determined based on predetermined color information of the color information. An object of the present invention is to provide an image processing device for a moving vehicle that can quickly and accurately determine a traveling route on a traveling road surface by determining whether or not it is a true traveling route based on color information.

(Means for Solving the Problems) In order to achieve the above object, an image processing device for a moving vehicle according to the present invention has the following configuration. That is, it is an image processing device for a moving vehicle that moves by detecting the position of a traveling route marked on a traveling road, and includes an input means for inputting an image of the traveling road surface as a color signal, and a color information from the input means. The vehicle includes a detecting means for detecting an edge portion of the image based on predetermined color information, and a determining means for determining a normal travel route based on the color information in the edge portion.

Further, the determination means includes a normalization means for normalizing the color information from the input means, and the determination means includes normalized R, G,
Input the color information of the three primary colors of B, calculate the upper and lower limits of each of the color information based on the average value of each normalized pixel on the scanning line of the image, and calculate the normalized value in the edge part. When the signal level of each color information is not within the range indicated by the upper limit value and the lower limit value, it is determined that the travel route is a normal travel route.

(Function) In the above configuration, an image of the road surface is input as a color signal using the input means. The edge portion of the image is detected based on predetermined color information among the input color information, and the driving route is determined to be a regular travel route based on the color information in this edge portion.

According to another claim, the normalized color information of the three primary colors RlG and B is input, and the color information is determined based on the average value of each normalized pixel on the scanning line of the image. The upper limit value and lower limit value are calculated, and when the signal level of each normalized color information in the edge part is not within the range indicated by the upper limit value and lower limit value, it is determined that the driving route is normal. Operate.

(Embodiments) Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Description of Functional Blocks (FIG. 1)] FIG. 1 is a functional block diagram showing a schematic functional configuration of a traveling path detection device in an automobile, which is a moving vehicle according to an embodiment.

In the figure, 100 reads the road ahead of the car using a color image input device such as a camera or CCD scanner, and converts it into color image signals of the three primary colors of R (red), G (green), and B (blue) for each scanning line. This is an input means for inputting information. 101 inputs and normalizes each of the RGB signals of the three primary colors, and generates an RGB signal (
This is a normalization means for obtaining rgb signals). Reference numeral 102 denotes an edge detection unit that detects an edge portion of the image on the scanning line based on a predetermined color signal among the RGB signals inputted by the input means 100, depending on whether the change level of the signal is larger than a predetermined value. In this embodiment, edges are detected based on red signal information. The selection of this color may be determined in advance based on the color of the traveling route, line, etc. to be read, and as will be described later, the color brane with the largest change is selected from among the color edge data of the read image signal. You may decide.

103 determines average values μr, μg, μb on the scanning line for each of the normalized r, g, and b signals, and determines upper and lower limit values to be compared based on these average values. Then, it is checked whether each of the normalized RGB signals of the pixel data of each color component corresponding to the edge portion inputted from the edge detection means 102 exists beyond the above-mentioned upper limit value and lower limit value. If there is no difference between the upper limit value and the lower limit value, it is determined that the route or line is the real one.

[Description of image processing device for detecting driving path in automobiles
(Fig. 2 to Fig. 7)] Fig. 2 is a diagram showing an example of the mounting position of a video camera for reading the driving route on a car. It may be provided on the upper side of the vehicle, that is, under the roof in the passenger compartment, or it may be provided in the front grille at the forefront of the vehicle body as shown at 21.

FIG. 3 is a block diagram showing a schematic configuration of an image processing device that is installed in an automobile and measures a running path according to the embodiment.

In the figure, 30 is a color video camera that reads the road, separates it into three primary color (RGB) signals, and outputs each as rask scan data.The RGB color signals from the color video camera 30 are used as images corresponding to each color. It is stored in the brain of the memory 31. That is, the image memory 31
has a brane corresponding to each of the RGB signals, for example, the R signal from the video camera 30 is connected to the brane 3 for R.
1-1. Brain 31-1 to 31 for each color
-3 has a memory capacity for at least one screen of the video camera 30. 32 is a normalization circuit that inputs and normalizes pixel information of a predetermined scanning line of each brane.

The position of the input scanning line and the number of scanning lines are determined depending on the complexity of the road on which the vehicle travels. The method for determining the position of scanning lines and the number of scanning lines is explained in the fourth section.
This will be explained with reference to Figures (A) to (D).

FIG. 4(A) to FIG. 4(D) all show image information for one screen read by the color video camera 30.
Figure (A) shows a case where the vehicle is located between the road edges 40 defining both sides of the road 41 and runs on the road 41, and 42 indicates the selected scanning line position on the screen. There is.

On the other hand, FIG. 4(B) shows a case where a car runs on a line 43 drawn on the road, and 44 indicates a selected scanning line. In this way, in the case of FIGS. 4A and 4B, no matter where the selected scanning line is located on the screen, the travel path can be determined based on the travel road edge 40 or line 43 without any particular problem. can be determined.

On the other hand, in the case of FIG. 4(C), there is an intersection of the traveling road 49 in front, so when the scanning line 46 is selected, the line 4
However, when the scanning line 47 is selected, the end of the road according to the line 45 cannot be detected, and the direction of travel of the car cannot be determined. Also, when the scanning line 48 is selected, the vehicle body is on the driving path 4.
9 can be detected. Further, in the case of FIG. 4(D), the car is traveling on the line 50 which is divided into a plurality of lines, and when the scanning lines 51 and 53 are selected, the car is traveling on the line 50 which is divided into a plurality of lines.
0 can be detected, but when the scanning line 52 is selected, a plurality of edge portions are detected, making it impossible to accurately determine the traveling direction, etc.

Therefore, it is necessary to determine the position and number of scanning lines to be selected based on the traveling direction, the shape of the travel path, etc. set in advance. For example, in the case of FIG. 4(C), it is detected that there is an intersection ahead by referring to at least three scanning lines 46 to 48, and after moving further forward, an edge portion formed by line 45 appears on scanning line 46. When the vehicle no longer exists, it may be detected that the vehicle has entered the intersection, and the direction of travel may be changed based on information indicating that the vehicle should proceed to the right or left.

Once the position and number of scanning lines to be referenced are determined based on the travel path, line shape, etc. in this way, the normalization circuit 32 reads out each of the RGB data of the pixels on the scanning line from the image memory 31 and normalizes them. become Next, this normalization will be explained.

Let the RGB components of pixel data on the i-th scanning line be RIJI G IJI B IJ. here,
i indicates that it is the i-th scanning line data, j is a number indicating the pixel position on the scanning line, and assuming that the number of pixels on one scanning line is N at maximum, 1≦j≦N. Now, let the normalized red, green, and blue color component values be r IJI gr
If g blj, then it is expressed as. Here, TIJ (stimulus sum)=RIJ+GIJ+BIJ.

This normalization eliminates the effects that appear on the image due to shadows, light reflections, and the like. In other words, if these influences are shown by C,
The brightness of each color is RIJ=C-RIJ', GIJ=C-GIJ', B
It is denoted by IJ=CB,,t. Here RIJ'
, G IJ', B1. ′ is the true image data. However, by normalizing as in the above equation, the C component, which is an influential component due to shadows and reflections, is reduced and eliminated because it is included in the denominator and numerator, and the light and shadow caused by this C are removed from the input image information. can eliminate the influence of

Next, the average value circuit 34 inputs these normalized data,
Based on these normalized values, the average value of each normalized color data on the scanning line is calculated. This calculation calculates the average value for each color as μrl+μg1. .mu.b, and the average value of TIJ is .mu.tI. Here, Nl is a value indicating the number of pixels on the scanning line other than the line. For example, in the case of the scanning line 42 in FIG. 4(A), the number of pixels corresponding to the portion indicated by 42' is shown. Therefore, this N1 is different for each scanning line, and the line 4
0 (in the case of FIG. 4(A)) becomes smaller as the width becomes thicker. This NI is determined based on location information obtained from map information given in advance, the distance traveled by the car, and the like.

When each average value is calculated and input in this way,
The upper and lower limit value determining unit 35 determines r, g, b based on these average values.
The upper limit value of pixel data of each color component (Ur+, Ug+
, Ub+ , Ut+ ) and the lower limit (flr+ , 1
2g1. I2b+, f2t+) is determined by the following formula.

Ur+ =gr+ +Cru (maximum value of red data) U
g l=μg++cgu (maximum value of green data) U
b I= u b l+ Cbu (Maximum value of blue data) Ut+ = μt+ +Ctu (Maximum value of T) I2
．． r, =μr+ -Cr+ (minimum value of red data)
Qg+ = μgl Cg+ (minimum value of green data) I2
b, =μb+Cbr (minimum value of blue data) β1.
=μt, -ct+(minimum value of T) Here, CXX is a positive value obtained empirically, and if the color of the floor or road surface is approximately equal to -, it will be small; If the color is non-uniform, it is set to a large value, and in this example, since the color of the floor surface is approximately uniform, it is set to about 0.03. However, Ctl has a value of about 10 to 20.

Reference numeral 33 denotes an edge detection unit that selects a predetermined frame of image data from among the RGB image data and detects an edge portion of the image data. For example, in the embodiment shown in FIG. 4(B), the line 43 is marked in blue, and the parts other than the line 43 are marked in light green. Figure 6 is a diagram showing the changes in pixel data at this time, where 1 scanning line corresponds to 2
It is composed of 56 pixels and 64 gradations. 61 is red (R)
62 shows the green edge data,
63 indicates blue edge data.

In this way, in this example, there is almost no difference between the color of the floor surface (light green) and the color of the line part (blue), so the red edge data changes the most, and the edge data is It is appropriate to perform part detection.

Further, when detecting an edge portion, a 3×3 Sobel operator for edge enhancement shown in FIG. 5 is applied to the red image data.

This provides information with emphasized edges. Of this edge-enhanced information, a portion larger than the threshold 1e11 is determined to be an edge.

This threshold value e1 is a value determined in consideration of the selected edge detection brain, the position of the scanning line, etc. This threshold e
1 may be determined empirically each time the environment such as the color or shape of the travel path changes, or may be provided corresponding to the position of the scanning line.

FIG. 7 is a diagram showing the relationship between the threshold value of the edge portion described above and the upper and lower limit values of the data of each color component.

Reference numeral 70 indicates image data whose edges have been enhanced based on red data, and an edge portion 75 is detected based on a threshold value e1. 7
1 to 74 respectively indicate the normalized values of red, green, and blue and the value of the stimulus sum T, and the relationship between each normalized value and stimulus sum value and the respective upper limit value and lower limit value is illustrated. .

Reference numeral 36 denotes a driving route determination unit that determines whether the edge is a real road edge or a line based on information from a comparison circuit 37 that checks pixel data of each color component in the edge portion detected by the edge detection unit 33. The comparison circuit 37 examines the pixel data of the point determined to be an edge portion by the edge detection section 33, and calculates the average value of each color component and stimulus sum at that point, and the upper and lower limit values determined by the upper and lower limit value determining section 35. It is checked whether the average value of each color at that point exceeds the upper limit or the lower limit, and if it exceeds it, it is determined that it is the edge of the road or a line.

When the true running road edge or line is detected by the running route determination unit 36 in this way, the edge position information is output to the running control unit 38 as a numerical value indicating, for example, in which pixel on the scanning line the edge is located. do.

When the travel control unit 38 inputs the position information of the road edge or line, it calculates the position and deviation of the direction while referring to the current display screen, the position of its scanning line, and the direction and speed of the current travel. , performs controls such as changing the driving direction of the vehicle.

[Operation description (Fig. 3, Fig. 8 to Fig. 12)] In the description of the image processing device related to Fig. 3 mentioned above, the normalization circuit 3
The explanation has been made assuming that each part of the second class is configured by hardware, but these are C
It may be controlled by software using the PU.

FIG. 8 is a flowchart showing control of the image processing device for road detection in the case of software control.

In step S1, color image data is input from the video camera 3o and stored in each brain of the image memory 31 for each color component. In step S2, a predetermined or empirically determined scanning line is selected, and each RGB data of the scanning line is input and normalized to rIJ, g.
IJ, b As well as finding IJ, find the stimulus sum TIJ. In step S4, the average value of each normalized color component and stimulus sum data (μr+, ugi, ul)+
) is determined, and the average value μtl of the stimulus sum of 71° is determined. Then, in step S5, upper limit values (Url, Ugl, Ubl, Ut) of image data of each color component are determined based on these average values.
l) and lower limit value (J2r+, 2gr, ffb+
, nt+).

In step S6, the color component of the image data for which an edge is to be determined is determined, and the color data is multiplied by, for example, a Sobel operator to determine a threshold value el for determining an edge. This threshold value el may be set in advance, or may be determined empirically based on the color of the image or the like.

In steps 87 to SIO, it is sequentially checked whether the value (le, 1) at each pixel position of the scanning line is larger than the threshold value el. Here, N indicates the total number of pixels on one scanning line, and is set to N=256 in this embodiment, for example.

If all the data on one scanning line is checked and there is no part corresponding to the edge portion, the process advances from step SIO to step S2, the next scanning line is selected, and the above-described operation is performed.

If a portion larger than the threshold el is detected in step S8, the process proceeds to step Sll, and the red data rlJ at the position indicated by j at that time is smaller than the minimum value (I2r+) of the red data found in step S5. or is greater than or equal to the maximum value (Urn). In this way, if it is between the minimum value and the maximum value, the process proceeds to step S12, where the green data is checked and the transition is made. However, if it is not within the range indicated by the minimum value and maximum value, the actual road edge or line traveling is detected. Assuming that it is a route, the process advances to step S15, and the value of j at that time is determined to be the travel route position.

In this way, similarly to step Sll, the green pixel data is checked in step S12, the blue pixel data is checked in step S13, and the stimulus sum TI is checked in step S14. Check whether it is less than the value. When all these conditions are satisfied, the process advances to step S9, where j is incremented by 1 and the process moves on to checking the next dot (pixel data). Then, in step S16, based on the value of j, the screen data, the scanning line position, etc., the running direction, speed, etc. of the automobile are determined, and driving control is performed to control the driving of the automobile.

FIG. 9 is a diagram showing an example of a screen showing a travel route read by a video camera in the embodiment.

Reference numeral 80 indicates a running route, which is a blue cloth tape pasted on a light green floor surface 81, and the car is controlled to run on this line 80.

FIG. 10 is a diagram showing pixel data on the scanning line 82 of the image data of FIG. 9.

83 is red (R) data, 84 is green (G) data, 8
5 indicates blue (B) data, and since there is almost no difference in color between the floor surface 81 and the line portion 80, the blue data hardly changes. Image data indicating edge information for each color exists between -100 and 100. 86
shows the normalized red data, whose average value is 0.3
It is 15. 87 indicates normalized green data and its average value is 0.335.88 indicates normalized blue data and its average value is 0.349. Furthermore, 89 indicates the value of the stimulus sum TI, whose average value is 104.520.
It becomes.

Here, an edge portion 91 larger than the threshold el is detected based on the red edge data 83, and the edge portion 91 is detected.
It is checked whether the normalized values of red, green, and blue corresponding to 1 and the stimulus sum are each greater than or equal to the set minimum value and less than the maximum value, and if this condition is not satisfied, 91 is the true travel route 9o. It is determined that

Similarly, FIG. 11 shows a case where a line 110 is marked in blue on a light green floor surface 111. Further, 112 is a reflected portion caused by the fluorescent light on the ceiling that is reflected on the floor surface 111. FIG. 12 is a diagram showing the image data of FIG.
5 to 117 indicate image data indicating red, green, and blue edge information, respectively, and 118 to 120 indicate red, green, and blue edge information, respectively.
The normalized image data of the blue data is shown, and 121 shows the value of the stimulus sum TI. The average value of data 118 is 01328, the average value of data 119 is 0.334, the average value of data 120 is o,339, and the average value of stimulus sum 121 is 113.719. Here the red data is 115
Based on the threshold value e.

The edge portion 122 is detected by
Since each normalized value and stimulus sum value in 22 are less than the maximum value set for each color data and not more than the minimum value, 122
is detected as the true travel route 114 shown in FIG.

Reference numeral 123 indicates an area affected by the reflected area 112 of the fluorescent light on the ceiling that is reflected on the floor 111. In this way, the reflected portion 112 of the ceiling fluorescent light reflected on the floor surface 111
The influence of image data 1 showing edge information in each color
15 to 117, the edge portion 122 can be immediately detected as a candidate for the running line, and only this part is confirmed as a running line using color information, so the running line can be detected in real time compared to the case where only color information is used. can be detected.

[Description of Other Embodiments (FIGS. 13 and 14)] FIG. 13 is a diagram showing another embodiment, in which a metal portion 132 is exposed on a flesh-colored floor surface 131. There is. L
30 indicates a line portion marked in red, and 133 indicates a referenced scanning line.

FIG. 14 is a diagram showing data on the scanning line 133 of the screen shown in FIG. 13. 140-142 are red respectively,
Indicates image data indicating green and blue edge information, 143
-145 each indicate normalized values of red, green, and blue image data, and 146 indicates the stimulation TIJ. As in the previous embodiment, each of these data has the pixel position on the scanning line on the horizontal axis, and the image data 140 to 142 are all within the range from -100 to 100. data 143
The average value of is 0.365, the average value of data 144 is 0.3
09, the average value of data 145 is 0.325, 14 for stimulation
The average value of 6 is 117.121.

In the figure, 147 indicates a portion where data changes due to the metal frame 132, and 148 indicates a portion where data changes due to the regular line 130 defining the travel route.

In this way, in order to avoid determining that metal embedded in the floor or the like is a road surface, upper and lower limits for each color component may be determined individually and used as criteria for determination. Similarly, if it is acceptable to consider metal parts as actual travel routes, upper and lower limits may be set so that metal parts can also be detected.

In addition, in the above-mentioned embodiment, the surface for detecting edges is a specific brane (here, red), but the stimulus sum value T
This can be done based on IJ. This is because if an edge is detected on a screen of a certain color, that data will always be reflected on the surface showing the stimulation T, as shown in FIGS. 10 and 12. However, in this case, when multiplying by the 3×3 Sobel operator shown in FIG. 5, the lines i+1 . Even in i-1, it is necessary to find the stimulation TIJ for each pixel.

In this example, the detection of the end of a line in a moving vehicle moving along a line marked on the floor in a factory or the like was explained, but the detection is not limited to this, and the detection of the center line on a normal road surface is not limited to this. It can also be applied to vehicles that move by detecting lines that define lanes, speed limits, and other characters written on the road. In this case, it goes without saying that the number and position of the selected scanning lines, the upper and lower limits of each color, etc. are changed as appropriate depending on the condition of the road on which the vehicle is traveling.

As explained above, according to this embodiment, it is possible to accurately determine the edges of lines, etc. marked on the floor, road, etc., so it is possible to accurately guide moving vehicles such as cars along these lines, etc. Can be done.

In addition, edge parts such as lines are detected based on specific color data, and the detected edge points are compared with color information to determine the driving route. This has the effect of reducing the time required to detect the travel route and allowing edge determination in real time.

(Effects of the Invention) As described above, according to the present invention, there is an effect that a traveling route such as a line indicating a traveling route can be detected accurately and at high speed without being affected by reflection of light on the traveling route.

[Brief explanation of the drawing]

Fig. 1 is a functional block diagram of the image processing device in the embodiment, Fig. 2 is a diagram showing an example of the mounting position of the video camera in the automobile of the embodiment, and Fig. 3 is a schematic configuration of the image processing device in the automobile of the embodiment. 4 (A) to (D) are diagrams showing an example of a running route and line image data that defines the running route, and scanning line positions in the image data. Figure 6 shows the Sobel operator, Figure 6 shows RGB image data used for edge determination, Figure 7 shows an example of edge detection, normalized color data, and upper and lower limits of stimulus sum values. 8 is a flowchart showing the driving route determination process in the driving control of the automobile according to the embodiment, FIG. 9 is a diagram showing an example of image data of the floor surface and the lines marked thereon, and FIG. Figure 11 shows an example of each color component data created based on the data on the scanning line of the image data shown in Figure 11. Figure 11 shows image data of the floor surface and line with fluorescent lamp light reflected on the floor surface. Figure 12 is a diagram showing an example of each color component data created based on the data on the scanning line of the image data shown in Figure 11. Figure 13 is a diagram showing an example of metal on the floor surface of another example. Figure 14 shows an example of image data for the floor and lines when the image data is embedded, and Figure 14 shows an example of each color component data created based on the data on the scanning line of the image data shown in Figure 13. FIG. In the figure, 30... Color video camera, 31... Image memory, 32... Normalization circuit, 33... Edge detection section, 34... Average value circuit, 35... Upper and lower limit value determination. Part, 36... Traveling route determination part, 37... Comparison circuit, 3
8... Traveling control unit, 100... Input means, 101.
... Normalization means, 102 ... Edge detection means, 103
...It is a means of judgment. too 101 Figure 1: (A) (B) (C)
(D) Figure 4 Figure 9 Figure 10 Figure 11 0 50 too 150
200 250 Fig. 12 + 30 Fig. 13 Fig. 14

Claims (2)

[Claims] (1) An image processing device for a moving vehicle that moves by detecting the position of a travel route marked on a travel road, which includes an input means for inputting an image of the travel road surface as a color signal, and color information from the input means. An image of a moving vehicle, comprising: a detection means for detecting an edge portion of the image based on predetermined color information; and a determination means for determining a regular travel route based on the color information in the edge portion. Processing equipment. (2) The determination means includes a normalization means for normalizing the color information from the input means, and the determination means includes normalized R, G,
Input the color information of the three primary colors of B, calculate the upper and lower limits of each of the color information based on the normalized average value of each pixel on the scanning line of the image, and calculate the normalized value in the edge part. According to claim 1, when the signal level of each color information is not within the range indicated by the upper limit value and the lower limit value, it is determined that the driving route is a normal driving route. image processing device for moving vehicles.