JPH02116909A - Image processor for moving car - Google Patents

Abstract

PURPOSE:To easily extract an area at the time of running in a city area in which there are many obstacles by photographing the front image and the upper image of a running road, determining a candidate of a running route, based on the upper image, and determining the running route, based on the front image of the candidate. CONSTITUTION:When there is no obstacle and a road is not complicated, the road end and the center line is recognized surely by holding a camera 2 in the horizontal direction, therefore, there is no problem is determining a course. In the case of encountering a Y-shaped road, the road end is not detected easily. In such a case, by turning the camera to the upper part, fetching the upper image and detecting an electric wire, etc. from this image, the extended direction of a running road is detected from the direction of this electric wire. As for a fact of utilizing the electric wire, since it is an existing equipment, it is unnecessary to lay newly a marker and a beacon. In such a way, the running road is determined surely.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to an image processing device for a moving vehicle that captures an image of a traveling path and determines the traveling direction.

(Prior Art) In a mobile vehicle that recognizes the outside world and performs autonomous driving control, a tangled edge or the like is generally detected and running control is performed along this tangled edge. For example, JP-A-61-100
No. 898 sequentially specifies passing singular points, sets a planned route to a destination, and guides a vehicle along the route. Also, JP-A No. 61-240307
As shown in No. 1, an image is captured and the travel of the vehicle is controlled based on the line at infinity in the image.

(Problems to be Solved by the Invention) In this prior art, the direction of the traveling road is assumed to be known, such as when the moving vehicle faces a direction substantially along the traveling road. Furthermore, even if the direction of the road is unknown, the driving environment is assumed to be a fairly simple model (for example, indoors, a road exclusively for automobiles, etc.). Conventionally, the traveling direction of a moving vehicle could be determined based on such a premise or process.

However, unlike roads exclusively for automobiles, on real roads, especially in urban areas, the surrounding environment is complex, with vehicles parked on the roadside, and it is difficult to determine which direction the driving route is from the current location of the moving vehicle. In many cases, it is impossible to detect whether the vehicle is moving forward, and therefore it is difficult to determine the direction of travel.

Therefore, the present invention was proposed in order to eliminate the drawbacks of the above-mentioned conventional example, and its purpose is to
An object of the present invention is to propose an image processing device for a moving vehicle that can easily extract the driving path environment and determine the direction of travel.

(Means and operations for achieving the object) The configuration of the image processing device for a moving vehicle according to the present invention for achieving the above object includes: an imaging means for taking a forward image and an upper image of a running path; The present invention is characterized by comprising a candidate determining means for determining a driving route candidate based on a forward image of the candidate, and a route determining means for determining a driving route based on a forward image of the candidate.

(Embodiment) An embodiment of the image processing apparatus of the present invention will be described below with reference to the accompanying drawings.

Figure 1 shows a moving vehicle 1 equipped with the image processing device of this embodiment.
FIG. 1A is a side view of the moving vehicle 1, and FIG. 1B is an external view of the moving vehicle 1.
The figure is a rear view. This mobile vehicle 1 is equipped with wheels 5 for running and steering, and casters 4. 2 is a video camera, which is directed to the mobile vehicle 1 by a support arm 3. This camera is supported rotatably around an axis 6 of an arm 3. Therefore, the camera 2 can photograph the outside world at any angular position ranging from an angular position of −α degrees below the horizontal direction to an angle of vertical (90 degrees) above. That is, if it is determined that the direction of travel cannot be determined based on the image in front of the vehicle, an image of the upper part is taken to recognize electric wires, street lights, etc., and this information is used to determine the direction of travel.

FIG. 2 shows an image captured by the camera 2 while the vehicle was traveling on a highway. In the figure, 22 indicates an imaging range. 20 is an infinity line (horizontal line). In the image, unless there is an obstacle in front, the extension line of the left binding end 23 and the extension line of the right binding end 25 intersect, and the intersection point P (vanishing point = vanishing point)
int ) will now lie on the horizontal line 2o. horizontal line 2
0 may be an actual horizontal line on the earth's surface, or may be a virtual electric wire drawn horizontally to the vanishing point P. In the example in Figure 2, there are no obstacles and the road is not complicated, so the tangled ends,
Since you can definitely recognize the center line, there will be no problem deciding on your course.

However, when a Y-junction is encountered, as shown in FIG. 3, it is not easy to detect the tangled ends. This is because there are many parked vehicles at intersections such as Y-junctions. In such a case, the camera 2 is directed upward to capture an upper image to obtain an image as shown in FIG. Electric wires and the like are detected from this image, and the direction in which the travel path extends is determined from the direction of the electric wires. This is because the electric wires are often laid along the direction of the running route. The biggest advantage of using electric wires is that they are existing equipment and there is no need to install new signs or beacons. In particular, although beacons and the like can be installed on main roads, they cannot be installed on branch roads, so route determination based on electric wire detection according to this embodiment is effective. That is, at least you can move from an alley to a major road with your own blade.

FIG. 5 is a block diagram of an image processing device according to an embodiment. Reference numeral 10 denotes a frame buffer, in which images taken by the camera 2 are stored as RGB signals. Rotation drive unit 11
rotates the support arm 3 to change the angle of the camera 2. Reference numeral 12 denotes a tied edge detection unit that detects a tied edge from the contents of the frame buffer 10. During normal running, this detection unit 12 detects a tied edge. Reference numeral 16 denotes a determination unit that determines whether the tangled end detected by the detection unit 12 is a normal one. command. When the camera points upward, the upward image is captured into the frame buffer 10,
From this image, the driving route candidate determination unit 13 determines driving route candidates. When a candidate is determined, the camera 2 is directed forward again with respect to the rotation drive unit, and a front image of the determined travel route candidate is taken and captured into the frame buffer 10. The candidate determination unit 14 detects a road area from this forward image,
Determine the direction of travel. In this case, even if the road environment is complex, the candidates are narrowed down, so the accuracy of candidate determination is improved and the extraction of the road area is relatively easy.

FIG. 6 is a flowchart of a control procedure according to the embodiment, which is executed by the CPU 15. In step S2, an image of the front of the moving vehicle is captured. In step S4, the tangled end detection section 12 is caused to detect a tangled end. In step S6, it is determined whether the detected bound end is a regular one. If it is normal, the process proceeds to step S22, and the course direction is determined based on the detected tangled edge.

If the tangled edge cannot be regarded as an extension direction of the travel path, the camera 2 is directed upward in step S8, and an upper image (for example, FIG. 4) is captured in step S10. In step S12, electric wires are detected from this upper image. In images taken with the camera pointing upward, the wires are generally visible against the sky. Therefore, it is extremely easy to detect and recognize a black wire in a white or blue background image.
A well-known method of finding 8-connected contours using a mask consisting of 3×3 small pixel areas can be applied. If the electric wire is recognized, in step S14, a plurality of running route candidates are determined based on the direction of the electric wire. The reason for selecting multiple candidates is that electric wires are generally separated in many cases.

Of the multiple candidates, it is reasonable to consider that the most likely one is the one with the longest distance among the electric wires recognized as substantially straight. This is because the branch line is short. Even if it is necessary to detect power lines at an intersection, the image of the power lines is continuous even if they are bent, so the direction in which the road is curved can be detected.

In step S16, the camera 2 is directed in the direction of this candidate (forward direction), and an image thereof is obtained in step S18. In step S20, the road area of this image is recognized. Conventionally, if a tangled edge or the like cannot be recognized, it is necessary to recognize the road area in all directions, but in this embodiment, the direction is narrowed down to the minimum number of candidate directions, so the recognition accuracy of the road area is improved. As mentioned above.

Note that objects that can be used to estimate the traveling direction from the upper image are not limited to electric wires, but include, for example, street lights during night driving, sodium lamps in tunnels, traffic lights at intersections, and the like. Therefore, the upper image is not limited to the vertically upward image, and may be any negative value within the range where these objects can be photographed as images.

(Effects of the Invention) As described above, the configuration of the image processing device for a moving vehicle according to the present invention includes an imaging means that takes a forward image and an upper image of a traveling route, and a driving route candidate that determines a traveling route candidate based on the upper image. The present invention is characterized by comprising a candidate determining means for determining a driving route, and a route determining means for determining a driving route based on a front image of the candidate.

In the upper image, for example, electric wires and the like are included in a simple background image, so it is extremely easy to recognize the direction in which the electric wires and the like extend. Therefore, it is possible to infer the running route direction from this extension direction, narrow down the candidates, and reliably determine the running route. It is particularly effective for extracting road areas when driving in urban areas where there are many obstacles in front of the road.

[Brief explanation of the drawing]

Figures 1A and 1B are a side view and a rear view, respectively, showing the appearance of a moving vehicle used in an embodiment of the present invention, and Figure 2 illustrates an image obtained when driving on a highway in the embodiment. Figure 3 is a diagram showing a Y-junction as an example of a complex road environment; Figure 4 is a diagram showing an example of an upper image obtained by the image processing device of the embodiment; FIG. 6 is a block diagram showing the configuration of the image processing apparatus according to the embodiment. FIG. 6 is a flowchart relating to the control procedure of the embodiment. In the figure, 1... moving vehicle, 2... camera, 3... camera support arm, 4... caster, 5... wheel, 10...
Frame buffer, 11... Rotation drive unit, 12...
Folded edge detection unit, 13... Candidate determination unit, 14... Candidate determination unit, 15... CPU, 16... Tangled edge determination unit, 20
. . . horizontal line, 22 . . . imaging range, 23 . . . pressure path end, 25 . Figure 2. Figure 1A Figure 8 Figure 4 Figure 6

Claims (1)

[Claims] (1) An imaging device that takes a forward image and an upper image of the driving route; a candidate determining device that determines a driving route candidate based on the upper image; and a route that determines a driving route based on the forward image of the candidate. An image processing device for a moving vehicle, comprising: determining means.