JP2998965B2 - Travel control device for mobile vehicles - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a traveling control device for a mobile vehicle that controls traveling autonomously, and more particularly, to a traveling control device using two different sensor systems, an image sensor system and a distance sensor system. The present invention relates to an improvement for improving recognition accuracy by fusing obstacle recognition by fuzzy inference.

(Prior Art) In recent years, various mobile vehicles that travel while autonomously avoiding obstacles have been proposed.

In such autonomous traveling control, first, the position, size, shape, and the like of an obstacle are recognized, and an avoidance action corresponding to each obstacle is performed based on the information.

As an autonomous vehicle technology, for example, Japanese Patent Application Laid-Open No. 64-26913
There is a number. In this technology, when an obstacle is detected, an area including the obstacle is set along one edge of the travel path as an obstacle area zone, and the remaining travel path is set as a virtual travel path of the own vehicle. The running control of the own vehicle is performed on the road. Fuzzy inference is used for traveling control on this virtual traveling road.

In the technology of the Japanese Patent Application Laid-Open No.
An ultrasonic sensor and a laser radar are used. The image from the camera is used for the setting of the virtual traveling path and the traveling control on the virtual traveling path. On the other hand, outputs from the ultrasonic sensor and the laser radar are used for fail-safe control.

(Problems to be Solved by the Invention) That is, in JP-A-64-26913, although fuzzy inference is used for traveling control on a virtual traveling road, another control system for fail-safe is provided in parallel. In an emergency, the travel control of the fuzzy inference is stopped, and the fail-safe control is activated instead.

A visual system such as a camera has a night vision region, and it is necessary to use an ultrasonic sensor to supplement the night vision region. However, it is extremely difficult to evaluate the visual system such as a camera and the auditory system such as an ultrasonic sensor using the same criterion because the sensor output itself is different. For this reason, as disclosed in Japanese Patent Application Laid-Open No. 64-26913, even if the external world recognition is performed by mixing the visual system and the auditory system, the external world recognition will not be the result of switching between the two systems. It has to rely on only one of them.

Therefore, the present invention obtains the respective external recognition results from the different external recognition signals, and performs the running control based on the two different external recognition results without performing the switching driving control as in the related art. An object of the present invention is to propose a traveling control device for a mobile vehicle capable of performing obstacle recognition by fusing two different external recognition results by fuzzy inference.

(Means and Actions for Achieving the Object) To achieve the above object, a traveling control device for a mobile vehicle according to the present invention includes: a first signal generation unit that generates a first signal representing an image of an external world; A first recognition unit that receives the first signal and performs external world recognition, and generates a second signal indicating an actual distance to an external object independently of the first signal generation unit. A second signal generation unit, a second recognition unit that receives the second signal and performs external recognition, and a fuzzy inference that applies the same fuzzy rule to each of the recognition results of the two recognition units. It is provided with inference means for performing processing and inferring the traveling trajectory of the own vehicle, and traveling control means for controlling traveling of the own vehicle based on the inference result.

The first signal is an image signal of the entire outside world, and the second signal is a distance signal to a specific object. Each signal generation means should be set according to the characteristics of the signal. Therefore, according to claim 2 which is a preferred aspect of the present invention, the first
Signal generating means is configured to generate the first signal representing an image having a middle or long distance viewing angle, and the second signal generating means determines a distance to an object at a short distance. The second signal shown is set to be generated.

According to a third aspect of the present invention, which is a preferred aspect of the present invention, if each of the signal generating means is set as described above, the first recognizing means determines that the distance to the obstacle is equal to the travel distance in the image. It is converted into information on the position of the center of gravity of the possible area. Finding the position of the center of gravity of the travelable area in the image can be performed at a higher speed than directly calculating the distance to the obstacle, and fuzzy inference can be performed at a higher speed.

(Embodiment) The principle configuration of a preferred embodiment of the present invention will be described below with reference to the attached FIG. The basic configuration of the embodiment is
As shown in FIG. 1, first recognition means for inputting a first signal for external world recognition and performing external world recognition, and a signal for external world recognition, wherein the first signal is Is a second recognizing means for inputting second signals having different characteristics and recognizing the outside world, inference means for performing fuzzy inference processing on the recognition results of the two recognizing means, and Running control means for controlling the running of the vehicle.

According to the fuzzy rule, the traveling trajectory is inferred by fusing each recognition result based on at least two or more different signals, so that flexible judgment and smooth traveling are realized.

Hereinafter, the present invention will be described in more detail using a specific embodiment applied to a motor-driven vehicle with reference to FIGS.

<Hardware Configuration> FIG. 2 is a block diagram showing a hardware configuration of the travel control device of the present embodiment. FIG. 3A is a plan view showing the positional relationship between the vehicle 100 equipped with the travel control device of the present embodiment and each sensor, and FIG. 3B is a side view of the same.

In these figures, the visual sensor system comprises a camera 1. The auditory sensor system comprises eight ultrasonic sensors 8 (8a _RF , 8b _RF , 8a _LF , 8b _LF , 8a _RS , 8b _RS , 8a _LS , 8
b _LS ). These sensor outputs are sent to a data processing computer 4 via an I / O controller 6 for fuzzy inference.

The image of the outside world is captured from the color CCD camera 1,
It is stored in the frame memory 2. A region in which the computer 4 can travel is extracted from the image in the memory 2 with the support of the processor 5 dedicated to image processing. As described below,
From the image of the travelable area, the position of the center of gravity and the displacement (EL, ET described later) of the center of gravity position from the reference are detected.
The computer 4 was a commercially available personal computer, and the image processing processor 5 was a data flow driven pipeline processor LSI board for speeding up the processing.

That is, the computer 4 infers and generates a traveling trajectory from the eight sensor outputs of the auditory system and images of the visual CCD camera according to fuzzy inference. Inferred running track is output to the motor controller 11, the controller 11 drives the left and right motors 10 _R, 10 _L.

The touch sensor 7 (7a to 7d) marked "T" is mounted on each of the four corners of the vehicle 100, and infrared sensors marked "I" are mounted on the front and rear. These sensors are for emergencies, and their outputs are directly input to the motor controller 11.

FIG. 3A shows an arrangement of eight ultrasonic sensors. 8a _RF , 8
b The _RF sensor is a group that covers the right area at the front of the vehicle. The sensors of 8a _LF and 8b _LF are sets that cover the left area at the front of the vehicle body. The sensor of 8a _RS and 8b _RS is a group that covers the right side area of the vehicle body side. The sensors of 8a _LS and 8b _LS are sets that cover the left side area of the vehicle body side. Where, for reference,
The subscripts R for right, L for left, F for front and S for side. The reason why one sensor is used as one set is to improve accuracy. Therefore, a set of sensors that cover the right area at the front of the vehicle body
The outputs of 8a _RF and 8b _RF are S _RF , the sensors 8a _LF and 8b _LF output the left side area of the front of the vehicle body are S _LF , and the sensors 8a _RS and 8b _RS output the right side area of the vehicle body are S _LS. , sensor 8a _LS responsible for the left side area of the vehicle body side portion, the output of 8b _LS and S _RS. As is well known, the distance to the target is known from the arrival time of the echo signal detected by the ultrasonic sensor. That is, S _RF , S _LF ,
S _LS and S _RS are distance data to the obstacle.

FIG. 3B illustrates the difference between the imaging range 15 of the camera 1 and the detection range 16 of the ultrasonic sensor 8 described above. In order to show the difference in the detection range in front of the vehicle, FIG.
Of the eight ultrasonic sensors, only the front one is shown. In FIG. 3B, the imaging range of the camera is 15, and the long and middle distances are the imaging ranges. The detection range 16 of the ultrasonic sensor 8 at the front is mainly a middle distance area and a short distance area. Thus, the detection range in this embodiment is 3
The reason for dividing into two is as follows. That is, the visual sensor has the advantage of high resolution, but requires a high degree of processing to recognize an obstacle, and has the disadvantage of having a night vision region. On the other hand, the auditory system sensor has the disadvantages of low resolution and difficulty in detecting a long distance, but has the advantage of being able to detect a failure with high accuracy at a short distance. It is the arrangement of each sensor of the present embodiment that makes use of these advantages and compensates for the disadvantages.

<Obstacle Recognition> The feature of the present embodiment is a process for inferring the traveling trajectory by judging the output of the external recognition result from the above-described camera sensor and ultrasonic sensor based on the same standard. Is realized by integrating these recognition results into a fuzzy inference device. In the device of this embodiment, no knowledge is given about the driving environment. In other words, this travel control device has only low-level functions, and the device itself makes decisions based on what it actually "sees" and "hears". This can be better understood with reference to FIG.

FIG. 4 is a block diagram showing the processing functions in the computer 4 of the control device.

In the figure, the fuzzy inference unit 25 infers a traveling trajectory based on the recognition result from the sensor, and calculates a steering change angle ΔST and a traveling speed increase / decrease ΔV from the current position.
Is output to the motor driver 26 in the motor controller 11. The motor driver 26 calculates the rotation speeds N _L and N _{R of} the left and right motors 10 _L and 10 _R based on ΔST and ΔV, and outputs them to these motors. The rotation speed of the motor (vehicle speed V) is detected by a sensor (not shown), fed back to the inference unit 25, and used for updating the vehicle position.

The inputs to the fuzzy inference unit 25 are distance data S _RF , S _LF , S obtained from the four sets of ultrasonic sensors via the short distance recognition unit 24.
_LS, (transverse displacement of the center of gravity) S _RS and data EL that medium long-distance recognition unit 27 detects, ET (longitudinal displacement of the center of gravity), and A (the area of the travelable area).

The data processing in the middle and long distance recognition unit 27 will be described. The detection unit 27 extracts a “travelable area” from the image obtained by the visual sensor 1, detects the center of gravity of the area, and further detects displacements EL and ET of the center of gravity from the reference.

 The detection of the displacements EL and ET will be described.

FIG. 5 shows an image captured from the camera 1 while the moving vehicle is traveling along the corridor 30 where the obstacle 31 is placed. The travelable road may be detected by the following procedure.

First, a part that is on the same plane as the plane to which the vehicle belongs and that has a similar hue or the like is extracted. This is because a wall, an obstacle, and the like often have different color components from the traveling road and the like. At this time, select a color edge having little noise such as a color edge. However, assuming that the color components of the traveling path may be similar to the color components of the wall or the like, a portion that intersects perpendicularly with the traveling path is excluded from the travelable area. In other words, the vertically intersecting portion becomes the boundary of the travelable area.

In the initial state, it is assumed that the travelable area always exists before the own vehicle. Thereafter, in the process of continuing the search for the travelable area, the newly searched travelable area is merged, and the travelable area is updated. However, in this search, since the amount of information is small in a distant image, only a region below the infinity line is set as a search region.

FIG. 6 shows an example of a travelable area when there is no obstacle. The running path in this example is a straight corridor. The travelable area of such a traveling path has a triangular shape as shown in FIG. The reason why the travelable area is set to be a triangle in the example of FIG. 6 is that the travel road is represented by a straight road,
This is because they can be approximated. In FIG. 6, X ₀
Y ₀ indicates the x- and y-direction coordinates of the vanishing point at infinity, and X ₁
Is the position of the own vehicle in the x direction. Here, the origin of the space of the frame memory is indicated by 0 in FIG. G is the center of gravity of the travelable area indicated by a triangle. When there are no obstacles on the travel path, the area of the travelable area becomes the maximum.
FIG. 7 shows a case where an obstacle appears at a long distance on this travel path. According to the above-described procedure for searching for a runnable area, the area occupied by the obstacle is deleted from the triangular runnable area in FIG. 6, and the area is reduced. In this case, the obstacle ahead appears first above the travelable area, and then, as the moving vehicle approaches the obstacle, the image of the obstacle moves downward while increasing in size. In view of this, from the viewpoint of the movement of the center of gravity of the travelable area, the movement of the center of gravity G in the image moves from above to below as the obstacle approaches. In other words, the position of the center of gravity G is a measure of the degree of approach to the obstacle. Therefore, the speed control of the moving vehicle can be used for the position G of the center of gravity.

The features of this method are as follows. Assuming a case where a moving vehicle approaches an obstacle at a constant speed, for example,
The area of an image of a distant obstacle is small, and the area of an image of a nearby object is large. On the other hand, similarly, the width of the distant traveling road is narrow, and the width of the near traveling road is large. Therefore, as described above, if the center of gravity is the center of gravity of the image of the travelable area in which obstacles and the like have been removed from the image of the travel path, the vertical position of the center of gravity depends on the distance to the obstacle. Conventionally, when recognizing the distance from an image to an obstacle, the image is converted into a three-dimensional (or two-dimensional space) space. According to the method of the present embodiment, this conversion processing is not required, and the speed control becomes faster.

A description will be given of where the reference position R is set. As is well known, the center of gravity of the triangle is defined by the line segment connecting the vertex (the vanishing point in the example of FIG. 6) and the midpoint of the base (X _{1 in} the example of FIG. 6) being two points from the vertex. It is in a position proportionally distributed to one. Further, in the state of FIG. 6, that is, the position of the center of gravity G in the absence of obstacles (the G _0), in the y-axis direction, G ₀ = 1 / 3Y ₀
In the position. If an obstacle appears, then the center of gravity
G _X always falls below this 2 / 3Y ₀ position. That is, represented the center of gravity position at any point in G _i, a _{0 <G i <1 / 3Y} 0 ‥‥‥ (1). Therefore, the reference position R is set on a line segment connecting the vertex of the triangle and the midpoint of the base, and R is set in the y coordinate system.
Take 1 / 6Y ₀ position. Raise or lower the speed of the transport vehicle in this embodiment is to think on whether the center of gravity position G _i where to find by the R in the reference. That is, for example, G _i is if PB (positive big = quite large) than R, for example, so as to increase the speed.

Here, the vertical displacement ET of the center of gravity is defined. In this embodiment, ET = G _iy −R _y ‥‥‥ (2). Here, G _iy is y direction component of G _i, the R _y is the y direction component of R.

This will be described more specifically with reference to FIG. The figure shows the vehicle
A state in which the vehicle 100 travels on the traveling path 30 while avoiding the obstacle 31 will be described. The running position changes with abcd,
The image at each time is indicated by ABCD. Then, on the right, with respect to the reference position R, or the center of gravity G _i is how changes are indicated. When there is no obstacle, the image to should the kind of D is obtained, then the center of gravity G _i are matched to G _0. When the image is shifted to the state of A, the center of gravity G _i is lowered than G _0. When the image is shifted to the state of B, the center of gravity G _i is further lowered, it will fall to a position below the reference R.

Next, steering control will be described. Steering control is performed based on the lateral displacement of the center of gravity G _i.
In this embodiment, since the mounting direction of the camera 1 matches the traveling direction of the vehicle, the line segment lowered from the vanishing point matches the traveling direction of the vehicle. Then, when there is no obstacle, such as the image D of FIG. 8, the center of gravity G _i of the image are Notsu on this segment. If, when the obstacle appears to the left, the center of gravity G _i of the image moves to the right from the line. If the obstacle appears to the right, the center of gravity G _i are moved to the left. On the other hand, the area of the image of the obstacle at a distance is small, and the area of the image at the near point is large, so when the obstacle is at a distance, the displacement of the center of gravity to the left and right is small, and the obstacle is near. The displacement of the center of gravity to the left and right is large. In vehicles operated by humans, changes in steering when obstacles are far away are small, and changes in steering when obstacles are close are large. If the displacement information is used as it is as the steering control information, the steering control of the autonomous driving can be automated.

Here, the lateral displacement EL of the center of gravity is defined. In this embodiment, the _{EL = G ix -R x ‥‥‥ (} 3). Here, G _ix is x direction component of G _i, R _x is the x direction component of R.

In FIG. 8, with a change in the image, showing the manner in which the center of gravity G _i is displaced to the left and right.

In the calculation of the travelable area in the equation (1), when the travel path is a corridor or the like, there are many cases where walls have irregularities, so it is better to consider the margin. Further, when the center of gravity approaches 1 / 3Y ₀ or around 0, the error greatly affects the control. Therefore, it is better to consider the distribution of the projection of the travelable area on the x-axis.

The above is the outline of the speed control and the steering control using the displacement of the position of the center of gravity.

In the system shown in FIG. 4, the speed control and the steering control using the displacement of the position of the center of gravity described above are applied to a long-distance region. As shown in FIG. 4, at the middle distance, the visual sensor 1 and the ultrasonic sensor 8 are used at the same time. In this case, the information is mainly from the visual sensor, and the information is from the ultrasonic sensor. That is, even if the ultrasonic sensor detects the presence of an obstacle in the middle range, it does not use it directly for running control, but suggests the presence of the obstacle to a recognition system using a visual sensor. stop. This will be described in detail below.

FIG. 9A shows an image captured by the camera 1 when the obstacle is not in the middle distance range. The image has an area of 256 pixels × 256 pixels. At this time, the search for the travelable area is performed by:
The “scan” of the image is performed from the lower side to the upper side of the image to detect a boundary of a travelable area (for example, a boundary between a traveling road and a wall). Then, the middle and long distance recognition unit 27 normally
The above scanning is performed every 12 pixels. If an obstacle is found within the middle distance, the distance calculation unit 20 informs the window setting unit 21 to that effect. Then, the window setting unit 21 sets a window as a search area in the entire image as shown in FIG. 9B. Then, the above-described “scanning” is performed at the highest resolution (in this example, for each pixel) within the set window.

When there is no obstacle within the middle distance, even if the obstacle is searched for by coarse scanning as shown in FIG. 9A, the accuracy is sufficient, and a high search speed is secured. On the other hand, when the ultrasonic sensor finds an obstacle in the middle range, instead of narrowing the search area as shown in FIG. 9B, the accuracy is improved by increasing the amount of information. This is because the intermediate range requires obstacle recognition with higher accuracy than the long range.

Thus, in the middle distance range, the obstacle is recognized with high accuracy, and the control amounts EL and ET according to the recognition result are obtained.

<Recognition fusion results with Fuajiruru> Thus, the distance data S _RF to the obstacle from the short distance recognition unit 24, S _LF, S _LS, is S _RS, also, from the middle long distance recognition unit 27, the travelable area Lateral displacement EL and longitudinal displacement ET of the center of gravity
The area A is input to the fuzzy inference unit 25. As described above, the distance data S _RF , S _LF , S _LS , and S _RS , the lateral displacement EL, the longitudinal displacement ET, and the area A are variables having different properties. However, in this embodiment, the same fuzzy rule is used. By applying inference of the traveling trajectory by applying, flexible coupling of these heterogeneous quantities was made possible.

FIG. 10A, FIG. 10B, FIG. 10C, FIG. 11A, and FIG. 11B summarize the inference algorithms in steering and speed control, respectively. Here, the speed control is expressed as a change ΔV in speed with respect to the current speed V, and the steering control is expressed as a change ΔST in angle with respect to the current steering angle ST.

In these tables, with respect to the antecedent part, for example, ET, there is no vertical displacement of ET = ZO center of gravity ET = PS Slight displacement in the positive direction ET = PB Large displacement in the positive direction ET = NS Slight displacement in the negative direction ET = NB A membership function that has a large displacement in the negative direction is selected. Also,
For the consequent part, for example, ΔV, ΔV = ZO No speed change ΔV = PS Slight speed increase ΔV = PB Overspeed increase ΔV = NS Slight speed decrease ΔV = NB The function has been chosen. The inference of the antecedent part used the so-called minimax method in fuzzy logic, and the center of gravity value was used to calculate the result of the consequent part.
In addition, a membership function having a triangular distribution was selected.

If the tables in FIG. 10A, FIG. 10B, FIG. 10C, FIG. 11A, and FIG. 11B are expressed as rules, as an example, if (V is PM, EL is PM) then (ΔST is PM) if (ET is PM, A is PM, V is PS｝ then (ΔV is PM) if (S _RF is PS, S _RS is PM) then (ΔST is PS) if (S _LF is PS, S _LS is PB) then (ΔST is NS) The steering change angle ΔST is to change the steering to the left if POSITIVE and to the right if NEGATIVE.

According to the rules shown in the tables of FIGS. 11A and 11B, ΔV is determined in consideration of the area A of the drivable area, depending on how the obstacle appears, the drivable area is, for example, vertical. This is because, in such a case, the position of the center of gravity is located on the upper side, and if the area is not taken into account, control is performed to increase the speed.

<Effects of Embodiment> According to the embodiment described above, the recognition of an obstacle obtained from the auditory system sensor and the recognition of the obstacle obtained from the visual system sensor are flexibly combined by the fuzzy rule. As a result, the traveling trajectory is inferred, so that obstacle-free traveling without error is realized. Conventionally, recognition results by different sensor systems are processed only by switching control as in the above-mentioned Japanese Patent Application Laid-Open No. 64-26913.
Although the running at the time of switching is discontinuous, the running becomes smooth due to the gradual combination of the recognition results as in the present embodiment.

: At medium and long distances, the displacement of the position of the center of gravity of the travelable area is used for steering control and speed control. As a result, inference can be performed only by data processing in the image space, which contributes to speeding up.

: In the middle distance range, when the recognition of an obstacle was obtained by the ultrasonic sensor, the processing speed was secured and the recognition accuracy was improved by increasing the resolution of the search area of the travelable area. By using one recognition result for the other recognition process without relying solely on the determination of one sensor system, the accuracy of the determination of the presence or absence of an obstacle could be improved.

The present invention is applicable not only to a motor-driven moving vehicle but also to a vehicle equipped with a general internal combustion engine.

In the above embodiment, two sensors, the ultrasonic sensor and the visual sensor, are used. However, the number of sensors is not limited to two, and may be three or more.

(Effects of the Invention) As described above, the traveling control device of the present invention provides an image signal (first signal) that enables overall recognition of the outside world.
And certain parts of the outside world (obstacles of high importance to recognize)
The same fuzzy rule is applied to two different recognition results with respect to the outside world, that is, a distance signal (second signal) to the same. As a result, the traveling trajectory is inferred by fusing the two recognition results having different properties, so that flexible judgment and smooth traveling can be realized.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of the present invention, FIG. 2 is a diagram showing a hardware configuration of an apparatus to which the present invention is applied, and FIG. 3A is an arrangement of each sensor used in the device of FIG. FIG. 3B is a diagram for explaining the division of the detection area of each sensor, FIG. 4 is a block diagram for explaining the processing in the computer 4 of FIG. 2 by functional division, and FIG. 5 is an embodiment. 6 and 7 are diagrams illustrating the principle of running control using the center of gravity G in the present embodiment. FIG. 8 is a diagram illustrating an image and a moving image as the vehicle travels. FIG. 9A and FIG. 9B are diagrams for explaining how the position of the center of gravity of the drivable area changes, and FIGS. 9A and 9B show scanning densities for searching for obstacles when there is no obstacle within a short distance and when there is a certain obstacle, respectively. FIG. 10A, FIG. 10B, FIG. 10C, FIG. 11A, and FIG. It is a diagram to explain the rules of the definitive Fuaji inference. In the figure, 1 ... camera, 2 ... frame memory, 3 ... monitor,
4 Data processing computer, 5 Image processing processor, 6 I / O controller, 7a to 7d Touch sensor, 8, 8a _RF , 8b _RF , 8a _LF , 8b _LF , 8a _RS , 8b _RS , 8a _LS , 8b _LS ……
Ultrasonic sensors, 9a, 9b ...... infrared sensor, 10 _R, 10 _L ...... motor, 11 ...... motor controller, 15 ...... vision zone, 16
... hearing area, 20 ... distance calculation unit, 21 ... window setting unit, 22 long distance recognition unit, 23 ... middle distance recognition unit, 24 ... short distance recognition unit, 25 ... fuzzy inference unit, 30 ... ... runway, 31 ...
... obstacles, 100 ... vehicles.

Claims (3)

(57) [Claims] A first signal generating means for generating a first signal representing an image of the outside world; a first recognizing means for receiving the first signal and performing outside world recognition; Independent of the signal generating means, a second signal generating means for generating a second signal indicating an actual distance to an external object, and a second recognition for inputting the second signal and performing external recognition Means, a fuzzy inference process of applying the same fuzzy rule to each of the recognition results of the two recognition means, and inference means for inferring the traveling trajectory of the own vehicle. A travel control device for a mobile vehicle, comprising: travel control means for controlling travel. 2. The method according to claim 1, wherein the first signal generation unit is configured to generate the first signal representing an image having a middle or long viewing angle. The travel control device for a mobile vehicle according to claim 1, wherein the second signal indicating the distance to an object at a distance is set to be generated. 3. The first recognizing means searches for a travelable area in an image obtained by the first signal generating means, finds a center of gravity of the runnable area, and outputs it to the inference means. The travel control device for a mobile vehicle according to any one of claims 1 to 2, wherein: