JP7301035B2 - Obstacle detection system - Google Patents

Description

The present invention relates to an obstacle detection system for detecting an object on a planned travel track of a vehicle.

2. Description of the Related Art A technology has been developed to ensure safety by mounting detectors such as cameras and object sensors on vehicles such as automobiles and railroads and detecting in advance objects on the planned travel track of the vehicle. The following patent documents 1 and 2 are known as prior art related to the same technique.

Patent Literature 1 describes controlling the direction of a radar or a camera based on navigation system information and driver line-of-sight information. According to the document, ``The control device 11 detects at least the presence or absence of an intersecting road or the number of lanes as a road configuration ahead of the traveling path of the own vehicle or on a predicted traveling trajectory based on information acquired from the navigation device 17 or the like. , the current position and traveling trajectory (or moving direction) of the own vehicle, and further according to the line-of-sight direction of the driver detected by the line-of-sight detection device 18, the detection direction of the radar 13 and the imaging direction of the camera 14. (see abstract).

Patent Document 2 describes using lane information to control the direction of headlights. The document states that "the illumination direction control unit determines a target swivel angle according to the detected lane extension direction, and controls a swivel actuator to bring the illumination direction of light from the vehicle headlamp closer to the target swivel angle. tracking control” (see abstract).

Japanese Patent Application Laid-Open No. 2005-231450 JP 2012-196982 A

The obstacle detection system not only detects the presence or absence of objects in front, but also grasps the planned travel route that the vehicle is scheduled to travel from now on, and grasps the relative position between the planned travel route and the object. , it can be determined whether the vehicle and the object will collide.

On the other hand, when an image system sensor detects a distant object, so-called zoom photography is used, in which the focal length of the lens is lengthened so that the image on the image sensor can be photographed with a size greater than or equal to a predetermined size. Although it is desirable to increase the size of the image pickup device in accordance with the focal length, it may be difficult to achieve this due to factors such as cost, size, and presence or absence of the image pickup device. If the image sensor size is small, the angle of view will inevitably become narrow. Narrowing the angle of view narrows the imaging range, so there is a possibility that the planned travel track will not be captured in the image. Also, even if part of the planned travel track is captured, there is a possibility that the range desired to be detected may not be sufficiently included.

Even if the planned travel trajectory is captured, there is a possibility that objects cannot be detected at night due to insufficient illuminance and contrast. In particular, when obtaining the illuminance of a distant object with the lighting equipment installed in the vehicle, the brightness of the road surface parallel to the light distribution direction may be lower than that of the three-dimensional object directly facing the light distribution direction. have a nature. As a result, it is expected that the detection performance of the planned travel track will be degraded.

SUMMARY OF THE INVENTION The present invention has been made in view of the problems described above, and is capable of reliably detecting a planned travel track and reliably detecting obstacles on the planned travel track even with a narrow-angle camera. It is an object of the present invention to provide an obstacle detection system capable of

The obstacle detection system according to the present invention roughly adjusts the shooting direction of the remote monitoring camera based on the planned traveling trajectory obtained by the near-field monitoring, and calculates the three-dimensional coordinates of the planned traveling trajectory from the distant image after the coarse adjustment. to further fine-tune the shooting direction of the remote surveillance camera.

According to the obstacle detection system according to the present invention, even with a narrow angle of view camera, it is possible to reliably capture the planned travel trajectory and detect obstacles within the detection range.

1 is a schematic configuration diagram of an obstacle detection system 1 according to Embodiment 1; FIG. FIG. 4 is a diagram schematically showing a situation in which the stereo camera 100 cannot photograph the vicinity of the vehicle; FIG. 3 is a schematic top view showing an example of a planned travel track of a vehicle; An example of the shooting range of each of the stereo camera 100 and the camera 400 is shown. 4 shows an image after roughly adjusting the imaging direction of the stereo camera 100. FIG. FIG. 10 is a diagram showing how a point of interest is determined after roughly adjusting the imaging direction of the stereo camera 100; The planned travel trajectory is a straight line, and the image shows a distant area beyond the detection range of the stereo camera 100 . Similar to FIG. 7, this is an example in which a distant object beyond the detection range is shown. This is an example of an uphill slope in front of the stereo camera 100 and a downhill slope ahead. This is an example where the farthest position of the detection range cannot be seen due to a utility pole or other structure. This is an example where the station is on the left of the planned travel track. 6 shows an image obtained by cutting out a portion of FIG. 5 photographed by the stereo camera 100. FIG. It shows how the position captured by the stereo camera 100 changes gradually. It shows how the position captured by the stereo camera 100 changes gradually. It shows how the position captured by the stereo camera 100 changes gradually. An example of a rotating mechanism 200 is shown. The state of the stereo camera 100 when the shooting direction is changed is shown. FIG. 17 is another representation of FIG. 2 is a functional block diagram of stereo camera 100. FIG. 5 is a graph showing exposure and sensitivity settings of the stereo camera 100 when detecting a three-dimensional object; 4 is a graph showing exposure and sensitivity settings of stereo camera 100 when image blurring is unlikely to occur. FIG. 10 is a diagram showing image acquisition timing when the frame rate is sufficiently longer than the exposure time; FIG. 10 shows the processing timing when combining frame images; FIG. 4 is an example of one of left and right images of the stereo camera 100. FIG. It is an example of an image captured by a varifocal camera. 2 is a configuration diagram of an obstacle detection system 1 according to Embodiment 2. FIG. It is an example of an image when the optical axis is directed downward. FIG. 11 is a configuration diagram of an obstacle detection system 1 according to Embodiment 3;

<Embodiment 1>
FIG. 1 is a schematic configuration diagram of an obstacle detection system 1 according to Embodiment 1 of the present invention. The obstacle detection system 1 is a system that is mounted on a vehicle to detect obstacles on a planned travel track of the vehicle. In this embodiment, an example in which the obstacle detection system 1 is mounted on a train will be described. The obstacle detection system 1 includes a stereo camera 100 (distant monitoring camera), a rotating mechanism 200 , a light projector 300 , a camera 400 (near monitoring section), and a controller 500 .

The stereo camera 100 is for detecting a distant obstacle by photographing a distant object of the vehicle. Stereo camera 100 is composed of two cameras. The rotation mechanism 200 changes the shooting direction of each camera of the stereo camera 100 . The light projector 300 irradiates light far away from the vehicle. Camera 400 photographs the vicinity of the vehicle (a position closer than the distance photographed by stereo camera 100). Controller 500 controls camera 400 and stereo camera 100 .

The stereo camera 100 sends the shooting direction information a based on the camera optical axis to the controller 500 . Similarly, the proximity camera 400 sends to the controller 500 the coordinate information b of the road surface or railroad track shown on the captured image. Based on the information a and b, the controller 500 sends a drive signal to the rotation mechanism 200 to orient the stereo camera 100 in a predetermined photographing direction. The rotation mechanism 200 directs the imaging direction of the stereo camera 100 in that direction. The controller 500 sends light distribution direction information to the light projector 300 based on the information a and b. The light projector 300 directs the light distribution direction in that direction.

FIG. 2 is a diagram schematically showing a situation in which the stereo camera 100 cannot photograph the vicinity of the vehicle. The stereo camera 100 has a lens with a long focal length and a narrow angle of view in order to detect distant obstacles. In general, the front monitoring camera is installed with the optical axis almost horizontal, so the narrower the angle of view as shown in Fig. 2, the farther the distance c at which the road surface starts to appear in the image, and the more the road surface appears in the image shown by d. The range of the road surface directly under or near the lens that does not have a lens is expanded.

FIG. 3 is a schematic top view showing an example of the planned travel track of the vehicle. As shown in FIG. 3, the smaller the radius of curvature of the planned travel track (for example, a road or railway), the greater the angle of view required. In other words, the longer the focal length, the less roads and railroads appear on the screen.

It is conceivable that the planned travel track such as a road or railroad track cannot be detected from the image, that it cannot be properly detected due to insufficient light at night, or that the planned travel track is not shown in the image. If there are multiple causes, it is preferable to take measures starting from the cause that can be reliably overcome. In this case, it is desirable to first operate the obstacle detection system 1 so that the planned travel route is captured in the image, and then perform exposure adjustment and image processing in order to reliably detect the planned travel route. Therefore, an operation example for reliably displaying the planned travel track in the image will be described below.

FIG. 4 shows an example of the imaging ranges of the stereo camera 100 and the camera 400, respectively. The thick black solid line on the outside is the shooting range of the camera 400 , and the thick black solid line on the inside is the shooting range of the right camera of the stereo camera 100 . The white dotted line on the image of the camera 400 is an area in which it is guaranteed that the planned travel trajectory will be captured regardless of the gradient or degree of curve of the road surface. This is possible by securing a sufficient angle of view of the camera 400 and appropriately setting the Y coordinate of the white dotted line. The controller 500 searches for the planned travel trajectory on the white dotted line and obtains the X coordinate (white circle in FIG. 4) of the planned travel trajectory. The controller 500 has information corresponding to the imaging direction of the stereo camera 100 (for example, the coordinates on the screen of the camera 400 at the center of the screen of the stereo camera 100), and the imaging of the stereo camera 100 is performed based on the screen center information and the coordinates of the white circle. Determine direction and amount. This corresponds to rough adjustment of the photographing direction of the stereo camera 100 based on the image of the camera 400 .

FIG. 5 shows an image after roughly adjusting the imaging direction of the stereo camera 100 . The rough adjustment here means to roughly determine the photographing direction before performing the fine adjustment described below.

FIG. 6 is a diagram showing how the point of interest is determined after the shooting direction of the stereo camera 100 is roughly adjusted. After the controller 500 roughly adjusts the photographing direction of the stereo camera 100, the stereo camera 100 measures the three-dimensional coordinates of the planned travel track displayed on the screen, and based on the result, determines a point of interest (see FIG. 6) as a target of the photographing direction. black circle). The fine adjustment here means to further finely adjust the shooting direction roughly determined by the rough adjustment, and the purpose is to determine the point of interest.

7 to 11 show examples of points of interest. In both figures, points of interest are indicated by black circles. Examples of points of interest will be described below with reference to these figures.

In FIG. 7, the planned travel trajectory is a straight line, and the distance beyond the detection range of the stereo camera 100 (the maximum distance at which an object can be detected from the image captured by the stereo camera 100 is defined in the specifications of the stereo camera 100). This is the image shown. In this case, the stereo camera 100 takes the farthest position in the detection range of the stereo camera 100 as the point of interest.

FIG. 8 is an example in which a distant object beyond the detection range is shown, as is the case with FIG. The stereo camera 100 takes the farthest position of the detection range as the focus point in FIG. 8 as well.

FIG. 9 shows an example in which an uphill is in front of the stereo camera 100 and a downhill is ahead. In FIG. 9, the farthest position of the detection range cannot be seen because the detection range slopes downward from the middle. In this case, the stereo camera 100 takes the farthest position that can be seen as the point of interest.

FIG. 10 shows an example in which the farthest position of the detection range cannot be seen due to a utility pole or other structure. In this case, the stereo camera 100 takes the farthest position that can be seen (the farthest position in the range not blocked by the structure) as the point of interest.

FIG. 11 shows an example where a station is on the left of the planned travel track. In this case, the stereo camera 100 focuses on a point a predetermined distance ahead of the vehicle stop position. For example, the point of interest can be a point that is ahead of the stop position and is closest to the photographable range. However, as long as it is ahead of the stop position, it does not necessarily have to be the nearest photographable point.

As described with reference to FIGS. 7 to 11, the stereo camera 100 detects (a) the farthest point of the detection specification of the stereo camera 100 on the planned travel route, and (b) the line of sight when the planned travel route is blocked by terrain or buildings. Either the farthest possible point or (c) a predetermined distance ahead of the planned stop position of the own vehicle on the planned travel track is set as the point of interest.

12 to 15 show the process of finely adjusting the photographing direction of the stereo camera 100. FIG. The fine adjustment operation will be described with reference to FIGS. 12 to 15. FIG.

FIG. 12 shows an image obtained by cutting out a portion captured by the stereo camera 100 in FIG. The white circles are the coordinates of the planned travel track described with reference to FIG. A black circle is a point of interest. The controller 500 first calculates a parallax image from the cropped images of the left and right cameras, and then calculates the three-dimensional coordinates of the planned travel trajectory, thereby determining the point of interest.

The vertical dotted line is the coordinate target of the point of interest. If the coordinates of the point of interest are in the area to the right of the dotted line on the right, the imaging direction of the stereo camera 100 is shifted to the right, or the cutout position is shifted to the right. If the coordinates of the point of interest are in the area to the left of the dotted line on the left, the imaging direction of the stereo camera 100 is shifted to the left, or the cutout position is shifted to the left. Arrows shown in FIG. 12 indicate shift amounts. These screen shifts may be performed according to either the shooting direction or the clipping area. However, in order to avoid image smearing, it is desirable to mechanically change the orientation of the camera at a speed that does not exceed the upper limit set according to the distance to the point of interest, and to adjust the shortfall within the clipping range. That is, the farther the point of interest is, the slower the rotational speed toward the camera is, and if the rotational speed reaches the upper threshold, the screen is shifted by finely adjusting the cropping range after reducing the rotational speed to the upper threshold or less.

13 to 15 show how the position captured by the stereo camera 100 changes gradually. Since it is assumed that the clipped image will be monitored by a person concerned, the clipping position is set so that the clipping area also changes smoothly. Therefore, the scenery gradually moves as shown in FIGS. 13 and 14, and when the point of interest comes to the dotted line position as shown in FIG. 15, the clipping ends.

FIG. 16 shows an example of the rotation mechanism 200. As shown in FIG. As shown in FIG. 16, the rotation mechanism 200 has a vertical rotation axis (black circle in FIG. 16) for each of the two left and right cameras. As another method, there is a method of providing a rotating shaft at the center of the housing as indicated by the white circle in FIG. Requires space. By rotating each camera, a narrow space can be realized even with a long baseline stereo camera as the distance increases, and the present embodiment can also be applied to a stereo camera with an independent left and right structure.

FIG. 17 shows the state of the stereo camera 100 when the shooting direction is changed.

FIG. 18 is another representation of FIG. FIG. 17 can be drawn as shown in FIG. 18 from a different point of view. From FIG. 18, it can be seen that the base line length and the position of the camera in the optical axis direction change depending on the shooting direction.

A change in the position along the optical axis changes the magnification of the image. Assuming that the magnification is M, the imaging direction is θ, and the baseline length is L, the magnification M of the distance X of the target object is expressed by the following equation. Since the object distance is unknown before obtaining the parallax, the magnification is calculated with X=“maximum detection distance”.

M (left) = (XL / 2 · sin θ) / X (Equation 1)
M (right) = (X + L / 2 · sin θ) / X (Equation 2)

Since the closer the object is, the size of the object differs between the left and right images, and the distance measurement error increases. Therefore, in order to set the detection range on the near side, it is necessary to consider the shooting direction. By correcting the left and right images with the magnification calculated by the above formula, it is possible to calculate the parallax of a distant object near the maximum detection distance in the conventional manner.

FIG. 19 is a functional block diagram of stereo camera 100. As shown in FIG. Stereo camera 100 includes imaging sensors 101 and 102 , distortion correction units 103 and 104 , image processing units 105 and 106 , parallax detection unit 107 , object detection unit 108 and calculation unit 109 . Image sensors 101 and 102, distortion correction units 103 and 104, and image processing units 105 and 106 perform processing corresponding to the left camera and right camera, respectively.

Based on the angle information a sent from the controller 500, the calculation unit 109 sends cut-out information c of the captured image to the sensors 101 and 102, sends rotated angle information d to the distortion correction units 103 and 104, and outputs baseline length information. e is sent to the object detection unit 108 . The sensors 101 and 102 perform synchronous photography at predetermined intervals, cut out images based on the sent cut-out information c, and send the cut-out images to the distortion correction units 103 and 104 . Distortion correction units 103 and 104 correct the sent images based on the optical distortion correction information acquired in advance and the sent rotated angle information d, and the corrected images are sent to image processing units 105 and 106 and parallax detection units. Send to 107. The image processing units 105 and 106 perform image processing such as contrast correction and noise reduction on the corrected images as necessary, and send the processed images to the parallax detection unit 107 . The parallax detection unit 107 generates parallax images from the corrected images sent from the image processing units 105 and 106, and generates parallax images from the processed images sent from the image processing units 105 and 106 as necessary. Each parallax image thus obtained is sent to the object detection unit 108 . The object detection unit 108 obtains three-dimensional information of each pixel based on the sent parallax image and base line length information e, and detects an object or a planned traveling trajectory.

As an alternative method to the above, when converting the calculated parallax into the distance of the object without performing magnification correction on the image, distance correction based on the left and right base line length difference and measurement based on the pixel coordinates of interest are performed. Distance correction may be performed. In this case, the error depending on the three-dimensional object distance is very small, and accurate distance measurement is possible. In this case, instead of the rotated angle information d in FIG.

The light projector 300 normally determines the light distribution direction so that the target point is illuminated. However, when searching for a planned travel track on the white dotted line of the camera 400 as shown in FIG. After that, the controller 500 calculates three-dimensional data along the planned travel trajectory with the white circle as a starting point, and shifts the light distribution direction farther. The light distribution angle and light intensity may be set based on the obtained image. That is, when it is determined from the pixel values of the acquired image that the illumination light is insufficient, the controller 500 increases the illuminance at the point of interest by at least either increasing the light distribution intensity or narrowing the light distribution angle. Conversely, when the illuminance of the point of interest is sufficiently obtained, the controller 500 widens the light distribution angle to the same extent as the angle of view of the camera.

One of the factors for determining whether the detected three-dimensional object is an obstacle is whether the three-dimensional object is in the vicinity of the planned travel route or whether it is approaching the vicinity of the planned travel route. Relative position information between is important. Therefore, in order to detect obstacles, it is important to detect the planned travel track. Sufficient illuminance can be obtained from both the three-dimensional object and the road surface during the day, so detection performance can be ensured. Since the illuminance is insufficient at night, it is necessary to improve the illuminance by the light projector 300 in addition to increasing the exposure time and imaging sensitivity. When the light projector 300 is mounted on a vehicle, it is difficult to obtain contrast on a road surface that is substantially parallel to the light projection direction than on a three-dimensional object in which the light projection direction and the irradiation surface face each other. . In the following, a device for that purpose will be described.

FIG. 20 is a graph showing exposure and sensitivity settings of stereo camera 100 when detecting a three-dimensional object. The horizontal axis of the graph is exposure time, and the vertical axis is sensitivity. During the day, set the exposure time to short and the sensitivity to low. The brightness is ensured by increasing the exposure time and sensitivity according to the pixel value of the acquired image. If the sensitivity is too high, noise will occur, so the upper limit g is set. If the exposure time is too long, image blurring occurs, so the upper limit values h1 to h2 are set according to the vehicle speed. This operation is used for normal three-dimensional object detection, and is processed by the path leading to sensor 101, distortion correcting section 103, and parallax detecting section 107 in FIG. be.

FIG. 21 is a graph showing exposure and sensitivity settings of stereo camera 100 when image blurring is unlikely to occur. If the planned travel track is a straight line or a curve with a constant radius of curvature, such as a white line or rail (i.e., if the radius of curvature is within a range where the radius of curvature varies, the same applies hereinafter), there is a feature that image blurring is less likely to occur. For example, in an image such as that shown in FIG. 7, the position of the planned travel track on the image is relatively stable over time. In this case, the image used for detecting the planned travel trajectory can be acquired with the exposure time sufficiently longer than the image for detecting the three-dimensional object, as shown in FIG.

FIG. 22 is a diagram showing image acquisition timing when the frame rate is sufficiently longer than the exposure time. In this case, as shown in FIG. 22, the image for detecting the planned trajectory is obtained (h3) after the image for detecting the three-dimensional object is obtained (h2). If the frame rate is not long enough to acquire the image for the planned travel trajectory, the image processing units 105 and 106 store the previous frame image and add it to the current frame image to equivalently increase the exposure amount. be able to.

FIG. 23 shows the processing timing when combining frame images. The upper part of FIG. 23 shows the exposure times and frame rates of the sensors 101 and 102 . The middle part of FIG. 23 shows the image storage timing by the image processing units 105 and 106 . The lower part of FIG. 23 shows the timing of addition processing of the current frame and the previous frame. In particular, when the planned travel trajectory is a straight line or a curve with a constant radius of curvature, image blurring is less likely to occur even if the frame images are added, so it is useful to improve the contrast by adding.

As another method when the frame rate is sufficiently long compared to the exposure time, it is also possible to perform viewpoint movement processing for replacing one of the left and right images of the stereo camera 100 with the other viewpoint and add it to the other image. In this case as well, an effect equivalent to that of frame addition can be obtained.

Although the method for enhancing the image contrast of the planned travel track has been described above, the image brightness can also be enhanced by a similar method. If at least one of the contrast and brightness can be increased, it can be said that the planned travel trajectory can be detected more reliably.

FIG. 24 is an example of one of left and right images of stereo camera 100 . FIG. 24 shows the right image when FIG. 7 is the left image. By performing the viewpoint movement processing on the road surface plane, it is possible to match the planned traveling trajectory on the road surface plane with the left image.

In the above description, it has been described that the imaging direction of the stereo camera 100 is moved in the horizontal direction, but the present invention is not limited to this. If there is an inclination, the point of interest moves up and down, so the clipping position may also be shifted up and down. Alternatively, the shooting direction may be mechanically shifted.

In the above description, it has been described that the three-dimensional coordinates of the planned travel trajectory are obtained from the parallax image in order to determine the point of interest, but the present invention is not limited to this. If the width of the planned travel track is known, the known road width may be used to calculate the three-dimensional coordinates of the planned travel track from the camera image/focal length/pixel pitch. In this case, the three-dimensional coordinates of the planned travel trajectory can be obtained without necessarily using a stereo camera.

In this embodiment, the total angle of view of the camera 400 is approximately 70 degrees. This value includes the necessary angle of view of 67 degrees estimated from the relationship between the acceleration and the radius of curvature published by the railway company and the mounting accuracy. Even if other sensors are used to monitor the vicinity of the vehicle, a left and right detection angle of view of 70 to 75 degrees is sufficient. Therefore, the detection angle of view of the camera 400 should be about 65 degrees to 75 degrees.

Although one light projector 300 is used in this embodiment, the same effect can be obtained by using a plurality of light projectors 300 . If there are more than one, they can be shared between distant projection and near projection. Conventional equipment can also be used as the projector for the vicinity.

<Embodiment 1: Summary>
The obstacle detection system 1 according to the first embodiment determines the direction of the stereo camera 100 (distant monitoring camera) and the light projector 300 based on the information of the near monitoring means that can reliably capture the planned traveling track such as the white line and the rail. After rough adjustment, the directions of the stereo camera 100 and the light projector 300 are finely adjusted based on the three-dimensional image acquired by the stereo camera 100 . As a result, the planned travel trajectory can be stably captured while ensuring an image magnification sufficient for object detection. That is, a highly reliable obstacle detection system 1 can be provided.

The obstacle detection system 1 according to the first embodiment acquires the image used for detecting the planned travel trajectory and the image used for detecting the three-dimensional object by changing the exposure conditions, or Image accuracy is ensured by frame addition or addition after viewpoint movement. As a result, even when sufficient illuminance and contrast cannot be ensured, such as at night, the brightness and contrast of the planned travel route can be emphasized, and the planned travel route can be detected with high accuracy.

<Embodiment 2>
In Embodiment 1, a wide-angle camera is used as a means for detecting the planned travel track in the vicinity. Either one of the left and right cameras of the stereo camera 100 can play this role. That is, either one of the left and right cameras is configured with a variable focus camera.

FIG. 25 is an example of an image captured by a varifocal camera. When shooting on the wide side (short focal length and wide angle) of the varifocal camera of the stereo camera 100, an image as shown in FIG. 25 is acquired, and when shooting on the tele side (long focal length and narrow angle), an image such as that shown in FIG. 12 is obtained. get.

FIG. 26 is a configuration diagram of the obstacle detection system 1 according to the second embodiment. A variable focus camera zoom mechanism is contained within the stereo camera 100 . The stereo camera 100 sets either one of the left and right cameras to the wide-angle side to shoot an image, and sends shooting direction information a based on the camera optical axis to the controller 500 . The stereo camera 100 sends to the controller 500 the coordinate information b of the road surface or railroad track appearing on the photographed image. Based on the information a and b, the controller 500 sends to the rotation mechanism 200 a driving signal for directing the stereo camera 100 in a predetermined photographing direction, and also requests that the varifocal camera be changed to the tele side. The controller 500 sends light distribution direction information to the light projector 300 based on the information a and b. Since other processing processes are the same as those of the first embodiment, description thereof is omitted.

In the present embodiment, the purpose of the wide-side image is to obtain the two-dimensional coordinates of the planned travel track, but the present embodiment is not limited to this. That is, three-dimensional information can be obtained by enlarging the image on the wide side or reducing the image on the tele side and stereoscopically viewing the left and right overlapping areas. In reality, the accuracy of distortion correction and parallelization affects the performance and accuracy of obtaining three-dimensional information even in wide-angle images, so there is a possibility that distortion correction processing will be required for both wide-angle and telephoto.

<Embodiment 3>
In Embodiment 1, a wide-angle camera is used as a means for detecting the planned travel track in the vicinity. At least one of the left and right cameras of the stereo camera 100 can play this role. That is, by rotating one of the left and right cameras in the pitch direction (vertical direction) to direct the optical axis downward, the planned travel trajectory can be reliably captured within the angle of view.

FIG. 27 is an image example when the optical axis is directed downward. In this case, an image such as the dotted line in FIG. 27 can be obtained, and an image such as the solid line in FIG. 27 is acquired based on the planned traveling track coordinates in the upper part of the image.

FIG. 28 is a configuration diagram of the obstacle detection system 1 according to the third embodiment. The pitch rotation unit 600 rotates either the left or right camera of the stereo camera 100 in the pitch direction. The stereo camera 100 shoots an image with its optical axis set to the lower side in the pitch direction, and sends shooting direction information a based on the camera optical axis to the controller 500 . Unlike the first embodiment, the shooting direction information a includes pitch direction information in addition to yaw direction information. The stereo camera 100 sends to the controller 500 the coordinate information b of the road surface or railroad track appearing on the photographed image. The controller 500 sends a drive signal to the rotation mechanism 200 and the pitch rotation section 600 to orient the stereo camera 100 in a predetermined shooting direction based on the information a and b. The controller 500 sends light distribution direction information to the light projector 300 based on the information a and b. Since other processing processes are the same as those of the first embodiment, description thereof is omitted.

In Embodiment 3, the stereo camera 100 has an operation mode for photographing a distant object and an operation mode for photographing a near object. The pitch rotation unit 600 tilts the camera optical axis downward in the close-up mode as compared to the long-distance shooting mode. It is not necessary to tilt the camera strictly along the vertical direction.

<Regarding Modifications of the Present Invention>
The present invention is not limited to the embodiments described above, and includes various modifications. For example, the above-described embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Also, part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

In the above embodiment, the camera 400 or the stereo camera 100 is used as means for detecting the state of the vicinity of the vehicle, but the state of the vicinity of the vehicle may be detected by means other than the camera. For example, it is conceivable to use a ranging sensor such as Lidar. A plurality of detection means may be used together.

Each processing unit (computing unit 109, distortion correction units 103 and 104, image processing units 105 and 106, parallax detection unit 107, object detection unit 108) provided in stereo camera 100 and controller 500 are circuit devices implementing these functions. It can also be configured by hardware such as the above, or can be configured by an arithmetic unit executing software implementing these functions. It may be implemented by combining hardware and software.

In the above embodiment, each processing unit (calculation unit 109, distortion correction units 103 and 104, image processing units 105 and 106, parallax detection unit 107, object detection unit 108) included in stereo camera 100 is at least partly It may be aggregated on the controller 500 . In this case, the controller 500 will perform the arithmetic processing.

1 Obstacle detection system 100 Stereo cameras 101, 102 Imaging sensors 103, 104 Distortion correction units 105, 106 Image processing unit 107 Parallax detection unit 108 Object detection unit 109 Operation unit 200 Rotation mechanism 300 Light projector 400 Camera 500 Controller

Claims (15)

An obstacle detection system for detecting an obstacle on a planned travel track of a vehicle,
a proximity monitoring unit that monitors the vicinity of the vehicle;
a remote monitoring camera that monitors a farther area than the range monitored by the near field monitoring unit;
an arithmetic unit that controls the near monitoring unit and the remote monitoring camera;
with
The calculation unit detects the planned travel trajectory using the result of monitoring by the proximity monitoring unit,
The computing unit roughly adjusts the photographing direction of the remote monitoring camera based on the detected planned travel trajectory,
The remote monitoring camera captures the coarsely adjusted shooting direction,
The calculation unit calculates the three-dimensional coordinates of the planned travel trajectory using the distant image captured by the remote monitoring camera,
The calculation unit determines a point of interest of the remote monitoring camera based on the three-dimensional coordinates,
The obstacle detection system, wherein the calculation unit finely adjusts the photographing direction of the remote monitoring camera based on the determined focus point. The point of interest is
the farthest detection point of the remote monitoring camera on the planned travel trajectory;
the furthest point that can be photographed by the remote monitoring camera when the planned travel path is blocked in the imaging direction of the remote monitoring camera;
a point ahead of a position where the vehicle is scheduled to stop on the planned travel track;
2. The obstacle detection system according to claim 1, wherein at least one of: The obstacle detection system further comprises a floodlight for irradiating the location photographed by the remote monitoring camera,
The computing unit adjusts the light distribution direction of the light projector so as to irradiate the point of interest with light,
When the brightness of the point of interest in the distant image is insufficient, the computing unit performs at least one of reducing the light distribution angle of the light projector and increasing the light intensity of the light projector. 2. The obstacle detection system according to claim 1. The computing unit cuts out an image region including the point of interest from the distant image,
The computing unit detects the obstacle using the clipped image area,
If the point of interest is not included within a predetermined range in the distant image captured by the remote monitoring camera in the coarsely adjusted shooting direction, the computing unit performs the fine adjustment to perform the clipping. 2. The obstacle detection system according to claim 1, wherein the point of interest is included in the image area obtained by the observation. The obstacle detection system further comprises a mechanism for changing the shooting direction by mechanically rotating the remote monitoring camera,
The computing unit rotates the remote monitoring camera at a first rotation speed when the point of interest is at a first distance from the vehicle,
The computing unit rotates the remote monitoring camera at a second rotation speed slower than the first rotation speed when the point of interest is at a second distance farther than the first distance from the vehicle,
When the rotation speed of the remote monitoring camera required to include the point of interest within the predetermined range exceeds an upper threshold, the calculation unit suppresses the rotation speed to the upper threshold or less, and reduces the image area. 5. The obstacle detection system according to claim 4, wherein the point of interest is included in the clipped image area by shifting the clipping position. the remote monitoring camera is a stereo camera having a first camera and a second camera,
The obstacle detection system further comprises a mechanism for changing the photographing direction by mechanically rotating the first camera and the second camera,
The calculation unit corrects a change in image magnification caused by rotating the first camera and the second camera, or measures distance caused by rotating the first camera and the second camera. correct the error,
The obstacle detection system according to claim 1, wherein the calculation unit detects the planned travel path and the obstacle using the corrected image magnification or the corrected distance measurement error. the remote monitoring camera is a stereo camera,
2. The obstacle detection system according to claim 1, wherein the calculation unit calculates the three-dimensional coordinates based on parallax images acquired by the remote monitoring camera. The proximity monitoring unit captures a proximity image used by the computing unit to detect the planned travel trajectory,
the remote monitoring camera captures the distant image used by the computing unit to detect the obstacle;
The near image and the far image are characterized in that at least one of the brightness of the near image is higher than the brightness of the far image, or the contrast of the near image is higher than the contrast of the far image. The obstacle detection system according to claim 1, wherein: the remote monitoring camera captures a plurality of the distant images in chronological order;
The calculation unit obtains an image of the planned travel track by adding together the distant images along a time series when the variation of the radius of curvature of the planned travel track is within an allowable range. The obstacle detection system according to claim 1. the remote monitoring camera is a stereo camera having a first camera and a second camera,
The calculation unit adds an image captured by the first camera at a first viewpoint and an image captured by the second camera at a second viewpoint different from the first viewpoint to obtain the planned travel trajectory. The obstacle detection system according to claim 1, wherein an image is acquired. The computing unit acquires a value specifying a road width of the planned travel track,
The obstacle detection system according to claim 1, wherein the calculation unit calculates the three-dimensional coordinates using the acquired road width value. 2. The obstacle detection system according to claim 1, wherein the proximity monitoring unit is configured by a Lidar or an image sensor having a wider angle of view than the remote monitoring camera. 2. The obstacle detection system according to claim 1, wherein the range of angles that can be monitored by said proximity monitor is 65 degrees to 75 degrees. the remote monitoring camera is a stereo camera having a first camera and a second camera,
The first camera is configured to be able to dynamically change the angle of view,
2. The obstacle detection system according to claim 1, wherein the proximity monitoring unit is configured by setting the angle of view of the first camera larger than the angle of view of the second camera. the remote monitoring camera is a stereo camera having a first camera and a second camera,
The first camera is configured such that its optical axis can be changed in the vertical direction,
The first camera is configured to be able to switch between a near-field mode for photographing the vicinity of the vehicle and a far-field mode for photographing a farther area than the near-field mode,
The obstacle detection system according to claim 1, wherein the first camera tilts the optical axis more downward in the near mode than in the far mode.

Images