JP2022077655A - Obstacle detection system - Google Patents

Abstract

To provide an obstacle detection system which cannot only detect a planned travel track surely even using a narrow view angle camera, but also detect an obstacle surely on the planned travel track.SOLUTION: An obstacle detection system in this invention roughly adjusts an imaging direction of a remote monitoring camera based on a planned travel track acquired by a neighborhood monitoring and furthermore finely adjusts the imaging direction of the remote monitoring camera, based on three dimensional coordinates of the planned travel track calculated from a remote picture after the rough adjustment.SELECTED DRAWING: Figure 1

Description

The present invention relates to an obstacle detection system that detects an object on a planned traveling track of a vehicle.

Technologies have been developed to ensure safety by mounting detectors such as cameras and object sensors on vehicles such as automobiles and railroads and detecting objects in the planned travel track of the vehicle in advance. The following Patent Documents 1 and 2 are conventional techniques related to this technique.

Patent Document 1 describes that the direction of the radar or the camera is controlled from the information of the navigation system and the line-of-sight information of the driver. According to the document, "The control device 11 detects at least the presence or absence of an intersecting road or the number of lanes as a road configuration in front of the traveling direction of the own vehicle or on the predicted traveling trajectory based on the information acquired from the navigation device 17 or the like. , The detection direction of the radar 13 and the imaging direction of the camera 14 according to the current position and the traveling trajectory (or the moving direction) of the own vehicle, and further according to the driver's line-of-sight direction detected by the line-of-sight detection device 18. The technique of "setting up" is described (see summary).

Patent Document 2 describes that the direction of the headlight is controlled by using the lane information. According to the same document, "The irradiation direction control unit determines the target swivel angle according to the detected lane extension direction, and controls the swivel actuator so that the irradiation direction of the light from the vehicle headlights approaches the target swivel angle. The technique of "performing follow-up control" is described (see summary).

JP-A-2005-231450 JP 2012-196982

The obstacle detection system not only detects the presence or absence of an object in front, but also grasps the planned travel track that the vehicle is going to pass, and also grasps the relative position between the planned travel track and the object. , It is possible to determine whether or not the vehicle and the object collide.

On the other hand, when a distant object is detected by an image sensor, so-called zoom photography is used in which the focal length of the lens is lengthened so that an image on an image sensor can be photographed with a size larger than a predetermined size. It is desirable to increase the size of the image sensor according to the focal length, but it may be difficult to realize due to factors such as cost, size, and the presence or absence of the image sensor. If the image sensor size is small, the angle of view is inevitably narrow. If the angle of view is narrowed, the shooting range will be narrowed, so there is a possibility that the planned travel track will not be reflected in the image. Also, even if a part of the planned travel track is reflected, there is a possibility that the range to be detected is not sufficiently included.

Even if the planned travel track can be captured, it may not be possible to detect the object due to insufficient illuminance or contrast at night. In particular, when the illuminance of a distant object is obtained by the lighting equipment installed in the own vehicle, the brightness may be lower on the road surface arranged parallel to the light distribution direction than on the three-dimensional object facing the light distribution direction. There is sex. It is expected that this will reduce the detection performance of the planned travel track.

The present invention has been made in view of the above problems, and it is possible to reliably detect an obstacle on a planned travel track as well as reliably detect an obstacle on the planned travel track even with a narrow angle of view camera. The purpose is to provide an obstacle detection system that can be used.

The obstacle detection system according to the present invention roughly adjusts the shooting direction of the distant surveillance camera based on the planned travel trajectory obtained by proximity monitoring, and is based on the three-dimensional coordinates of the planned travel trajectory calculated from the distant image after the rough adjustment. Then, fine-tune the shooting direction of the distant surveillance camera.

According to the obstacle detection system according to the present invention, even a narrow angle-of-view camera can reliably capture the planned travel track and detect obstacles in the detection range.

It is a schematic block diagram of the obstacle detection system 1 which concerns on Embodiment 1. FIG. It is a figure which shows typically the situation that the stereo camera 100 cannot take a picture of the vicinity of a vehicle. It is a top surface schematic diagram which shows the example of the planned traveling track of a vehicle. An example of the shooting range of each of the stereo camera 100 and the camera 400 is shown. The image after the shooting direction of the stereo camera 100 is roughly adjusted is shown. It is a figure which shows the state which determines the attention point after the shooting direction of a stereo camera 100 is roughly adjusted. This is an image in which the planned travel track is a straight line and a distance beyond the detection range of the stereo camera 100 is shown. Similar to FIG. 7, this is an example in which a distance beyond the detection range is shown. This is an example of an uphill slope in front of the stereo camera 100 and a downhill slope ahead of it. This is an example in which the farthest position of the detection range cannot be seen due to utility poles and other structures. This is an example of a station on the left side of the planned travel track. FIG. 5 shows an image obtained by cutting out a portion taken by the stereo camera 100. It shows how the position of the image taken by the stereo camera 100 gradually changes. It shows how the position of the image taken by the stereo camera 100 gradually changes. It shows how the position of the image taken by the stereo camera 100 gradually changes. An example of the rotation mechanism 200 is shown. The state of the stereo camera 100 when the shooting direction is changed is shown. It is another expression of FIG. It is a functional block diagram of a stereo camera 100. It is a graph which shows the exposure and sensitivity setting of a stereo camera 100 at the time of detecting a three-dimensional object. It is a graph which shows the exposure and sensitivity setting of a stereo camera 100 in the case where image blur is hard to occur. It is a figure which shows the image acquisition timing in the case where the frame rate is sufficiently long with respect to the exposure time. The processing timing when the frame images are added up is shown. This is an example of one of the left and right images of the stereo camera 100. This is an example of an image taken by a variable focus camera. It is a block diagram of the obstacle detection system 1 in Embodiment 2. This is an example of an image when the optical axis is directed downward. It is a block diagram of the obstacle detection system 1 in Embodiment 3.

<Embodiment 1>
FIG. 1 is a schematic configuration diagram of an obstacle detection system 1 according to the first embodiment of the present invention. The obstacle detection system 1 is a system that detects an obstacle on the planned traveling track of the vehicle by being mounted on the vehicle. In this embodiment, an example in which the obstacle detection system 1 is mounted on a train will be described. The obstacle detection system 1 includes a stereo camera 100 (distance monitoring camera), a rotation mechanism 200, a floodlight 300, a camera 400 (nearby monitoring unit), and a controller 500.

The stereo camera 100 is for detecting an obstacle in the distance by photographing the distance of the vehicle. The stereo camera 100 is composed of two cameras. The rotation mechanism 200 changes the shooting direction of each camera of the stereo camera 100. The floodlight 300 irradiates light to a distant place of the vehicle. The camera 400 takes a picture of the vicinity of the vehicle (a position closer than the distance taken by the stereo camera 100). The controller 500 controls the camera 400 and the stereo camera 100.

The stereo camera 100 sends the shooting direction information a with respect to the camera optical axis to the controller 500. Similarly, the proximity camera 400 sends the coordinate information b of the road surface or the track shown on the captured image to the controller 500. The controller 500 sends a drive signal for pointing the stereo camera 100 in a predetermined shooting direction to the rotation mechanism 200 based on the information a and b. The rotation mechanism 200 directs the shooting direction of the stereo camera 100 in that direction. The controller 500 sends light distribution direction information to the floodlight 300 based on the information a and b. The floodlight 300 directs the light distribution direction in that direction.

FIG. 2 is a diagram schematically showing a situation in which the stereo camera 100 cannot photograph the vicinity of the vehicle. In the stereo camera 100, the focal length of the lens is long and the angle of view is narrow in order to detect a distant obstacle. In general, the front surveillance camera installs the optical axis substantially horizontally, so the narrower the angle of view as shown in Fig. 2, the farther the distance c where the road surface begins to appear in the image shifts, and the more it appears in the image shown by d. The range of the road surface directly under or near the lens is expanded.

FIG. 3 is a schematic top view showing an example of a planned traveling track of a vehicle. As shown in FIG. 3, a curve having a smaller radius of curvature of a planned traveling track (for example, a road or a railroad track) requires an angle of view, and a curve having the same radius of curvature requires an angle of view as it is farther away. In other words, the longer the focal length, the less the roads and railroad tracks will appear on the screen.

It is considered that the planned travel track such as a road or a railroad track cannot be detected from the image, the planned travel track cannot be detected properly due to insufficient light intensity at night, or the planned travel track is not shown in the image. If there are multiple causes, it is preferable to take measures from the cause that can be reliably overcome. In this case, it is desirable to first operate the obstacle detection system 1 so that the planned travel track is reflected in the image, and then perform exposure adjustment and image processing in order to reliably detect the planned travel track. Therefore, in the following, an operation example for reliably projecting the planned traveling track in the image will be described.

FIG. 4 shows an example of the shooting range of each of the stereo camera 100 and the camera 400. The outer thick black solid line is the shooting range of the camera 400, and the inner thick black solid line is the shooting range of the right camera of the stereo camera 100. The white dotted line on the camera 400 image is an area where the planned traveling track is guaranteed to be reflected regardless of the slope of the road surface and the degree of curve. This is possible if the angle of view of the camera 400 is sufficiently secured and the Y coordinate of the white dotted line is appropriately set. The controller 500 searches for the planned travel track on the white dotted line and obtains the X coordinate (white circle in FIG. 4) of the planned travel track. The controller 500 has information corresponding to the shooting direction of the stereo camera 100 (for example, coordinates on the camera 400 screen at the center of the stereo camera 100 screen), and shoots the stereo camera 100 from the screen center information and the above-mentioned white circle coordinates. Determine the direction and its amount. This corresponds to coarsely adjusting the shooting direction of the stereo camera 100 based on the image of the camera 400.

FIG. 5 shows an image after the shooting direction of the stereo camera 100 is roughly adjusted. The rough adjustment referred to here is to roughly determine the shooting direction before the fine adjustment described below is performed.

FIG. 6 is a diagram showing how the point of interest is determined after the shooting direction of the stereo camera 100 is roughly adjusted. After the controller 500 roughly adjusts the shooting direction of the stereo camera 100, the stereo camera 100 measures the three-dimensional coordinates of the planned travel trajectory displayed on the screen, and based on the result, the target point of interest in the shooting direction (FIG. 6). Black circle) is decided. The fine adjustment here is to finely adjust the shooting direction roughly determined by the rough adjustment, and the purpose is to determine the point of interest.

7 to 11 show examples of points of interest. In each figure, the points of interest are indicated by black circles. Hereinafter, examples of points of interest will be described with reference to these figures.

In FIG. 7, the planned travel trajectory is a straight line, and the distance is longer than the detection range of the stereo camera 100 (the maximum distance that the stereo camera 100 can detect an object from the image captured by the stereo camera 100 is defined by the specifications of the stereo camera 100). It is a reflected image. In this case, the stereo camera 100 focuses on the farthest position in the detection range of the stereo camera 100.

FIG. 8 is an example in which a distance beyond the detection range is shown as in FIG. 7. In FIG. 8, the stereo camera 100 also focuses on the farthest position in the detection range.

FIG. 9 shows an example in which the front side of the stereo camera 100 is an uphill slope and the front side is a downhill slope. In FIG. 9, since the slope is downhill from the middle of the detection range, the farthest position of the detection range cannot be seen. In this case, the stereo camera 100 focuses on the farthest position that can be seen.

FIG. 10 is an example in which the farthest position of the detection range cannot be seen due to a utility pole or other structure. In this case, the stereo camera 100 focuses on the farthest position that can be seen (the farthest position in the range not blocked by the structure).

FIG. 11 is an example in which the station is on the left side of the planned travel track. In this case, the stereo camera 100 focuses on a predetermined distance ahead of the stop position of the vehicle. For example, the point of interest can be a point in front of the stop position and closest to the photographable range. However, as long as it is in front of the stop position, it does not necessarily have to be the nearest neighbor shooting possible point.

As described with reference to FIGS. 7 to 11, the stereo camera 100 has (a) the farthest point of the detection specifications of the stereo camera 100 in the planned travel track, and (b) the prospect when the planned travel track is obstructed by terrain or a building. The point of interest is either (c) the farthest possible point, or (c) a predetermined distance ahead of the planned stop position of the own vehicle in the planned travel track.

12 to 15 show a process of finely adjusting the shooting direction of the stereo camera 100. The fine adjustment operation will be described with reference to FIGS. 12 to 15.

FIG. 12 shows an image obtained by cutting out a portion of FIG. 5 taken by the stereo camera 100. The white circles are the coordinates of the planned traveling track described with reference to FIG. The black circle is the point of interest. The controller 500 first calculates a parallax image from the cut-out images of the left and right cameras, and then calculates the three-dimensional coordinates of the planned traveling trajectory to determine the point of interest.

The vertical dotted line is the coordinate target of the point of interest. If the coordinates of the point of interest are in the area to the right of the dotted line on the right side, the shooting direction of the stereo camera 100 is shifted to the right, or the cutout position is shifted to the right. If the coordinates of the point of interest are in the area to the left of the dotted line on the left side, the shooting direction of the stereo camera 100 is shifted to the left, or the cutout position is shifted to the left. The arrow shown in FIG. 12 indicates the shift amount. These screen shifts may be performed depending on the shooting direction or the cutout area. However, in order to avoid image flow, it is desirable to change the camera orientation mechanically at a speed that does not exceed the upper limit set according to the distance of the point of interest, and adjust the shortage within the cutout range. That is, the farther the point of interest is, the slower the rotation speed for the camera, and if the rotation speed reaches the upper limit threshold value, the rotation speed is set to be equal to or less than the upper limit threshold value, and the screen is shifted by finely adjusting the cutting range.

13 to 15 show how the position imaged by the stereo camera 100 gradually changes. Since it is assumed that the person concerned will monitor the cut-out image, the cut-out position is set so that the cut-out area also changes smoothly. Therefore, the landscape gradually moves as shown in FIGS. 13 to 14, and the cutting is completed when the point of interest comes to the dotted line position as shown in FIG.

FIG. 16 shows an example of the rotation mechanism 200. As shown in FIG. 16, the rotation mechanism 200 has a structure having a rotation axis (black circle in FIG. 16) in the vertical direction for each of the two left and right cameras. As another method, there is a method of providing a rotating shaft in the center of the housing as shown by the white circle in FIG. Requires space. By rotating each camera, a narrow space can be realized even with a long baseline stereo camera as the distance increases, and this embodiment can also be applied to a stereo camera having an independent left and right structure.

FIG. 17 shows the state of the stereo camera 100 when the shooting direction is changed.

FIG. 18 is another representation of FIG. FIG. 17 can be drawn as shown in FIG. 18 from a different point of view. From FIG. 18, it can be seen that the baseline length and the position in the optical axis direction of the camera change depending on the direction of the shooting direction.

The magnification of the image changes depending on the change in the position in the optical axis direction. When the magnification is M, the shooting direction is θ, and the baseline length is L, the magnification M of the distance X of the target object is expressed by the following equation. Since the object distance is unknown before the parallax is obtained, the magnification is calculated with X = "maximum detection distance".

M (left) = (XL / 2 · sinθ) / X (Equation 1)
M (right) = (X + L / 2 · sinθ) / X (Equation 2)

The closer the object is, the larger the size of the object is between the left and right images, and the larger the distance measurement error is. Therefore, in order to set the detection range on the near side, it is necessary to consider according to the shooting direction. By correcting the left and right images with the magnification calculated by the above formula, it is possible to calculate the parallax for a distant object near the maximum detection distance as before.

FIG. 19 is a functional block diagram of the stereo camera 100. The stereo camera 100 includes image sensor 101 and 102, distortion correction units 103 and 104, image processing units 105 and 106, parallax detection unit 107, object detection unit 108, and calculation unit 109. The image pickup sensors 101 and 102, the distortion correction units 103 and 104, and the image processing units 105 and 106 perform processing corresponding to the left camera and the right camera, respectively.

Based on the angle information a sent from the controller 500, the calculation unit 109 sends the cutout information c of the captured image to the sensors 101 and 102, sends the rotated angle information d to the distortion correction units 103 and 104, and base line length information. e is sent to the object detection unit 108. The sensors 101 and 102 perform synchronous shooting at predetermined intervals, cut out an image based on the sent cutout information c, and send the cut out image to the distortion correction units 103 and 104. The distortion correction units 103 and 104 correct the transmitted image based on the optical distortion correction information acquired in advance and the transmitted rotated angle information d, and the corrected image is corrected by the image processing units 105 and 106 and the parallax detection unit. Send to 107. The image processing units 105 and 106 perform image processing such as contrast correction and noise reduction on the transmitted corrected image as necessary, and send the processed image to the discriminant detection unit 107. The disparity detection unit 107 generates a disparity image from the corrected images sent from the image processing units 105 and 106, and also generates and generates a disparity image from the processed images sent from the image processing units 105 and 106 as needed. Each of the differential images is sent to the object detection unit 108. The object detection unit 108 obtains three-dimensional information of each pixel based on the sent parallax image and the baseline length information e, and detects an object or a planned travel trajectory.

As another method from the above, when converting the calculated parallax to the distance of the object without performing the magnification correction on the image, the distance correction based on the difference in the left and right baseline lengths and the measurement based on the coordinates of the pixel of interest are used. Distance correction may be performed. In this case, the error depending on the distance of the three-dimensional object is small, and accurate distance measurement is possible. In this case, instead of the rotated angle information d in FIG. 19, the distance correction information f may be sent from the calculation unit 109 to the object detection unit 108.

The floodlight 300 usually determines the light distribution direction so as to be illuminated around the point of interest. However, when the planned travel trajectory is searched on the white dotted line of the camera 400 as shown in FIG. 4, the light distribution direction is shifted laterally so as to sequentially illuminate the white dotted line in synchronization with the search operation. After that, the controller 500 calculates three-dimensional data along the planned travel track starting from the white circle, and shifts the light distribution direction to a distant place. The light distribution angle and light intensity may be set based on the acquired image. That is, when it is determined from the pixel values of the acquired image that the illumination light is insufficient, the controller 500 improves the illuminance at the point of interest by at least either increasing the light distribution intensity or narrowing the light distribution angle. On the contrary, when the illuminance of the point of interest is sufficiently obtained, the controller 500 widens the light distribution angle to the same extent as the camera angle of view.

As one factor to judge whether the detected three-dimensional object is an obstacle, whether the three-dimensional object is near the planned travel track or is approaching the planned travel track, etc., the planned travel track and the three-dimensional object Relative position information between them is important. Therefore, in order to detect obstacles, it is important to detect the planned travel track. Since sufficient illuminance can be obtained for both the three-dimensional object and the road surface during the daytime, the detection performance can be ensured. Since the illuminance is insufficient at night, it is necessary to improve the illuminance by the floodlight 300 in addition to increasing the exposure time and the imaging sensitivity. When the floodlight 300 is mounted on a vehicle, it is more difficult to obtain contrast on a road surface that is almost parallel to the floodlight direction than on a three-dimensional object in which the floodlight direction and the irradiation surface face each other. .. The following describes the device for that purpose.

FIG. 20 is a graph showing the exposure and sensitivity setting of the stereo camera 100 when detecting a three-dimensional object. The horizontal axis of the graph is the exposure time, and the vertical axis is the sensitivity. During the day, the exposure time is short and the sensitivity is set low. The exposure time and sensitivity are increased according to the pixel value of the acquired image to ensure the brightness. If the sensitivity is raised too much, noise will occur, so the upper limit value g is set. If the exposure time is too long, image blurring will occur, so the upper limit values h1 to h2 are set according to the vehicle speed. This operation is used for normal three-dimensional object detection, and is processed by the path leading to the sensor 101, the strain correction unit 103, and the parallax detection unit 107 in FIG. 19, and the path leading to the sensor 102, the strain correction unit 104, and the parallax detection unit 107. To.

FIG. 21 is a graph showing the exposure and sensitivity settings of the stereo camera 100 when image blurring is unlikely to occur. When the planned travel track is a straight line or a curve with a constant radius of curvature, such as a white line or rail (that is, when the radius of curvature changes within a certain range, the same applies hereinafter), there is a feature that image blurring is unlikely to occur. For example, in the image shown in FIG. 7, the position on the image of the planned traveling track is relatively stable with respect to the passage of time. In this case, the image used for detecting the planned traveling track can be acquired with the exposure time sufficiently longer than the image for detecting a three-dimensional object as shown in FIG. 21.

FIG. 22 is a diagram showing image acquisition timing when the frame rate is sufficiently longer than the exposure time. In this case, as shown in FIG. 22, the image for detecting a three-dimensional object may be acquired (h2) and then the image for detecting a planned trajectory may be acquired (h3). If the frame rate is not long enough to acquire the image for the planned travel track, the image processing units 105 and 106 save the image of the previous frame and add it to the image of the current frame to equivalently increase the exposure amount. be able to.

FIG. 23 shows the processing timing when the frame images are added up. The upper part of FIG. 23 shows the exposure time and frame rate of the sensors 101 and 102. The middle part of FIG. 23 shows the image storage timing by the image processing units 105 and 106. The lower part of FIG. 23 shows the timing of adding the current frame and the previous frame. In particular, when the planned travel track is a straight line or a curve with a constant radius of curvature, image blur is unlikely to occur even if frame images are added, so it is useful to improve the contrast by adding them.

As another method when the frame rate is sufficiently longer than the exposure time, a viewpoint moving process of replacing one of the left and right images of the stereo camera 100 with the other viewpoint can be performed and added to the other image. In this case as well, the same effect as frame addition can be obtained.

Although the method of increasing the image contrast of the planned traveling track has been described above, the image brightness can also be increased by the same method. As long as at least one of contrast and brightness can be increased, it can be said that the planned travel track can be detected more reliably.

FIG. 24 is an example of one of the left and right images of the stereo camera 100. The right image when FIG. 7 is the left image is as shown in FIG. 24. When the viewpoint movement process is performed on the road surface plane, the planned traveling track on the road surface plane can be matched with the left image.

In the above description, it has been described that the shooting direction of the stereo camera 100 is moved to the left-right direction, but the present invention is not limited to this. If there is an inclination, the point of interest moves up and down, so the cutting position may also be shifted up and down. Further, the shooting direction may be mechanically shifted.

In the above description, it has been described that the three-dimensional coordinates of the planned traveling track are obtained from the parallax image in order to determine the point of interest, but the present invention is not limited to this. When the width of the planned travel track is known, the three-dimensional coordinates of the planned travel track may be calculated from the camera image / focal length / pixel pitch using the known road width. In this case, it is possible to obtain the three-dimensional coordinates of the planned travel track without necessarily using a stereo camera.

In this embodiment, the total angle of view of the camera 400 is approximately 70 degrees. This is a value that includes the mounting accuracy in the required angle of view of 67 degrees estimated from the relationship between the acceleration and the radius of curvature published by the railway operator. Even when monitoring the vicinity of the vehicle using other sensors, it is sufficient if the left and right detection angles of view are 70 to 75 degrees. Therefore, the detection angle of view of the camera 400 needs to be approximately 65 to 75 degrees.

Although one floodlight 300 is used in this embodiment, the same effect can be obtained by using a plurality of floodlights 300. In the case of a plurality of cases, it can be shared between the distant projection and the near projection. Conventional equipment can also be used as a floodlight for the vicinity.

<Embodiment 1: Summary>
The obstacle detection system 1 according to the first embodiment determines the orientation of the stereo camera 100 (distant monitoring camera) and the floodlight 300 based on the information of the proximity monitoring means capable of reliably capturing the planned traveling track such as a white line or a rail. After making a rough adjustment, the orientations of the stereo camera 100 and the floodlight 300 are finely adjusted based on the three-dimensional image acquired by the stereo camera 100. As a result, it is possible to stably capture the planned travel track while ensuring a sufficient image magnification for object detection. That is, it is possible to provide a highly reliable obstacle detection system 1.

The obstacle detection system 1 according to the first embodiment obtains the image used for detecting the planned traveling track and the image used for detecting a three-dimensional object by changing the exposure conditions. Image accuracy is ensured by using frame addition and addition after moving the viewpoint. As a result, even when the illuminance and contrast cannot be sufficiently secured such as at night, the brightness and contrast of the planned travel track can be emphasized and the planned travel track can be detected with high accuracy.

<Embodiment 2>
In the first embodiment, a wide-angle camera is used as a means for detecting a planned traveling track in the vicinity. It is also possible to assign this role to either the left or right camera of the stereo camera 100. That is, one of the left and right cameras is configured by the variable focus camera.

FIG. 25 is an example of an image taken by a variable focus camera. When taken with the wide side (short focal length and wide angle) of the variable focal length camera of the stereo camera 100, the image as shown in FIG. 25 is acquired, and when taken with the tele side (long and narrow focal length), the image as shown in FIG. 12 is obtained. get.

FIG. 26 is a configuration diagram of the obstacle detection system 1 in the second embodiment. The zoom mechanism of the variable focus camera is included in the stereo camera 100. The stereo camera 100 sets one of the left and right cameras to the wide side to shoot an image, and sends the shooting direction information a with respect to the optical axis of the camera to the controller 500. The stereo camera 100 sends the coordinate information b of the road surface or the railroad track shown on the captured image to the controller 500. The controller 500 sends a drive signal for pointing the stereo camera 100 in a predetermined shooting direction to the rotation mechanism 200 based on the information a and b, and requests that the variable focus camera be changed to the telephoto side. The controller 500 sends light distribution direction information to the floodlight 300 based on the information a and b. Since the other processing processes are the same as those in the first embodiment, the description thereof will be omitted.

In the present embodiment, the image on the wide side is intended to obtain the two-dimensional coordinates of the planned traveling track, but the present embodiment is not limited to this. That is, three-dimensional information can be obtained by enlarging the image on the wide side or reducing the image on the tele side to view the overlapping area on the left and right in stereo. In reality, even in wide-side images, the accuracy of distortion correction and parallelization affects the acquisition performance and accuracy of three-dimensional information, so distortion correction processing for both wide and tele may be required.

<Embodiment 3>
In the first embodiment, a wide-angle camera is used as a means for detecting a planned traveling track in the vicinity. It is also possible to have at least one of the left and right cameras of the stereo camera 100 play this role. That is, by rotating either of the left and right cameras in the pitch direction (vertical direction) and directing the optical axis downward, it is possible to reliably capture the planned travel trajectory within the angle of view.

FIG. 27 is an example of an image when the optical axis is directed downward. In this case, an image like the dotted line in FIG. 27 can be obtained, and an image like the solid line in FIG. 27 is acquired based on the planned travel trajectory coordinates at the upper part of the image.

FIG. 28 is a configuration diagram of the obstacle detection system 1 in the third embodiment. The pitch rotation unit 600 rotates either the left or right camera of the stereo camera 100 in the pitch direction. The stereo camera 100 sets the optical axis to the lower side in the pitch direction, shoots an image, and sends the shooting direction information a with respect to the camera optical axis to the controller 500. Unlike the first embodiment, the photographing direction information a includes pitch direction information in addition to yaw direction information. The stereo camera 100 sends the coordinate information b of the road surface or the railroad track shown on the captured image to the controller 500. The controller 500 sends a drive signal for directing the stereo camera 100 in a predetermined shooting direction to the rotation mechanism 200 and the pitch rotation unit 600 based on the information a and b. The controller 500 sends light distribution direction information to the floodlight 300 based on the information a and b. Since the other processing processes are the same as those in the first embodiment, the description thereof will be omitted.

In the third embodiment, the stereo camera 100 has an operation mode for photographing a distant place and an operation mode for photographing a vicinity. In the near-field shooting mode, the pitch rotation unit 600 tilts the camera optical axis downward from the far-field shooting mode. It is not always necessary to tilt the camera exactly along the vertical direction, and it is sufficient that at least the distance shooting mode faces downward from the near shooting mode.

<About a modification of the present invention>
The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

Although the camera 400 or the stereo camera 100 has been described as the means for detecting the state in the vicinity of the vehicle in the above embodiment, the state in the vicinity of the vehicle may be detected by means other than the camera. For example, it is conceivable to use a distance measuring sensor such as Lidar. A plurality of detection means may be used in combination.

Each processing unit (arithmetic unit 109, distortion correction units 103 and 104, image processing units 105 and 106, parallax detection unit 107, object detection unit 108) and the controller 500 included in the stereo camera 100 are circuit devices that implement these functions. It can be configured by hardware such as, or it can be configured by executing software that implements these functions by an arithmetic unit. It may be implemented by combining hardware and software.

In the above embodiment, each processing unit (calculation unit 109, strain correction units 103 and 104, image processing units 105 and 106, parallax detection unit 107, object detection unit 108) included in the stereo camera 100 includes at least a part thereof. It may be aggregated on the controller 500. In this case, the controller 500 will perform the arithmetic processing.

1 Obstacle detection system 100 Stereo camera 101, 102 Imaging sensor 103, 104 Distortion correction unit 105, 106 Image processing unit 107 Parallax detection unit 108 Object detection unit 109 Calculation unit 200 Rotation mechanism 300 Floodlight 400 Camera 500 Controller

Claims (15)

It is an obstacle detection system that detects obstacles on the planned track of the vehicle.
A neighborhood monitoring unit that monitors the vicinity of the vehicle,
A distant surveillance camera that monitors a distance farther than the range monitored by the proximity monitoring unit,
An arithmetic unit that controls the neighborhood monitoring unit and the distant monitoring camera,
Equipped with
The calculation unit detects the planned travel track by using the monitoring result by the neighborhood monitoring unit.
The calculation unit roughly adjusts the shooting direction of the distant surveillance camera based on the detected track to be traveled.
The distant surveillance camera captures the coarsely adjusted shooting direction.
The calculation unit calculates the three-dimensional coordinates of the planned travel track using the distant image taken by the distant surveillance camera.
The calculation unit determines the point of interest of the remote surveillance camera based on the three-dimensional coordinates.
The calculation unit is an obstacle detection system characterized in that the shooting direction of the remote surveillance camera is finely adjusted based on the determined point of interest. The point of interest is
The farthest detection point of the distant surveillance camera on the planned travel track,
The farthest point that the distant surveillance camera can shoot when the planned travel track is obstructed in the shooting direction of the distant surveillance camera.
A point on the planned travel track ahead of the position where the vehicle is scheduled to stop,
The obstacle detection system according to claim 1, wherein the system is at least one of the above. The obstacle detection system further comprises a floodlight that illuminates a location photographed by the remote surveillance camera.
The calculation unit adjusts the light distribution direction of the floodlight so as to irradiate the point of interest with light.
When the brightness of the point of interest in the distant image is insufficient, the arithmetic unit may at least either narrow the light distribution angle of the floodlight or increase the light intensity of the floodlight. The obstacle detection system according to claim 1, which is characterized by this. The calculation unit cuts out an image region including the point of interest from the distant image, and obtains an image region.
The arithmetic unit detects the obstacle using the cut out image area, and determines the obstacle.
When the point of interest is not included in a predetermined range of the distant images taken by the distant surveillance camera in the roughly adjusted shooting direction, the calculation unit performs the fine adjustment to cut out the cutout. The obstacle detection system according to claim 1, wherein the point of interest is included in the image area. The obstacle detection system further includes a mechanism for changing the shooting direction by mechanically rotating the remote surveillance camera.
When the point of interest is at the first distance from the vehicle, the calculation unit rotates the distant surveillance camera at the first rotation speed.
When the point of interest is at a second distance farther than the first distance from the vehicle, the calculation unit rotates the remote monitoring camera at a second rotation speed slower than the first rotation speed.
When the rotation speed of the remote monitoring camera required to include the point of interest within the predetermined range exceeds the upper limit threshold value, the calculation unit suppresses the rotation speed to be equal to or lower than the upper limit threshold value and displays the image area. The obstacle detection system according to claim 4, wherein the cut-out position is shifted so that the point of interest is included in the cut-out image area. The remote surveillance camera is a stereo camera having a first camera and a second camera.
The obstacle detection system further includes a mechanism for changing the shooting direction by mechanically rotating the first camera and the second camera.
The calculation unit corrects the fluctuation of the image magnification caused by rotating the first camera and the second camera, or the distance measuring unit caused by rotating the first camera and the second camera. Correct the error,
The obstacle detection system according to claim 1, wherein the calculation unit detects the planned travel track and the obstacle by using the corrected image magnification or the corrected distance measurement error. The remote surveillance camera is a stereo camera.
The obstacle detection system according to claim 1, wherein the calculation unit calculates the three-dimensional coordinates based on the parallax image acquired by the remote surveillance camera. The neighborhood monitoring unit captures a neighborhood image used by the calculation unit to detect the planned travel track.
The distant surveillance camera captures the distant image used by the arithmetic unit to detect the obstacle.
The near image and the distant image are characterized in that the brightness of the near image is higher than the brightness of the distant image, or the contrast of the near image is higher than the contrast of the distant image. The obstacle detection system according to claim 1. The distant surveillance camera captures a plurality of the distant images in chronological order.
The calculation unit is characterized in that, when the fluctuation of the radius of curvature of the planned travel track is within an allowable range, the calculation unit acquires an image of the planned travel track by adding the distant images along a time series. The obstacle detection system according to claim 1. The remote surveillance camera is a stereo camera having a first camera and a second camera.
The calculation unit adds up an image taken by the first camera at the first viewpoint and an image taken by the second camera at a second viewpoint different from the first viewpoint, thereby causing the planned travel track. The obstacle detection system according to claim 1, wherein an image is acquired. The calculation unit acquires a value that specifies the road width of the planned travel track, and obtains a value.
The obstacle detection system according to claim 1, wherein the calculation unit calculates the three-dimensional coordinates using the acquired value of the road width. The obstacle detection system according to claim 1, wherein the proximity monitoring unit is composed of an image sensor having a wider angle of view than Lidar or the remote monitoring camera. The obstacle detection system according to claim 1, wherein the angle range that can be monitored by the neighborhood monitoring unit is 65 degrees to 75 degrees. The remote surveillance camera is a stereo camera having a first camera and a second camera.
The first camera is configured to dynamically change the angle of view.
The obstacle detection system according to claim 1, wherein the neighborhood monitoring unit is configured by setting the angle of view of the first camera to be larger than the angle of view of the second camera. The remote surveillance camera is a stereo camera having a first camera and a second camera.
The first camera is configured so that the optical axis can be changed in the vertical direction.
The first camera is configured to be able to switch between a near mode for photographing the vicinity of the vehicle and a distant mode for photographing a distance farther than the near mode.
The obstacle detection system according to claim 1, wherein the first camera tilts the optical axis further downward than in the far mode in the near mode.

Images