JP2015011502A - Forward situation determination device, and safe driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect approach of an opening part through which a passage and the outside of the passage are communicated.SOLUTION: A forward situation determination device includes: a sensor part imaging a forward portion of an advancing direction; an optical flow calculation part calculating size of an optical flow in each prescribed region of an image on the basis of the image imaged by the sensor part; and a scene determination part determining an imaged forward situation from a change of the size of the optical flow of each region. The scene determination part determines that an opening part crossing the advancing direction and opened is present in the forward portion from a change point of the size of the optical flow, and determines whether or not approach to the opening part present in the front portion is performed from movement of the change point of the size of the optical flow.

Description

  The present invention relates to a destination situation determination device and a safe driving support device.

  Technologies that support the driver's prospects at intersections with poor prospects have been proposed. Not only is it equipped with a camera that captures the left and right of the vehicle, but also switches from the normal navigation screen to the camera image that confirms the left and right, regardless of the driver's operation, such as the vehicle's stop state, deceleration acceleration, etc. This is done automatically based on information, the amount of protrusion of the front of the vehicle from the stop line, and the optical flow. Ultimately, it is an important assistive technology to prevent accidents at the intersections at intersections where the driver's vision is poor and the driver cannot see through, although the driver's visual confirmation is necessary.

JP 2010-002275 A JP 2009-246808 A

In the invention of Patent Document 1, the road width is detected by using vehicle speed information such as stop and low speed and the optical flow on the side, and it is determined whether the road width is equal to or less than a set value. A technique for displaying a video from a side camera has been proposed. However, it is not always necessary to monitor the outside world when stopping, and it may be stopped to check the map information. Even if the vehicle is traveling at a low speed for the purpose of confirmation, it is more sure to visually confirm if the field of view is open. When detecting the width of a road by optical flow, it is difficult to grasp the width of the road because there are changes in the environment such as the fence of a house or between houses.
Further, the invention of Patent Document 2 discloses a technique for detecting a stop line position and notifying the driver when the position where the camera is provided reaches an appropriate position beyond the stop line. The positional relationship with the object that obstructs the prospect is not always constant. The present invention has been made in view of such problems, and an object of the present invention is to provide a technique that can accurately detect the approach of an opening that passes through a passage and the outside of the passage.

The present invention solves the above-mentioned problems of the prior art,
A sensor unit that captures the forward direction of the traveling direction, an optical flow calculation unit that calculates the size of the optical flow for each predetermined region of the image captured by the sensor unit, and a change in the size of the optical flow for each region And a scene determination unit that determines the destination situation based on the destination situation determination apparatus.

The present invention also provides
A sensor unit that captures the image in the direction of travel, an optical flow calculation unit that calculates the size of the optical flow for each predetermined region based on the image captured by the sensor unit, A scene determination unit for determining a situation of the other side imaged from a change in the size of the optical flow, and a monitor, and the scene determination unit approaches an opening that intersects and opens in the traveling direction The present invention provides a safe driving support device that displays the image on the monitor when it is determined that the operation has been performed.

According to the front-end situation determination apparatus of the present invention, it is possible to provide a front-end situation determination apparatus that can accurately detect the approach of the opening that passes through the passage and the outside of the passage.
Moreover, according to the safe driving support apparatus of the present invention, it is possible to provide a safe driving support apparatus that can confirm a front image on a monitor only when approaching a place with poor visibility.

It is a block diagram of a former situation judging device in an embodiment. It is explanatory drawing for demonstrating the determination of the other party state by an optical flow. It is explanatory drawing for demonstrating the determination of the other party state by an optical flow. It is explanatory drawing for demonstrating the determination of the other party state by an optical flow. It is a block diagram of a safe driving support device in an embodiment. It is explanatory drawing for demonstrating the determination of the other party situation in the intersection where a right-and-left sight is bad. It is explanatory drawing for demonstrating the determination of the other party situation in the intersection with a bad view only on the right side. It is explanatory drawing for demonstrating the determination of the other party situation in the intersection with a favorable left-right sightline. It is explanatory drawing for demonstrating determination of an upward direction ahead situation. It is explanatory drawing for demonstrating the detection range in case a horizontal viewing angle is 180 degree | times.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .

FIG. 1 is a block diagram of a destination situation determination apparatus 1 according to an embodiment of the present invention.
The destination situation determination apparatus 1 includes at least a sensor unit 101, an optical flow calculation unit 102, a scene determination unit 103, an optical flow threshold setting unit 104, and a notification unit 105.

  The sensor unit 101 is an optical image detection unit such as a camera, for example. The sensor unit 101 outputs the captured image to the optical flow calculation unit 102 as image information.

  The optical flow calculation unit 102 calculates an optical flow from the image information, and outputs an average value of the optical flow size for each predetermined block to the scene determination unit 103.

  The scene determination unit 103 determines the state of the imaging range by the sensor unit 101 based on a predetermined threshold determined by the optical flow threshold setting unit 104.

  The notification unit 105 notifies the determination result by an image, sound, vibration, or the like.

An outline of how the situation of the other side is judged by the optical flow will be described.
FIG. 2 a is an image viewed from one end side of a circular passage. It is assumed that there is a circular outlet that is the same as the cross-sectional shape of the passage, and that the space is widened beyond that.
Assuming that the front-end situation determination device 1 advances forward in this passage, that is, toward the exit, the inner wall surface is uniform, so that the optical flow is directed outward from the center of the image as shown in FIG. 2b. The optical flow is as follows.

  FIG. 2C is a diagram in a case where a predetermined block for obtaining the average value of the optical flow calculation unit 102 is obtained in a radial manner. In this case, since the inner wall surface has no exit, the optical flow of each block has the same size.

The scene determination unit 103 determines the size of the optical flow for each predetermined block based on a threshold determined by the optical flow threshold setting unit 104.
When the areas of the respective blocks are made equal, the threshold value is equal to the distance from the center of the image, that is, the threshold value is circular, and the optical flow threshold value setting unit 104 determines the radius of the circular threshold value.

As the near-end situation determination apparatus 1 approaches the exit as shown in FIG. 3a, if the opening of the exit is a large space, movement cannot be detected, so the optical flow corresponding to the opening becomes short and small. The direction will not be uniform. The optical flow by the inner wall of the passage is radial but extremely short toward the outer periphery.
Therefore, if the threshold is set to a predetermined size and the size of the optical flow is determined, the approach of the opening can be detected.

  FIG. 4a shows a case where there is an outlet on the front face and an outlet or other passage (hereinafter referred to as an open portion) on the inner wall surface in the middle. There is an open portion 41 at a position close to its own position, and an open portion 42 at a far position.

  FIG. 4B is an optical flow in the case of FIG. 4A, and the size of the optical flow at a portion where the inner wall release portion is present is reduced. Also, the optical flow position is a portion 41b near the center of the image at the location of the open portion 41, and a location near the outer periphery of the image at the location of the open portion 42.

The predetermined block for obtaining the average value of the optical flow calculation unit 102 is a radial and a plurality of concentric circles as shown in FIG. 4c, and the scene determination unit 103 determines the size for each block, thereby indicating in which direction the open part is. Furthermore, it can be determined whether it is near or far.
Further, when the optical flow is not detected in the vicinity of the outer periphery, it can be determined that the opening portion has been approached, and the direction in which the opening portion is present can also be specified.

  It is difficult to measure the actual distance from the captured image because it depends on the size of the passage. However, as shown in FIG. 10, when using the sensor unit 101 with a horizontal viewing angle of 180 degrees in the imaging range, the outer peripheral part of the captured image is zero distance from the position of the sensor unit 101 with respect to the traveling direction. Obviously. Therefore, when the destination situation determination apparatus 1 travels in the traveling direction of FIG. 10, it is possible to determine the arrival at the open portion.

  The near-end situation determination apparatus 1 can detect whether the open portion is present in any direction of 360 degrees on the front side, not limited to the left and right sides.

  When it is detected that the optical flow is equal to or smaller than a predetermined size, a portion where the surface is interrupted is detected. Thus, it can be detected that another passage intersects the passage. Further, if the passage is considered as a road and the open portion is considered as an intersection, the situation ahead of the road can be determined. It becomes possible to use for the assistance for safe driving | running | working by mounting the prior situation determination apparatus 1 of this invention in a vehicle etc.

Hereinafter, the safe driving support device 10 in which the vehicle situation determination device 1 of the present invention is mounted on a vehicle will be described.
FIG. 5 is a configuration diagram of the safe driving support device 10 in which the notification unit 105 which is the notification unit in FIG.
The sensor unit 101 is a camera outside the vehicle attached to the front part of the vehicle, and has a horizontal viewing angle of 180 degrees.
An image captured by the sensor unit 101 is displayed on a display unit (not shown) of the navigation device 106 as needed, thereby assisting safe driving.
Specifically, by detecting the approach to the intersection by the configuration of the destination situation determination device 1 described above, and displaying the image captured by the sensor unit 101 when approaching the intersection on the display unit of the navigation device 106. , You can check the situation of the intersection.

FIG. 6 is an image in which the sensor unit 101 of the safe driving support apparatus 10 mounted on a vehicle or the like images the front of the vehicle.
FIG. 6a shows an intersection A1 with poor visibility due to a building or wall like B.
FIG. 6b shows an intersection A2 where one has poor visibility due to a building or wall like B and the other has good visibility like C.
FIG. 6c shows the intersection A3 with good visibility.
How the scene is determined from the optical flow calculated by the optical flow calculation unit 102 when the intersections A1 to A3 are imaged by the sensor unit 101 will be described.

FIG. 7a shows an optical flow calculated from image information obtained by imaging the intersection A1 shown in FIG. 6a.
Since the portion B is a building or a wall, a large optical flow is calculated. In the portion D, since a building or wall on both sides ahead of the intersection A1 is imaged from the front, a small optical flow is calculated.
If the purpose is to detect the intersection of roads, it is only the case of crossing in the left-right direction. Therefore, the detected optical flow is divided into blocks in the horizontal direction, and the size of the optical flow for each block is the destination. The existence of an intersection or doorway can be determined.

FIG. 8a shows the size of the optical flow in the lattice-like blocks G1 to G6 by the size of the area indicated by hatching. The average of the optical flow in the block may be the size of the optical flow in the block.
The optical flow threshold setting unit 104 sets a predetermined threshold indicated by S1 in FIG. 8A with respect to the size of the optical flow for each block calculated by the optical flow calculation unit 102. Based on the set threshold, the scene determination unit 103 determines the other situation.

A large optical flow can be detected in the blocks G1 and G6 around the image by the building or the wall in FIG. 6a. Since this is larger than the threshold value S1, it is determined that the imaging locations of G1 and G6 are surfaces parallel to the traveling direction.
The G2 and G5 blocks can detect only a small optical flow with respect to G1 and G2. Since this is smaller than the threshold value S1, it is determined that there is no surface parallel to the traveling direction.
Since G3 and G4 are imaged farther from the center, a large optical flow does not occur. However, if an optical flow larger than the threshold S1 is detected, the direction of travel is still beyond that. It is determined that there is a parallel surface.

  The scene determination unit 103 determines whether there is an intersection or an entrance at G2 and G5 from the change point of the optical flow, and whether the intersection or the entrance is the nearest or far away.

FIG. 7b shows an optical flow calculated from image information obtained by imaging the intersection A2 shown in FIG. 6b. Unlike FIG. 7a, FIG. 7b shows a smaller optical flow in area C. FIG.
Therefore, in FIG. 8b, since the magnitude of the optical flow of the block G1 is smaller than the threshold value S1, it is determined that there is no plane parallel to the traveling direction at the imaging location of G1.
Except for the block G1, it is the same as FIG. 8a.

  From these pieces of information, the scene determination unit 103 determines whether there is an intersection or doorway at G5 and whether the intersection or doorway is closest or far away. Further, on the left side, it is determined that the part has good visibility up to the position of G2.

FIG. 7c shows an optical flow calculated from image information obtained by imaging the intersection A3 shown in FIG. 6c. Unlike FIG. 7a, FIG. 7c shows a smaller optical flow in area C. FIG.
Therefore, in FIG. 8c, since the magnitudes of the optical flows of the blocks G1 and G6 are smaller than the threshold value S1, it is determined that there are no planes parallel to the traveling direction at the imaging positions of G1 and G6.
Except for blocks G1 and G6, this is the same as in FIG. 8a.

  From these pieces of information, the scene determination unit 103 determines that the left side is up to the position G2, and the right side is up to the position G5.

  If the vehicle equipped with the safe driving support device 10 is moving forward, the magnitude of the detected optical flow changes. Therefore, the safe driving support device 10 can determine the distance to the intersection based on the change in the size of the optical flow, and whether the intersection has good or bad visibility, and can support the safe driving of the driver who drives the vehicle. Become.

  By using the near-end situation determination apparatus 1, when approaching at least one of the left and right intersections where the line of sight is poor, the driver can be alerted to the intersection approach by an image, sound, or the like by the notification unit 105.

According to the safe driving support device 10 using the ahead situation determination device 1, the driver displays a front image on the monitor of the navigation device 106 when at least one of the left and right intersections is approached. Can judge the situation of the road which crosses at the intersection which cannot be seen directly from the image.
Further, when both the left and right approach an intersection with good visibility, it is not necessary to display the destination situation on the navigation device 106 because the situation of the road intersecting at the intersection can be seen.

  In addition, when approaching at least one of the left and right intersections, it is determined to switch the monitor to the intersection image for a predetermined time or a predetermined distance ahead of time, and the driver is notified by sound or synthesized sound. You may alert the intersection approach with the notification means.

Although the determination by the preceding situation determination device 1 in the safe driving support device 10 described above is the determination of an intersection that intersects in the horizontal direction, three-dimensional detection may be performed including the vertical direction.
For example, FIG. 9a is an image obtained when the vehicle exits on the road A4 from a space where the top, bottom, left, and right are closed, such as a garage, a tunnel, and an underground parking lot.
As shown in FIG. 9b, the optical flow is large because it is surrounded by walls on the top, bottom, left, and right, and is parallel to the traveling direction, and the optical flow is small because the front building or wall is perpendicular to the traveling direction.
The calculation of the optical flow may be performed for each block divided by the above-mentioned radial and concentric circles, but may be divided into a lattice shape and may be an average value of the size of the optical flow for each block.

FIG. 9c is a perspective view showing the optical flow size of G1: H1, G1: H2,... G6: H5 for each block in the height direction.
The scene determination unit 103 determines the other situation from the size of the optical flow for each block based on a predetermined threshold S2 set in the optical flow threshold setting unit 104. The threshold value S2 in FIG. 9c is represented by a plane having a predetermined height.
The height representing the optical flow size of the block at the outer periphery of the image is the highest.
In the image, G2: H4, G3: H4, G4: H4, and G5: H4 on the ground are somewhat distant from the outer periphery, and therefore the height indicating the size of the optical flow is slightly lower than the outer periphery. The magnitude of the optical flow at these locations is larger than the threshold value S2.
The front building or wall is in the area of (G2 to G5): (H2 to H3), but because of the perspective view of FIG. 9 (G2 to G5): H3 is not visible, but (G2 to G5): H2 It is about the same height. The size of the optical flow in this area is lower than the threshold plane S2.

Since there is a change in the magnitude of the optical flow between G1 and G2 and between G5 and G6, there is a part where the wall in the left-right direction is interrupted, and it is determined that the space is open ahead it can. This is the same as the intersection detection described in FIGS.
Furthermore, since there is a change in the optical flow between H1 and H2, it can be determined that there is a space where the ceiling is interrupted between H1 and H2 and opened upward.
For example, when the vehicle exits from the garage, when the vehicle is in the immediate vicinity of the exit, an image of the front captured by the safe driving support device 10 is displayed on the monitor of the navigation device, so that a falling object ( The presence or absence of falling rocks, flying objects from typhoons, etc. can be confirmed.

Of course, not only the upper part but also the lower part is effective in the following cases.
If it is just before going downhill or climbing uphill and the road ahead is flat or downhill, the vehicle's hood may hinder the visibility of the situation ahead. It is possible to support safe driving by judging such a situation and displaying the situation of the other party that cannot be viewed on the monitor in a timely manner.

In addition, the change of the ratio of the size that constitutes the optical flow when traveling at various intersections etc. in advance is statistically organized for each scene and accumulated, and this is compared with the actual traveling value It is also conceivable to determine the scene. The value for each scene that is stored in advance can be used with weight depending on the degree of effectiveness for scene determination by machine learning from several parameters such as travel speed, time zone, distance from intersection, etc. Good.

DESCRIPTION OF SYMBOLS 1 Previous situation determination apparatus 101 Sensor part, 102 Optical flow calculation part, 103 Scene determination part,
104 optical flow threshold setting unit, 105 notification unit 10 safe driving support device, 106 navigation device

Claims (5)

A sensor unit that images the other side of the traveling direction;
An optical flow calculation unit that calculates the size of the optical flow for each predetermined region of the image captured by the sensor unit;
A scene determination unit for determining the status of the destination based on a change in the magnitude of the optical flow for each region,   The scene determination unit, based on a change point of the magnitude of the optical flow, determines whether or not there is an opening that intersects and opens in the traveling direction. Item 3. The preceding situation determination device according to Item 1.   The destination situation determination according to claim 2, wherein the scene determination unit determines whether or not the opening existing in the destination has been reached based on movement of a change point of the magnitude of the optical flow. apparatus.   The preceding situation determination apparatus according to claim 1, further comprising a notification section that notifies with sound or an image according to the other situation determined by the scene determination section. A sensor unit for imaging in the direction of travel; and
An optical flow calculation unit that calculates the size of the optical flow for each predetermined region based on the image captured by the sensor unit;
A scene determination unit that determines the situation of the other side imaged from a change in the size of the optical flow for each area;
A monitor,
With
The said scene determination part displays the said image on the said monitor, when it determines with having approached the opening part which cross | intersects the advancing direction and is opened, The safe driving assistance apparatus characterized by the above-mentioned.

Images