JP2015158874A - Perimeter monitoring apparatus and perimeter monitoring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a perimeter monitoring apparatus and a perimeter monitoring method capable of improving accuracy for discriminating a moving object.SOLUTION: A plurality of specific feature points are tracked from each of images temporally continuously obtained by a camera imaging peripheries of a vehicle, and the specific feature points are grouped in 110. If it is determined in a first determination process that the vehicle is in the process of turning, and it is determined in a second determination process that a predetermined number or more of specific feature points out of the grouped specific feature points have a vector directed toward the vehicle, an object relating to the specific feature points is not detected as a notification target moving object in 118. If a result of the second determination process indicates that the grouped specific feature points have vectors in a plurality of directions, the object relating to the grouped feature points is detected as a notification target moving object in 116.

Description

  The present invention relates to a periphery monitoring apparatus and method.

  When the vehicle is turning right, the nose view monitor device that does not refer to the flow vector on the screen in the right side area and does not refer to the flow vector on the screen in the left side area when the vehicle is turning left. It is known (see, for example, Patent Document 1).

JP 2005-276057

  However, in the configuration described in Patent Document 1 above, the flow vector on the screen of the side area related to the turning direction is not uniformly referenced, so when there is a moving object in the side area related to the turning direction, There is a possibility that the accuracy of determining a moving object is lowered.

  The present invention provides a periphery monitoring apparatus and method with improved accuracy in determining a moving object.

A periphery monitoring device according to an aspect of the present invention includes a camera that captures an image of the periphery of a vehicle,
First specific determination as to whether or not the vehicle is turning by tracking a plurality of specific feature points from each image obtained by continuously capturing images with the camera in time, grouping the plurality of tracked specific feature points When the first determination process determines that the vehicle is turning, a vector in which a predetermined number or more of the plurality of specific feature points that are grouped is directed in the direction toward the vehicle. A second determination process is performed to determine whether or not a predetermined number or more of the plurality of grouped specific feature points is directed toward the vehicle as a result of the second determination process. In the case of having a vector, the objects related to the plurality of grouped specific feature points are not detected as the moving object to be notified, and as a result of the second determination process, the plurality of grouped specific feature points are If having a vector oriented in the direction, and a processing unit for detecting an object according to a plurality of specific feature points to the grouping as the movement of the notification object.

  According to the present invention, it is possible to obtain a periphery monitoring apparatus and method with improved accuracy for determining a moving object.

The block diagram of the periphery monitoring apparatus which is one Example of this invention is shown. The figure showing an example of the detection area for the moving object detection set in the image | video which the camera image | photographed with the periphery monitoring apparatus of a present Example is shown. In the periphery monitoring device of the present embodiment, a case where a network structure (grating) in which a periodic pattern for closing a side groove on a road surface is repeated is included in a camera image is shown. The flowchart of an example of the control routine which the image | video recognition part of ECU performs in the periphery monitoring apparatus of a present Example is shown. The example of the vector of the feature point grouped into the same group is shown. The example of the vector of the feature point grouped into the same group is shown.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

  FIG. 1 shows a configuration diagram of a periphery monitoring apparatus 10 according to an embodiment of the present invention. The periphery monitoring device 10 according to the present embodiment is a device that monitors, for example, the periphery of the vehicle mounted on the vehicle, and detects an obstacle present around the vehicle. The periphery of the vehicle is, for example, an area that becomes a blind spot for the driver of the vehicle. The obstacle is, for example, a moving object such as another vehicle or a person that moves relative to the host vehicle.

  As shown in FIG. 1, the periphery monitoring device 10 includes a camera 12 that captures the periphery of the vehicle. The camera 12 has a relatively wide angle lens or fisheye lens. The camera 12 is disposed at the front of the vehicle, such as a bumper or grill at the front, a lower spoiler at the rear of the vehicle, or below the back glass. The camera 12 is directed to the front of the vehicle body or the rear of the vehicle body and obliquely downward, and can capture an image within a predetermined range including a position that becomes a blind spot especially for the vehicle driver. The camera 12 is a digital camera that captures the predetermined range with a predetermined pixel.

  The periphery monitoring device 10 includes an electronic control unit (hereinafter referred to as an ECU) 14 mainly composed of a microcomputer. The above camera 12 is connected to the ECU 14. Video information captured by the camera 12 is supplied to an ECU 14 that performs image processing. The ECU 14 has a video recognition unit 16 to which video information of the camera 12 is input. As will be described in detail later, the video recognition unit 16 detects at least a moving object by recognizing an image of the vehicle periphery supplied from the camera 12 based on the optical flow method.

  The ECU 14 has an image drawing processing unit 18 to which a detection result in the video recognition unit 16 is input. A monitor 20 provided in the passenger compartment is connected to the image drawing processing unit 18. The monitor 20 is disposed on, for example, a console panel or a multi-information display that can be visually recognized by the vehicle driver. Based on the detection result from the video recognition unit 16, the image drawing processing unit 18 provides notification of an image drawn on the monitor 20 and a warning displayed on the monitor 20.

  For example, the image drawing processing unit 18 performs image processing of video captured by the camera 12. For example, a specific portion of the video is cut out or distortion correction of the video is executed. The image drawing processing unit 18 superimposes and displays the detection result by the video recognition unit 16 on the video captured by the camera 12. For example, the approaching direction of the moving object to the own vehicle based on the detection result is displayed, or the moving object is highlighted. The image drawing processing unit 18 causes display according to user settings by the vehicle driver. For example, display is performed in accordance with a display method for notifying the detection status or selection of an object. The monitor 20 performs display in accordance with a display command from the image drawing processing unit 18.

  A vehicle speed sensor 40, a steering angle sensor 42, and a yaw rate sensor 44 are connected to the ECU 14.

  FIG. 2 is a diagram illustrating an example of a detection area for detecting a moving object set in an image captured by the camera 12 in the periphery monitoring device 10 according to the present embodiment. 3 shows a camera image 32 of a mesh-like structure 30 (hereinafter referred to as “periodic structure 30”) in which a periodic pattern for closing a side groove on the road surface is repeated in the periphery monitoring device 10 of the present embodiment. The case where it is included is shown. The periodic structure 30 is, for example, grating. FIG. 4 shows a flowchart of an example of a control routine executed by the video recognition unit 16 of the ECU 14 in the periphery monitoring device 10 of the present embodiment. 5 and 6 are diagrams showing examples of feature point vectors grouped in the same group, FIG. 5 shows a case of feature points of a moving object, and FIG. 6 shows a periodic structure 30. Show the case. The moving object is, for example, a pedestrian. 5 and 6, black circles conceptually represent a set of feature points.

  Hereinafter, description will be given with reference to the flowchart of FIG.

  In this embodiment, the video information captured and supplied by the camera 12 is input to the video recognition unit 16 of the ECU 14 (step 100). When the video information from the camera 12 is input, the video recognition unit 16 recognizes the video from the camera 12 based on the optical flow method as follows. Note that the image recognition by the video recognition unit 16 is repeatedly performed every predetermined time. The image recognition by the video recognition unit 16 is performed for each of a plurality of videos obtained at different timings that are temporally continuous.

  Next, when performing video image recognition, the video recognition unit 16 first sets a detection area that needs to be processed for moving object detection in the input video (step 102). For example, as shown in FIG. 2, in the input video, on the XY coordinates, a point (X_L1, Y_L1), a point (X_L2, Y_L1), a point (X_L1, Y_L2), and a point (X_L2, Y_L2), the region L, and the region R surrounded by the point (X_R1, Y_R1), the point (X_R2, Y_R1), the point (X_R1, Y_R2), and the point (X_R2, Y_R2) Two areas are set as detection areas.

  The detection area to be set may be the entire area of the video captured by the camera 12. In addition, the detection area to be set is only a part of the entire area of one image except the area where the body of the vehicle is reflected in consideration of speeding up and efficiency of image processing. May be. The area in which the vehicle body of the host vehicle is reflected is an area in which, for example, a bumper is reflected. The detection area may be determined in advance based on the directivity of the camera 12 or the blind spot of the driver. In addition, when the detection area is limited to a part of the entire video area, the detection area has a relatively large influence on the traveling of the host vehicle if an obstacle exists in the detection area, for example. It may include a region in front of where a position close to the host vehicle is projected within and a region of left and right end portions in which a position closer to the center is projected when moving the host vehicle. The time when the host vehicle is moving is, for example, when turning.

  The video recognition unit 16 removes noise from the image of the detection area by leveling the image of the detection area set as described above in the entire area of the input video using a noise removal filter (step 104). ). Then, by performing edge extraction using the edge extraction filter on the leveled detection area image, feature points are extracted from the detection area image (step 106).

  Next, the image recognizing unit 16 uses a plurality of temporally continuous images including the image from which the feature points have been extracted in the above step 106, and determines in advance the area near the feature points extracted in the above step 106 as a target. A feature point that moves at a predetermined speed range is identified and tracked (step 108). Note that tracking means specifying the same feature point in the video sequentially obtained in time series. The upper limit value of the predetermined speed range may be varied according to the vehicle speed. Specifically, after matching the feature points constituting the same object extracted from the past video with the feature points extracted in the above step 106, based on the difference in the coordinate positions of these feature points. Then, the movement vector amount of the feature points constituting the same object is measured, and the feature points moving at a speed in the predetermined speed range are extracted. A feature point whose number of times of tracking exceeds a predetermined number of times (hereinafter referred to as a feature point tracking number threshold) is a feature point to be grouped in the next step 110. The number of times of tracking may be counted by the number of frames of the video of the camera 12, for example.

  Note that the feature point tracking number threshold may be fixed regardless of the direction of the movement vector of the feature point. Also, the feature point tracking number threshold may be varied according to the direction of the feature point movement vector. In this case, the feature point tracking frequency threshold may be set larger when the direction of the movement vector of the feature point is the predetermined direction than when the direction is the other direction. As a result, the timing at which the feature points related to the fixed object on the road surface such as the periodic structure 30 can be grouped can be delayed. The predetermined direction corresponds to a direction in which a feature point related to a fixed object on the road surface such as the periodic structure 30 flows in the image, and may be set according to the steering angle of the vehicle.

  After the processing of step 108, the video recognition unit 16 is assumed to constitute the same moving object among the feature points moving at a speed in the predetermined speed range tracked as described above in the entire detection area of the camera video. The feature points are grouped together and tracked (step 110). Note that “tracking” means specifying the same grouped feature points in the video sequentially obtained in time series. A feature point related to a group in which the number of times the grouped feature points have been tracked (the number of frames of the video of the camera 12) exceeds a predetermined number (hereinafter referred to as a grouping tracking number threshold) is a notification target group (feature point) .

  The grouping tracking number threshold may be fixed regardless of the direction of the movement vector of the feature points. Further, the grouping tracking number threshold may be varied in accordance with the direction of the feature point movement vector. In this case, the grouping tracking number threshold may be set larger when the direction of the movement vector of the feature point is the predetermined direction than when the direction is the other direction. Thereby, even if the feature point which concerns on the fixed object on the road surface like the periodic structure 30 is not excluded temporarily in step 118 mentioned later, the alerting | reporting timing can be delayed. The predetermined direction corresponds to a direction in which a feature point related to a fixed object on the road surface such as the periodic structure 30 flows in the image, and may be set according to the steering angle of the vehicle.

  The grouping is performed so that an image area where feature points moving at a speed in a predetermined speed range among all the pixels in the detection area in the camera image are densely formed is formed. The video region is, for example, a video region having a predetermined number or more of feature points per predetermined width or a video region having a density of feature points or more of a predetermined density. In addition, the image area formed by grouping as described above may be limited to a predetermined shape on a camera image such as a rectangular shape or a square shape. Further, the grouping performed in the entire area of the camera video detection area may be divided into two or more areas. For example, if a plurality of moving objects exist apart, grouping is performed for each moving object. . That is, a plurality of groups are formed.

  Then, the image recognition unit 16 determines whether the vehicle is turning within a predetermined vehicle speed range based on the output signals from the vehicle speed sensor 40, the steering angle sensor 42, and the yaw rate sensor 44 after the processing of step 110. Is determined (step 112). The predetermined vehicle speed range is a low speed range larger than 0, and may be, for example, a range of 0 to 8 km / h. Whether or not the vehicle is turning may be determined based on an output signal from one of the rudder angle sensor 42 and the yaw rate sensor 44, or map data of a navigation device (not shown) and the own vehicle. The determination may be made based on the relationship with the position. If the vehicle is turning within the predetermined vehicle speed range, the routine proceeds to step 114. On the other hand, if the vehicle is not turning within the predetermined vehicle speed range, the routine proceeds to step 116.

  In step 114, the video recognition unit 16 determines whether or not all the movement vectors of the feature points belonging to the same group are directed in a predetermined direction corresponding to the turning direction. The predetermined direction is a direction toward the vehicle, and may be set according to the turning direction, the vehicle speed, and the like. The direction toward the vehicle may be a direction toward the center of the image of the camera 12, for example. The turning direction may be determined based on the steering angle. For simplicity, when turning right, the predetermined direction may be 45 degrees downward diagonally to the left, and when turning left, the predetermined direction may be downward 45 degrees diagonally right. The predetermined direction does not need to be a specific angle, and may be a direction having an angle range. If all the movement vectors of the feature points belonging to the same group are directed in a predetermined direction corresponding to the turning direction, the process proceeds to step 118; otherwise, the process proceeds to step 116.

  In step 116, the video recognition unit 16 determines that a group in which all of the movement vectors of each feature point are not directed in a predetermined direction corresponding to the turning direction is a group related to the moving object. When such moving object determination is made, the video recognition unit 16 supplies the detection result to the drawing processing unit 18, and the drawing processing unit 18 drives the presence of the moving object based on the detection result from the video recognition unit 16. By performing a process for visually informing the driver, a notification such as a warning display for notifying the driver that there is a moving object around the vehicle is given to the monitor 20. At the same time as this visual notification, an acoustic notification such as a warning sound or a buzzer may be generated to audibly notify the driver that there is a moving object around the host vehicle. . In the case of step 116, in the example shown in FIGS. 5 and 6, all of the groups 72L, 72R, 70L, and 70R are notification targets.

  In step 118, the video recognition unit 16 determines that the group in which all the movement vectors of each feature point are in the predetermined direction corresponding to the turning direction is not a group related to the moving object. Specifically, the video recognition unit 16 determines that the group in which all the movement vectors of the feature points are in a predetermined direction corresponding to the turning direction is the periodic structure 30. In this case, the periodic structure 30 is not a notification target. That is, when the non-moving object determination or the periodic structure determination is performed, the image recognition unit 16 supplies the detection result to the drawing processing unit 18 so that the drawing processing unit 18 informs the driver of the presence of the moving object. Do not perform processing for visual notification. That is, the accuracy of determining a moving object is improved.

  By the way, since the pedestrian which is an example of a moving object has a movement of limbs, the movement vector of each feature point in the groups 70L and 70R related to the pedestrian is directed in various directions as shown in FIG. In FIG. 5, each feature point in the group 70 </ b> L has three-direction movement vectors in the lower right direction, right right side, and upper right direction, and the same applies to the group 70 </ b> R. On the other hand, since the periodic structure 30 is a fixed object on the road surface and does not have irregular movement like a pedestrian, the movement vector of each feature point related to the periodic structure 30 is as shown in FIG. In addition, everything is in a predetermined direction. Specifically, during a left turn, in the group 72L related to the periodic structure 30 located on the left front side of the vehicle, the movement vectors of the feature points are all in the same diagonally downward direction. Further, during a right turn, in the group 72R related to the periodic structure 30 located on the right front side of the vehicle, the movement vectors of the feature points are all in the same diagonally downward left direction.

  Therefore, according to the process shown in FIG. 4, the groups 72R and 72L related to the periodic structure 30 can be accurately excluded from the notification target. That is, during the left turn, the group 72L related to the periodic structure 30 located on the left front side of the vehicle receives an affirmative determination in step 114 and is excluded from the notification target. At this time, the other groups 72R, 70L, and 70R are notification targets. Further, during a right turn, the group 72R related to the periodic structure 30 located on the right front side of the vehicle receives an affirmative determination in step 114 and is excluded from the notification target. At this time, the other groups 72L, 70L, and 70R are notification targets.

  In the process shown in FIG. 4, it is determined whether or not all the movement vectors of the feature points belonging to the same group are directed in a predetermined direction corresponding to the turning direction. Therefore, whether or not the movement vectors of a predetermined number or more of the feature points belonging to the same group are, for example, whether the movement vectors of most of the feature points are in a predetermined direction corresponding to the turning direction. You may judge. The majority may be 90% or more, for example. Also, the variance value in the direction of the movement vector of each feature point is calculated, and it is determined whether or not the variance value is within a predetermined value and the average direction of the movement vector of each feature point is in the predetermined direction. May be.

  Although each embodiment has been described in detail above, it is not limited to a specific embodiment, and various modifications and changes can be made within the scope described in the claims. It is also possible to combine all or a plurality of the components of the above-described embodiments.

10 Perimeter monitoring device 12 Camera 14 Electronic control unit (ECU)
16 Image recognition unit 20 Monitor 30 Periodic structure

Claims (3)

A camera that images the surroundings of the vehicle;
First specific determination as to whether or not the vehicle is turning by tracking a plurality of specific feature points from each image obtained by continuously capturing images with the camera in time, grouping the plurality of tracked specific feature points When the first determination process determines that the vehicle is turning, a vector in which a predetermined number or more of the plurality of specific feature points that are grouped is directed in the direction toward the vehicle. A second determination process is performed to determine whether or not a predetermined number or more of the plurality of grouped specific feature points is directed toward the vehicle as a result of the second determination process. In the case of having a vector, the objects related to the plurality of grouped specific feature points are not detected as the moving object to be notified, and as a result of the second determination process, the plurality of grouped specific feature points are If having a vector oriented in the direction, and a processing unit for detecting an object according to a plurality of specific feature points to the grouping as the movement of the notification target, the peripheral monitoring unit.   When all of the plurality of grouped specific feature points have a vector directed in a direction toward the vehicle while the vehicle is turning, the processing device is configured to notify the grouped objects related to the plurality of specific feature points as the notification target. The periphery monitoring device according to claim 1, which is not detected as a moving object. Tracking a plurality of specific feature points from each image obtained by continuously capturing images with a camera that captures the periphery of the vehicle,
Grouping the plurality of specific feature points tracked;
Performing a first determination process as to whether or not the vehicle is turning,
If it is determined by the first determination process that the vehicle is turning, whether or not a predetermined number or more of the plurality of grouped specific feature points has a vector directed in the direction toward the vehicle. The second determination process of
As a result of the second determination process, when a predetermined number or more of the plurality of grouped specific feature points has a vector directed in the direction toward the vehicle, the grouped specific feature points Without detecting the object as the moving object to be notified,
As a result of the second determination processing, when the plurality of grouped specific feature points have vectors directed in a plurality of directions, the objects related to the plurality of grouped specific feature points are detected as moving objects to be notified. Peripheral monitoring method including.

Images