JP6970910B2 - Mobile detection system for vehicles - Google Patents

Description

The present invention relates to a moving object detection system for a vehicle.

Conventionally, various systems have been proposed for detecting moving objects such as other vehicles and pedestrians by sensors such as cameras and radars that monitor the front and rear of the vehicle for the purpose of driving support (see, for example, Patent Document 1). ..

Japanese Unexamined Patent Publication No. 2015-143970

In such a system, the detection and recognition results of each sensor are collected in the integrated processing unit and filtered in the process of integrated judgment in the upper processing of measurement-recognition-judgment. Therefore, there is a problem that the judgment in the integrated processing unit becomes complicated and the hardware is required to have high processing power.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a moving object detection system for a vehicle with less burden on hardware.

As a result of diligent studies by the present inventors in order to solve the above problems, by sharing the detection information at the sensor level and using it for filtering, the burden of higher-level processing is small / higher-level processing is required. I came up with the present invention with the knowledge that it is possible to construct a system that does not.

That is, the moving object detection system for vehicles according to the present invention is
A first image pickup unit equipped with a function to detect a moving object in front,
A second imaging unit equipped with a function to detect moving objects in the rear or side, and
A controller capable of intercommunication with the first and second imaging units is provided.
The second image pickup unit has a function of transmitting an overtaking flag on the same side if the winker on the same side is not operated when a moving object having a predetermined speed or higher disappears on the right side or the left side of the field of view.
Wherein the controller, when receiving the overtaking flag has a function to forward the overtaking flag to the first imaging unit,
When the first imaging unit detects the appearance of a moving object at a predetermined speed or higher from the right side or the left side of the field of view, it issues an alarm on the same side if there is no overtaking flag on the same side within a predetermined time immediately before the detection. It has a function to notify and prohibit the issuance of an alarm if there is an overtaking flag on the same side within a predetermined time immediately before detection.

In the moving object detection system for a vehicle according to the present invention, mutual detection information (flag) is shared between the first and second image pickup units via the controller, so that the first image pickup unit can be the second image pickup unit. The detection information (passing flag) transferred from the image pickup unit can be used for filtering, and the controller only sends and receives mutual detection information between the first and second image pickup units, so the system has less hardware burden. Has the advantage of being able to build.

It is a block diagram which shows the moving body detection system which concerns on 1st Embodiment of this invention. It is a schematic diagram which shows the basic concept of the moving body detection system which concerns on this invention. It is a schematic diagram which shows the basic concept of filtering in the moving body detection system which concerns on this invention, (a) is a virtual front image, (b) is a left rear image, (c) is a front image, (d) is a right rear image. Is shown. It is a road plan view corresponding to FIG. 3 which shows the situation where the filtering in the moving body detection system which concerns on this invention is applied. It is a flowchart which shows the routine in a rear camera. It is a flowchart which shows the routine in the front camera. It is a flowchart which shows the routine in a controller. It is a block diagram which shows the moving body detection system which concerns on 2nd Embodiment of this invention.

(First Embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
In FIG. 1, the moving object detection system 10 for a vehicle according to the first embodiment of the present invention includes a front camera 12 (first image pickup unit) that images the front of the vehicle and a rear camera 13 (second image pickup) that images the rear of the vehicle. Unit), and a controller 11 capable of intercommunication with the front camera 12 and the rear camera 13.

The front camera 12 and the rear camera 13 both have an image sensor (an image sensor such as CMOS or CCD) that captures a field image imaged through an optical system, and image data obtained by the image sensor at a predetermined frame rate. In addition to being provided with an image processing unit that outputs a video signal to the controller 11, it also has a function of detecting a moving object from a time-series field image (field image) and a communication function of transmitting and receiving the detection information. These will be described later.

The controller 11 can acquire vehicle information 14 such as a vehicle speed, a steering angle, a winker switch, and a shift position sensor through an in-vehicle network (CAN). Further, the front camera 12 and the rear camera 13 are connected to each other via serial communication (UART) so as to be intercommunication with each other, and the video signal cables of the front camera 12 and the rear camera 13 are connected to the front camera 12 and the rear camera 12. It has a function of synthesizing the image of the camera 13 on a display device 15 such as a navigation device and displaying the image, or switching and displaying the image according to the shift position.

(Mobile detection function)
The front camera 12 and the rear camera 13 according to the present embodiment implement an image processing algorithm for extracting an optical flow of feature points from a time-series field image (field image) as a moving object detection function. That is, first, the feature points of each image frame are extracted. Feature points can be extracted as corners of the edges (contour lines, luminance boundaries) of the image. Next, by extracting the movement locus of the corresponding feature points between the two image frames before and after, the moving body can be detected as a feature point group having the same or similar vector components.

At this time, if the own vehicle is traveling forward, the road or fixed structure is detected as an optical flow toward the vanishing point in the image of the rear camera 13, and is detected as an optical flow away from the vanishing point in the image of the front camera 12. Therefore, by extracting an optical flow that has a vector component different from those and is recognized independently, a moving object can be detected as a group of feature points having the same or similar vector components.

Further, if the vehicle is stopped, the road and the fixed structure do not have the optical flow, so that all the detected optical flows indicate the moving body. In either case, the optical flow (vector) indicates the amount of movement of the moving body for each frame, and corresponds to the relative speed with respect to the own vehicle.

By the way, in the above processing, since the processing completion at the frame rate is prioritized, it does not have a function of extracting only the optical flow and recognizing an image of what the moving body is. Therefore, for example, when a moving body appears from the end of the field image of the front camera, it cannot be immediately distinguished whether it is a moving body that crosses the front of the own vehicle or a moving body that overtakes the own vehicle on the side.

For example, FIG. 2A schematically shows the extraction of the optical flow in the image 20 of the front camera 12 stopped at an intersection or the like, and the pedestrian 3 crossing the front of the road from the left side of the image 20 has a moving speed. Is slow, so it can be dealt with by detecting it as a moving object by the optical flow 3f in the image 20. However, at the moment when the vehicle 2'passing the right side appears from the right end of the image 20, is it a moving body that crosses the front of the own vehicle from right to left, or is it a moving body that overtakes the right side and moves away? It cannot be immediately distinguished from the optical flow 2f alone. If it is a moving object that goes away, it does not need to be the target of the alarm.

(Sharing and filtering of detection information between the rear camera and the front camera)
Here, if the vehicle 2 is a vehicle overtaking the right side, it should be reflected in the image 30 of the rear camera 13 as shown in FIG. 2B immediately before that. Therefore, when the optical flow 2r disappearing on the left side (right side in the traveling direction of the own vehicle) is detected in the image 30 of the rear camera 13, the moving speed of the moving body 2 is estimated from the optical flow 2r, and based on the optical flow 2r. , By predicting the time when the moving body 2 appears in the field image 20 of the front camera 12 and prohibiting the issuance of an alarm regarding the optical flow 2f appearing from the same side at that time, unnecessary issuance is prevented. can.

That is, by sharing the moving object detection information of the rear camera 13 in the front camera 12 and using it for filtering in the moving object detection of the front camera 12, higher-level processing for controlling the detection information of the front camera 12 and the rear camera 13 is performed. Filtering at the sensor level of the front camera 12 and the rear camera 13 is possible without the need for an integrated determination unit.

As a constraint condition for limiting the detection information to be shared in advance in order to realize such a filtering function, as described above.
i) Since the direction in which the overtaking vehicle disappears from the field image 30 of the rear camera 13 is fixed, the detection area (30L, 30R) can be limited to either the left or right side.
ii) Have a moving speed that distinguishes it from pedestrians and bicycles,
iii) It is assumed that the vehicle does not change course to the same side (the winker on the same side is not operated).

Although the detection areas 30L and 30R are schematically shown in FIGS. 2 and 3, detection information (passing flag described later) is detected when the optical flow disappears from either the left or right end of the field image 30 of the rear camera 13. ) Is issued, in other words, the appearance time on the same side of the field image 20 in the front camera 12 is calculated based on the movement speed estimated from the optical flow in the frame immediately before the disappeared frame. The area detected in is only a narrow area adjacent to either the left or right end.

Further, in FIG. 3, the field image (30) of the rear camera 13 is divided into left and right and displayed as left and right rear field images 31 and 32 on both sides of the field image 20 of the front camera 12. FIG. 4 shows the road condition corresponding to FIG. 3, in which the own vehicle 1 is stopped in the left lane 41 of the road toward the intersection 40, and the field of view 13a (field of view image 32) of the rear camera 13 shows. Another vehicle 4 trying to pass through the right lane 42 is shown, but it is not yet shown in the field of view 12a of the front camera 12 at this position.

In such a situation, as shown in FIG. 3D, when an optical flow 4r (moving body 4) having a predetermined speed or higher disappearing on the left side (right side in the traveling direction of the own vehicle) of the rear view image 32 is detected. If the right turn blinker operation is not performed, the overtaking flag is issued from the rear camera 13, and the alarm is prohibited for the right side detection area 20R of the field image 20 of the front camera 12 shown in FIG. 3 (c). The optical flow 4f appearing in the same area 20R is regarded as an overtaking vehicle 4'.

Further, as shown in FIG. 3B, if an optical flow 6r (moving body 6, bicycle) below a predetermined speed that disappears on the right side (left side in the traveling direction of the own vehicle) of the rear view image 31 is detected, turn left. No alarm will be issued unless the blinker operation is performed. Moreover, since the speed is less than the predetermined speed, it is not subject to the overtaking flag.

On the other hand, as shown in FIG. 3C, when an optical flow 3f (moving body 3, pedestrian) having a speed lower than a predetermined speed is detected on the left side of the front camera 12 (field image 20), the overtaking flag on the same side is displayed. Since it has not been issued, an alarm will be issued.

Further, as shown in FIG. 3A, in a situation where the detection information (passing flag) associated with the right side detection area 20R is not issued from the rear camera 13, the field image 22 of the front camera 12 is optical. When the flow 5f is detected, an alarm is issued assuming that the moving body 5 is from the front right side other than the right rear side, and therefore the front right side.

Hereinafter, examples of operation routines of the rear camera 13, the front camera 12, and the controller 11 constituting the mobile body detection system 10 will be described with reference to FIGS. 5 to 7.

(Routine in the rear camera)
In FIG. 5, the mobile object search (feature point extraction-optical flow extraction) by the rear camera 13 is started at the same time as the mobile object detection system 10 is activated (step 100).

First, the feature points are extracted for each image frame captured by the rear camera 13, and the corresponding feature points are moved with respect to the immediately preceding frame, that is, the optical flow is extracted, and the vehicle travels on the left or right side of the traveling lane of the own vehicle. The presence or absence of a moving object (optical flow) in the lane is checked (step 101).

If the optical flow independent of the road or fixed structure is not detected, or if the optical flow is detected only in the center of the rear view image 30, the mobile search is continued.

When a moving object (optical flow) is detected in the left or right driving lane of the own vehicle, vehicle information is acquired via the controller 11 (step 102/103), and the corresponding left / right winker. When is operated, the left rear / right rear alarm flag is transmitted from the rear camera 13 to the controller 11 (step 106/107).

When the disappearance of the moving object (optical flow) to the corresponding left / right end in the field image is detected when the corresponding left / right winker is not operated (step 111), the left / right overtaking flag , And the estimated vehicle speed of the moving object immediately before disappearance is transmitted from the rear camera 13 to the controller 11 (step 112/113).

(Routine in front camera)
In FIG. 6, the mobile object search (feature point extraction-optical flow extraction) by the front camera 12 is started at the same time as the mobile object detection system 10 is activated (step 120).

Feature points and optical flows are extracted for each image frame captured by the front camera 12, and the appearance of moving objects (optical flows) from the left or right end of the field image 20 is checked (step 121). ). If no optical flow independent of the road or fixed structure is detected, the mobile search will continue.

When the appearance of a moving object (optical flow) from the left or right end of the field image 20 is detected, the presence or absence of the corresponding left / right overtaking flag and vehicle information such as steering angle and vehicle speed are used as the controller. Referenced via 11 (step 122/123).

If the corresponding left / right overtaking flag is not displayed (step 124/125), does the vehicle status of the own vehicle match the notification conditions for the moving object appearing from the corresponding left / right side? Whether or not it is checked (step 126/127).

If they match, the left front / right front warning flag is transmitted to the controller 11 (step 128/129), and if they do not match, the mobile search is continued.

On the other hand, if the corresponding left / right overtaking flag is issued in step 124/125, it is checked whether or not the corresponding left / right overtaking flag was issued in the previous loop (step 130). / 131).

If the corresponding left / right overtaking flag was not issued in the previous loop, the notification prohibition timer is reset (step 132/133), and the notification prohibition timer starts counting.

If the corresponding left / right overtaking flag is issued in the previous loop and the count of the alarm prohibition timer is started, the count of the alarm prohibition timer is continued.

When the count of the alarm prohibition timer is completed (134), the moving object search by the front camera is started again.

(Routine in controller)
In FIG. 7, the monitoring of vehicle information by the controller 11 is started at the same time as the activation of the mobile detection system 10 (step 140).

When the left / right overtaking flag and the estimated vehicle speed are received from the rear camera 13 (step 141), the corresponding left side of the field image 20 of the front camera 12 is based on the corresponding left / right overtaking flag and the estimated vehicle speed. / Calculate the time when a moving object (passing vehicle) is expected to appear on the right side (step 142/143).

Next, vehicle information such as the left / right overtaking flag, the predicted appearance time, the steering angle, and the vehicle speed is transferred to the front camera 12 (step 144/145).

When the alarm flag is received (step 150), an alarm is issued corresponding to the position of the alarm flag (left front / left rear / right front / right rear) (step 151).

(Second Embodiment)
In the mobile detection system 10 of the first embodiment described above, the case where the controller 11 is connected to the front camera 12 and the rear camera 13 via serial communication (UART) so as to be intercommunication is shown. In the mobile body detection system 10'in the form, the controller 11 is connected to the front camera 12 and the rear camera 13 via a local area network (LAN) so as to be able to communicate with each other. Other configurations are the same as in the first embodiment.

In the moving object detection system 10'of the second embodiment, the left or right overtaking flag transmitted from the rear camera 13 is shared by the controller 11 and the front camera 12 at the same time, and the appearance prediction time and the vehicle calculated by the controller 11 are shared. Information can also be transmitted to the front camera 12.

Further, in any of the embodiments, the controller 11 can count / reset (steps 132 to 134) the alarm prohibition timer of the front camera 12 when the overtaking flag is issued, and the rear camera 13 can also be used. The acquisition of vehicle information (step 102/103) and the blinker confirmation (step 104/105) in the above can also be performed by the controller 11. In these cases, the front camera 12 and the rear camera 13 exclusively detect and issue a flag, and the controller 11 issues an alarm and prohibits the issue by the overtaking flag.

In each of the above embodiments, the case where the front camera 12 (first image pickup unit) and the rear camera 13 (second image pickup unit) are used is shown, but the front camera 12 (first image pickup unit) and the left and right sides are shown. A camera (second imaging unit) can also be used to configure the system.

Although some embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications and modifications can be made based on the technical idea of the present invention. To add.

1 Own vehicle 2, 4, 5 Other vehicle 3 Pedestrian 6 Bicycle 10, 10'Mobile detection system 11 Controller 12 Front camera (first imaging unit)
13 Rear camera (second imaging unit)
14 Vehicle information 15 Display devices 20, 22 Front camera image (field of view image)
30, 31, 32 Rear camera image (field of view image)

Claims (5)

A first image pickup unit equipped with a function to detect a moving object in front,
A second imaging unit equipped with a function to detect moving objects in the rear or side, and
A controller capable of intercommunication with the first and second imaging units is provided.
The second image pickup unit has a function of transmitting an overtaking flag on the same side if the winker on the same side is not operated when a moving object having a predetermined speed or higher disappears on the right side or the left side of the field of view.
Wherein the controller, when receiving the overtaking flag has a function to forward the overtaking flag to the first imaging unit,
When the first imaging unit detects the appearance of a moving object at a predetermined speed or higher from the right side or the left side of the field of view, it issues an alarm on the same side if there is no overtaking flag on the same side within a predetermined time immediately before the detection. A moving object detection system for vehicles that has the function of notifying and prohibiting the issuance of an alarm if there is an overtaking flag on the same side within a predetermined time immediately before detection. A first image pickup unit equipped with a function to detect a moving object in front,
A second imaging unit equipped with a function to detect moving objects in the rear or side, and
A controller capable of intercommunication with the first and second imaging units is provided.
The second imaging unit, when the right or left side above a predetermined speed of the moving body of the visual field is lost, a function of transmitting the overtaking flag ipsilateral,
Wherein the controller, when receiving the overtaking flag has a function to forward the overtaking flag if the winker ipsilateral unactuated to said first imaging unit,
When the first imaging unit detects the appearance of a moving object at a predetermined speed or higher from the right side or the left side of the field of view, it issues an alarm on the same side if there is no overtaking flag on the same side within a predetermined time immediately before the detection. A moving object detection system for vehicles that has the function of notifying and prohibiting the issuance of an alarm if there is an overtaking flag on the same side within a predetermined time immediately before detection. The function of detecting the moving object of the first and second imaging units is implemented as a means for extracting the optical flow of the images captured by the first and second imaging units, claim 1 or 2. 2. The moving object detection system for vehicles described. The second image pickup unit further has a function of estimating a relative velocity from the optical flow of the moving body and transmitting it to the controller when the moving body disappears.
3. The controller further has a function of calculating a predicted time for the moving object to appear in the first image pickup unit based on the relative speed and transferring the predicted time to the first image pickup unit. The described vehicle moving object detection system. The controller is provided integrally with either the first or second image pickup unit, or is mounted as a program module in the control unit of either the first or second image pickup unit. Item 4. The moving object detection system for a vehicle according to any one of Items 1 to 4.

Images