JP4140588B2 - Moving object detection device - Google Patents

Description

  The present invention relates to a moving object detection apparatus.

Conventionally, a movable space extraction unit that extracts a movable area where no obstacle exists from a captured image, a moving obstacle extraction unit that extracts a moving obstacle, and an obstacle based on the output of these extraction units 2. Description of the Related Art A moving object detection device is known that detects the jumping out of an obstacle by obtaining the position and direction. In this apparatus, when detecting the jumping out of a moving object, an optical flow is detected, and a vanishing point of the optical flow is detected (for example, refer to Patent Document 1).
JP-A-9-259282

  However, in the conventional apparatus, the optical flow must be detected to find the vanishing point, and the processing load is not small, resulting in an increase in cost for hardware.

  The present invention has been made to solve such conventional problems, and an object of the present invention is to provide a moving object detection apparatus that reduces the processing load and prevents an increase in cost.

The moving object detection apparatus of the present invention includes an imaging means, an edge detection means, an edge movement history creation means, a speed direction detection means, and a pop-up detection means. The image pickup means picks up either the front or the side of the vehicle with a single eye , the edge detection means determines the image direction from the image pickup direction of the image pickup means , and performs edge processing based on the image direction taken by the image pickup means Is. The edge movement history creating means inputs the image edge-processed by the edge detecting means and records the edge position in the image along with time information in time series. The speed direction detection means is configured to detect the edge movement speed and direction from the edge position and time information recorded in time series by the edge movement history creation means. The pop-up detection means detects the pop-out of the moving object based on the moving speed and the moving direction detected by the speed direction detecting means.

  According to the present invention, edge processing is performed on a captured image, and the moving speed and moving direction of the edge are detected. For this reason, the moving speed and moving direction of an object etc. are detected. Then, the jumping out of the moving object is detected from the detected moving speed and moving direction of the edge. Therefore, the jumping out of the moving object is detected based on the moving speed and moving direction of the object. As described above, in the present invention, since processing with a large burden such as calculation of an optical flow and calculation of a vanishing point is not executed, only the edge is detected and its movement is judged, so that the processing burden is reduced and the cost is reduced. It is possible to provide a moving object detection device that prevents the increase.

  Preferred embodiments of the present invention will be described below with reference to the drawings. In the drawings, the same or similar elements are denoted by the same reference numerals, and description thereof is omitted.

  FIG. 1 is a configuration diagram of a moving object detection device according to an embodiment of the present invention. A moving object detection apparatus 1 shown in the figure detects a moving object such as a person, an animal, or a vehicle, and is preferably mounted on a car and used to warn the driver of the moving object popping out. .

  Specifically, the moving object detection apparatus 1 includes a camera (imaging unit) 10 that captures an image around the vehicle, and an edge detection unit (edge detection unit) 20 that performs edge processing on an image captured by the camera 10.

  Further, the moving object detection device 1 receives an image subjected to edge processing by the edge detection unit 20 and records an edge position in the image in time series together with time information (edge movement history creation means). 30 and a speed detecting unit (speed direction detecting means) 40 for detecting the moving speed and moving direction of the edge from the edge position and time information recorded in time series by the edge moving history calculating unit 30.

  Furthermore, the moving object detection device 1 is based on the moving speed and the moving direction detected by the speed detecting unit 40, and a dangerous object detection unit (jumping detection means) 50 that detects the jumping out of the moving object, and the dangerous object detection unit 50. When a jumping out of a moving object is detected by the above, an alarm unit 60 for notifying that effect by voice or the like is provided.

  That is, the moving object detection device 1 is configured to detect an edge, determine the moving speed of the edge, and detect the jumping out of the moving object.

Here, the edge detection unit 20 is configured to detect an edge extending in a predetermined image direction. This predetermined image direction is determined by the imaging direction of the camera 10. Usually, the jumping out of the moving object occurs from the side of the vehicle. For this reason, the edge detection unit 20 determines to detect an edge extending in the vertical direction of the image so that the movement of the object can be detected in the horizontal direction of the image when the camera 10 images the front of the vehicle. That is, the edge detection unit 20 detects an edge extending in the image vertical direction based on a change in brightness in the image horizontal direction. Further, when the camera 10 is imaging the side of the vehicle, the edge detection unit 20 determines to detect an edge extending in the lateral direction of the image so that an object approaching the host vehicle can be detected. That is, the edge detection unit 20 detects an edge extending in the horizontal direction of the image based on a change in brightness in the vertical direction of the image.

  Further, the edge movement history calculation unit 30 has a function of initializing the edge position and time information recorded in time series. This initialization is based on at least one of information on the steering angle, information from the yaw rate sensor, information on the difference between the left and right wheels, information on the turn signal, and information on the road map and the vehicle position. Done. That is, when it is detected that the imaging direction itself has changed due to the turning angle of the handle or the like, the edge movement history calculation unit 30 uses the edge position and time information recorded in the past to make sense for the detection of the moving object. In order to avoid this, initialization is performed.

  Further, the speed detection unit 40 is configured to set a speed detection area when detecting the edge moving speed and detect the moving speed from the edge in this area. Further, the speed detection unit 40 is configured to input at least one information among weather information, time zone information, and luminance information in the image. The speed detection unit 40 enlarges the speed detection area when a decrease in the SN ratio of the image is detected based on the input information, that is, when the influence of noise becomes strong. As a result, the speed detection unit 40 increases the amount of edge information to reduce the influence of false detection due to noise.

  In addition, the dangerous object detection unit 50 is based on the position information of the host vehicle and at least one of the width of the road on which the host vehicle travels, the presence or absence of a sidewalk, the number of lanes, weather information, and time zone information. Thus, it is configured to determine whether or not the object detected as the moving object is noise. For example, when the host vehicle is traveling on the right lane on a two-lane road on one side, it is highly likely that the moving object will jump out from the left side. For this reason, the dangerous substance detection unit 50 determines that there is a high possibility of noise when it detects the jumping out of the moving object from the right side. On the other hand, the dangerous substance detection unit 50 determines that it is not noise when it detects the jumping out of the moving object from the left side. In this way, the dangerous object detection unit 50 is configured to determine the pop-out occurrence probability and perform appropriate pop-out detection.

  Next, the operation of the moving object detection device 1 according to this embodiment will be described. In the following description, it is assumed that the camera 10 captures the vehicle front direction, and the edge detection unit 20 detects the edge extending in the image vertical direction.

  First, the camera 10 captures an image around the vehicle and acquires an image in the vehicle front direction. Next, the edge detection unit 20 performs edge processing and detects an edge extending in the vertical direction of the image. And the camera 10 and the edge detection part 20 repeat the said process, and detect an edge one by one.

  Next, the edge movement history calculation unit 30 sequentially inputs the images subjected to the edge processing by the edge detection unit 20, and records the edge positions in the images in time series together with time information.

  Here, the edge position and time information recorded in the edge movement history calculation unit 30 will be described. FIGS. 2A and 2B are explanatory diagrams showing the state of recording by the edge movement history calculation unit 30, wherein FIG. 2A shows the state of recording at time t-2, and FIG. 2B shows the state of recording at time t-1. (C) shows the state of recording at time t.

  First, as shown in FIG. 2A, if an edge is detected by the edge detection unit 20 at a position x1 at time t-2, the edge movement history calculation unit 30 records the edge position x1 as a movement history. . Further, the edge movement history calculation unit 30 gives time information h for the edge position x1. At this time, the edge movement history calculation unit 30 sets “time information h = 1” for the edge position x1.

  Next, as shown in FIG. 2B, if an edge is detected by the edge detection unit 20 at a position x2 at time t-1, the edge movement history calculation unit 30 records the edge position x2 as a movement history. To do. At this time, the edge movement history calculation unit 30 sets “time information h = 1 + H” for the edge position x1 detected at time t−2, and sets “time information h for the edge position x2 detected at time t−1. = 1 ”.

  Furthermore, as shown in FIG. 2C, if an edge is detected by the edge detection unit 20 at a position x3 at time t, the edge movement history calculation unit 30 records the edge position x3 as a movement history. At this time, the edge movement history calculation unit 30 sets “time information h = 1 + 2H” for the edge position x1 detected at time t−2, and “time information h for the edge position x2 detected at time t−1. = 1 + H ”. The edge position x3 detected at time t is set to “time information h = 1”.

  As described above, the edge movement history calculation unit 30 records the edge position together with the time information in time series. In this recording process, the edge movement history calculation unit 30 inputs information such as the turning angle of the handle, and initializes the history when the imaging direction itself is changed.

  Next, the speed detecting unit 40 detects the moving speed and moving direction of the edge from the edge position and time information recorded in time series by the edge moving history calculating unit 30. Here, the moving speed of the edge can be obtained by dividing the edge position by the moving time.

  At this time, the speed detection unit 40 does not refer to the entire movement history, but sets a speed detection area when detecting the movement speed of the edge. Then, the speed detection unit 40 detects the edge moving speed and the moving direction from the edge position and time information detected in this region. Further, the speed detection unit 40 inputs at least one information among weather information, time zone information, and luminance information in the image, and detects a decrease in the SN ratio of the image based on the input information. Increase the size to reduce the influence of false detection due to noise. That is, the speed detection unit 40 increases the amount of information for detecting the moving speed and moving direction of the edge by increasing the speed detection area, and reduces the influence of noise.

  Then, after detecting the moving speed and moving direction of the edge, the dangerous object detection unit 50 detects the jumping out of the moving object. FIG. 3 is an explanatory diagram of the pop-out detection process performed by the dangerous object detection unit 50.

The dangerous object detection unit 50 determines whether the moving object is approaching the host vehicle from the detected moving direction. For example, when the camera 10 is capturing an image in the forward direction of the vehicle, it is assumed that the edge a and the edge b are detected as shown in FIG. 3 and the edges a and b are moving in the left direction of the image.

  First, the edge a in the vicinity of the left end of the image moves in a direction away from the center of the image. For this reason, it can be said that the edge a is moving in the direction away from the own vehicle from the fact that the camera 10 images the front direction of the vehicle. Therefore, the dangerous object detection unit 50 does not detect the edge a as a moving object popping out.

  On the other hand, the edge b slightly on the right side of the image moves in a direction approaching the center of the image. For this reason, it can be said that the edge b is moving in the direction approaching the own vehicle from the fact that the camera 10 images the vehicle front direction. Therefore, the dangerous substance detection unit 50 detects that the moving object has jumped out of the edge b.

  The dangerous object detection unit 50 is based on the position information of the own vehicle, and at least one of the road width, the presence / absence of a sidewalk, the number of lanes, the weather information, and the time zone information. Then, it is determined whether or not the object detected as the moving object is noise. Specifically, in the description of FIG. 3, the dangerous object detection unit 50 detects the edge b as a jumping out of a moving object. For example, when the host vehicle is traveling in the right lane on a two-lane road, Therefore, the jumping out of the moving object is unlikely to occur due to the presence of a median strip or the like. For this reason, when driving on the right lane of a two-lane road on one side, the dangerous object detection unit 50 determines the edge b as noise.

  Further, the dangerous object detection unit 50 determines the possibility of popping out according to the weather and time zone, and performs the same process as described above. For example, when traveling on a road adjacent to an elementary school or the like, the possibility of a moving object jumping out increases at school attendance and school leaving time. Further, when traveling on a road adjacent to an athletic park or the like, the possibility of a moving object jumping increases if it is clear, and the possibility of a moving object jumping decreases if it is rainy. Then, the dangerous object detection unit 50 determines the possibility of such a jump and performs the same process as described above.

  Next, when the dangerous object detection unit 50 detects the jumping out of the moving object, the alarm unit 60 notifies the vehicle occupant to that effect. Then, the vehicle occupant is urged to respond quickly to the moving object.

  Next, an example of detailed processing of the moving object detection device 1 according to the present embodiment will be described. FIG. 4 is a flowchart illustrating an example of detailed processing of the moving object detection device 1 according to the present embodiment.

  First, when the ignition switch of the vehicle is turned on and the apparatus 1 is activated, the camera 10 captures an image around the vehicle and acquires an image (ST1). Next, the edge detection unit 20 performs edge processing to detect the edge position (ST2).

  Then, the edge movement history calculation unit 30 records the edge position obtained in step ST2 together with the time information, and updates the movement history (ST3). Thereafter, the speed detection unit 40 detects the moving speed and moving direction of the edge (ST4). Then, the dangerous object detection unit 50 determines whether or not there is a pop-out object based on the moving speed and moving direction of the edge detected by the speed detecting unit 40 (ST5). That is, as described with reference to FIG. 3, the dangerous substance detection unit 50 determines whether or not the edge is approaching the host vehicle from the moving direction. Further, it is also determined from the edge moving speed whether or not there is a possibility of collision with the host vehicle.

  When the dangerous object detection unit 50 determines that the object that is the source of the edge is not approaching the host vehicle or does not collide with the host vehicle as a result of the determination, there is no jumping object. Judgment is made (ST5: NO). And this apparatus 1 advances a process to step ST7.

  On the other hand, if the dangerous object detection unit 50 determines that there is a possibility that the object that has generated the edge approaches the own vehicle and collides with the own vehicle as a result of the determination, the dangerous object detection unit 50 determines that there is a jumping object. (ST5: YES). Then, the dangerous object detection unit 50 detects the edge as a protruding object (ST6). Next, the dangerous object detection unit 50 transmits information on the pop-out object to the alarm unit 60 and causes the alarm unit 60 to notify the information.

  Thereafter, the present apparatus 1 determines whether or not the continuation condition is satisfied (ST7). Here, the apparatus 1 determines whether or not the apparatus start condition and the imaging direction change condition are satisfied as the continuation condition. That is, for example, when the ignition switch or the like is turned off, the apparatus 1 determines that the continuation condition is not satisfied because the apparatus activation condition is not satisfied (ST7: NO), and ends the process. Further, for example, when the edge movement history calculation unit 30 detects a change in the direction of the vehicle from the difference in rotation between the left and right wheels, the apparatus 1 determines that the movement history is initialized and the continuation condition is not satisfied (ST7: NO), the process is terminated. In this case, the process is started again from step ST1, and edge detection and movement history update are performed.

  On the other hand, when it is determined that the apparatus activation condition and the imaging direction change condition as the continuation condition are satisfied (ST7: YES), the apparatus 1 enters a standby state for Δt time (ST8). Then, after waiting, the apparatus 1 repeats the process from step ST1 again.

  In this way, according to the moving object detection apparatus 1 according to the present embodiment, the captured image is subjected to edge processing, and the edge moving speed and direction are detected. For this reason, the moving speed and moving direction of an object etc. are detected. Then, the jumping out of the moving object is detected from the detected moving speed and moving direction of the edge. Therefore, the jumping out of the moving object is detected based on the moving speed and moving direction of the object. As described above, in the present embodiment, processing with heavy burdens such as optical flow calculation and vanishing point calculation is not executed, but only edge detection is performed and its movement is judged. It is possible to provide a moving object detection apparatus that prevents an increase in the amount of movement.

  In addition, since the image direction for edge processing is determined based on the imaging direction of the camera 10, the camera 10 captures the front of the vehicle in consideration of the fact that a moving object usually jumps out from the side of the vehicle. In this case, an edge extending in the vertical direction of the image may be detected so that the movement of the object can be detected in the horizontal direction of the image. Further, when the camera 10 is imaging the side of the vehicle, an edge extending in the horizontal direction of the image may be detected so that an object approaching the host vehicle can be detected. Thereby, in this embodiment, by detecting an edge in a direction necessary for detection of a moving object in edge processing, it is possible to perform appropriate pop-up detection, and further, edge processing is not performed for other directions. The processing load can be further reduced.

  Further, the edge movement history calculation unit 30 includes at least a steering wheel turning angle information, information from a yaw rate sensor, left and right wheel rotation difference information, turn signal information, and road map and own vehicle position information. Based on one information, the edge position and time information recorded in time series are initialized. Here, when it is detected that the imaging direction itself has changed, the edge position and time information recorded in the past do not make sense for detection of a moving object. For this reason, the edge movement history calculation unit 30 performs initialization when a change in the imaging direction of the camera 10 is detected based on any of the above information. Thereby, appropriate pop-out detection can be performed according to the movement of the vehicle.

  In addition, when a speed detection area is set for detecting the moving speed of the edge, and a decrease in the S / N ratio of the image is detected based on at least one of the weather information, the time zone information, and the luminance information in the image, Increase the speed detection area. Thus, by increasing the speed detection area, the amount of information for detecting the moving speed and the backward movement of the edge is increased, and the influence of noise is reduced. Accordingly, it is possible to perform appropriate pop-up detection by performing speed detection according to changes in the input image.

  In addition, the dangerous object detection unit 50 is based on the position information of the host vehicle and at least one of the width of the road on which the host vehicle travels, the presence or absence of a sidewalk, the number of lanes, weather information, and time zone information. Then, it is determined whether or not the object detected as the moving object is noise. Here, for example, when the host vehicle is traveling in the right lane on a two-lane road on one side, it is highly possible that the moving object will jump out from the left side. On the other hand, the jumping out of the moving object from the right side is unlikely to occur due to the presence of a central separator or the like. In this way, the dangerous object detection unit 50 can determine the occurrence probability of popping out. Therefore, noise can be determined based on the occurrence probability of popping out, and appropriate popping out detection can be performed.

  As described above, the present invention has been described based on the embodiment, but the present invention is not limited to the above embodiment, and may be modified without departing from the gist of the present invention. For example, in the above embodiment, the case where the moving object detection device 1 is mounted on a vehicle has been described as an example. However, the present invention is not limited thereto, and may be mounted on, for example, a ship or an airplane.

It is a block diagram of the moving object detection apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows the mode of recording by an edge movement log | history calculation part, Comprising: (a) shows the mode of recording at the time t-2, (b) shows the mode of recording at the time t-1, (c) Indicates the state of recording at time t. It is explanatory drawing of the pop-out detection process by a dangerous substance detection part. It is a flowchart which shows an example of the detailed process of the moving object detection apparatus which concerns on this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Moving object detection apparatus 10 ... Camera (imaging means)
20: Edge detection unit (edge detection means)
30 ... Edge movement history calculation unit (edge movement history creation means)
40: Speed detection unit (speed direction detection means)
50 ... Dangerous substance detection part (pop-out detection means)

Claims (5)

  Monocular imaging means for imaging either the front or side of the vehicle;
  Whether to detect an edge extending in the vertical direction of the image or in the horizontal direction of the image is determined based on the imaging direction of the imaging means, and when it is determined to detect an edge extending in the vertical direction of the image, the brightness of the horizontal direction of the image is determined. By detecting edges extending in the vertical direction of the image based on the change and detecting edges extending in the horizontal direction of the image, it is possible to detect edges extending in the horizontal direction of the image based on changes in brightness in the vertical direction of the image. Edge detection means for performing edge processing on an image captured by the imaging means;
  Edge movement history creation means for inputting an image edge-processed by the edge detection means and recording the edge position in the image together with time information in time series;
  Speed direction detecting means for detecting the moving speed and moving direction of the edge from the edge position and time information recorded in time series by the edge movement history creating means;
  Pop-out detection means for detecting pop-out of a moving object based on the moving speed and moving direction detected by the speed direction detecting means;
  A moving object detection apparatus comprising:   The edge history creating means has a function of initializing edge position and time information recorded in time series, information on a steering wheel turning angle, information from a yaw rate sensor, information on a difference between left and right wheel rotations, a turn signal The moving object detection device according to claim 1, wherein initialization is performed based on at least one of information and information on a road map and a vehicle position.   The speed direction detecting means sets a speed detection region for detecting the moving speed of the edge, and reduces the SN ratio of the image based on at least one of weather information, time zone information, and luminance information in the image. 3. The moving object detection device according to claim 1, wherein, when detected, the speed detection area is enlarged. 4.   The pop-out detection means moves based on the position information of the host vehicle, at least one of the width of the road on which the host vehicle runs, the presence or absence of a sidewalk, the number of lanes, the weather information, and the time zone information. 4. The moving object detection device according to claim 1, wherein it is determined whether or not the object detected is noise.   Either the front or side of the vehicle is imaged with a single eye, it is determined based on the imaging direction whether an edge extending in the image vertical direction or the image horizontal direction is detected, and it is determined to detect an edge extending in the image vertical direction In this case, an edge extending in the vertical direction of the image is detected based on a change in brightness in the horizontal direction of the image, and if it is determined to detect an edge extending in the horizontal direction of the image, the image is determined based on the change in brightness in the vertical direction of the image. Edge processing is performed on the captured image by detecting edges extending in the horizontal direction, and the edge position in the image is recorded in time series together with time information, and the edge is moved from the edge position and time information recorded in time series. A moving object detection apparatus that detects a jump of a moving object by detecting a speed and a moving direction.

Images