JP2014232516A - Object change detection device, warning device, object change detection method, and warning method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object change detection device configured to improve accuracy of detecting change of an object, a warning device, an object change detection method, and a warning method.SOLUTION: An object change detection unit 28 detects change of an object O in an image captured by a camera 12. An object change detection threshold calculation unit 26 changes a condition of the object change detection unit 28 detecting the change of the object O, so that the change cannot be detected as vibration of the object O in the captured image is larger. Even if background other than the object O in a detection area A in a frame surrounding the object O in the captured image changes significantly along with vertical vibration of the object O, a projection or the like is prevented from being wrongly detected on the basis of the change of the background. The accuracy of detecting the change of the object O can be improved, accordingly.

Description

  The present invention relates to an object change detection device, an alarm device, an object change detection method, and an alarm method for detecting a change in an object.

  A technique for detecting a pedestrian based on captured image information has been proposed. For example, Patent Document 1 discloses a plurality of model images prepared in advance in a frame set to surround a pedestrian in a captured image with respect to a captured image captured by a pair of far-infrared cameras 4. An apparatus for detecting a pedestrian by performing a pattern matching process using the same is disclosed. The device of Patent Document 1 determines that a person is a pedestrian if it is raining or if the outside air temperature is determined to be equal to or lower than the threshold value, and if it matches the pattern A model image. In addition, the device of Patent Document 1 matches any one of patterns A, C, and E when it is not in a rainy state, the outside air temperature is larger than the threshold value, and the current time is outside the set time zone. It is determined that it is a pedestrian. Furthermore, if the device of Patent Document 1 is not in a rainy state, the outside air temperature is larger than the threshold value, and is within the set time zone, the pedestrian can be matched with any of the patterns B, D, and F. It is determined that

JP 2007-58805 A

  However, in such a technique, for example, the amount of change in the vertical direction of the center of gravity of the pedestrian differs greatly between when the pedestrian as the object is walking and when the pedestrian is running. Along with this, the vibration in the vertical direction of the frame set to surround the pedestrian also increases. In such a case, the area other than the pedestrian in the frame, that is, the background fluctuation also increases, and depending on the situation, the detection accuracy of the pedestrian's behavior may be affected.

  The present invention has been made in view of the above problems, and provides an object change detection device, an alarm device, an object change detection method, and an alarm method capable of improving the detection accuracy of an object change. With the goal.

  The present invention relates to an imaging unit that captures an image of the surroundings of a vehicle, a detection unit that detects a change in an object detected in a captured image captured by the imaging unit, and a greater vibration of the object in the captured image. And a change means for changing the condition for the detection means to detect a change in the object to a side that is difficult to be detected.

  According to this configuration, the imaging unit images the surroundings of the vehicle, the detection unit detects a change in the object detected in the captured image captured by the imaging unit, and the changing unit detects the object in the captured image. The greater the vibration of the object, the more the detection means changes the condition for detecting the change of the object to the side where it is difficult to detect. Thereby, with the vertical vibration of the object in the captured image, background fluctuations other than the object in the frame set to surround the object in order to detect the object in the captured image Even if it becomes large, it will be suppressed that a pop-out or the like is erroneously detected based on a change in the background. As a result, it is possible to improve the detection accuracy of the change of the object.

  In this case, it is preferable that the detection unit detects a change in the object based on image information in a frame set so as to surround the object in the captured image.

  According to this configuration, the detection unit detects a change in the object based on the image information in a frame set so as to surround the object in the captured image. For this reason, with the vertical vibration of the object in the captured image, the fluctuation of the background other than the object in the frame also increases. However, even in such a case, the changing means changes the condition for the detection means to detect the change of the object to the side where it is difficult to be detected as the vibration of the object in the captured image increases. It is possible to suppress erroneous detection due to changes in the position and improve detection accuracy of changes in the object.

  In this case, as the vibration of the object in the captured image increases, the changing unit sets a threshold value that is a condition for the detection unit to detect a change in the object, and the detection unit represents the image information in the frame. It is preferable to detect the change in the object when the change in the feature amount distribution exceeds the threshold set by the changing unit.

  According to this configuration, the change unit sets the threshold value, which is a condition for the detection unit to detect a change in the target, as the vibration of the target in the captured image increases, and the detection unit sets the image within the frame. A change in the object is detected when the change in the feature quantity distribution representing the information exceeds the threshold set by the changing means. For this reason, even if the fluctuation of the background other than the object in the frame becomes large due to the vertical vibration of the object in the captured image, the background changes due to the high threshold value. On the basis of this, it is possible to suppress erroneous detection of a pop-out or the like. As a result, it is possible to improve the detection accuracy of the change of the object.

  Moreover, this invention is an alarm device provided with the alarm alerting | reporting means which alert | reports an alarm, when the change of a target object is detected by the detection means of the target object change detection apparatus of the said invention.

  According to this configuration, the alarm notification unit that notifies an alarm when a change in the object is detected by the detection unit of the object change detection device of the present invention is provided. For this reason, it is possible to notify the change of the object with higher accuracy while suppressing the alarm from being erroneously notified due to the erroneous detection due to the vibration of the object.

  The present invention also includes an imaging process for imaging the periphery of the vehicle, a detection process for detecting a change in the object detected in the captured image captured by the imaging process, and a vibration of the object in the captured image. It is an object change detection method provided with the change process which changes the conditions which detect the change of an object in a detection process to the side which is hard to be detected, so that it is large.

  In this case, it is preferable that the detection step detects a change in the object based on image information in a frame set so as to surround the object in the captured image.

  In this case, as the vibration of the object in the captured image increases, the change process sets a threshold value that is a condition for the detection process to detect a change in the object, and the detection process represents image information in the frame. It is preferable to detect the change in the object when the change in the feature amount distribution exceeds the threshold set by the changing step.

  Moreover, this invention is an alarm method provided with the alarm alerting | reporting process of alerting | reporting when the change of an object is detected by the detection process of the said object change detection method of the said invention.

  According to the object change detection device, the alarm device, the object change detection method, and the alarm method of the present invention, it is possible to suppress erroneous detection of a jump or the like based on the background change, and to detect the change in the object. Can be improved.

It is a block diagram which shows the popping-out detection apparatus of embodiment. It is a flowchart which shows operation | movement of the pop-out detection apparatus of embodiment. It is a figure which shows the center coordinate in the image of the detected target object. It is a figure which shows the center coordinate in the image of the target object of 1 frame after FIG. It is a figure which shows an example of the graph for a change detection.

  An object change detection device, an alarm device, an object change detection method, and an alarm method according to an embodiment of the present invention will be described with reference to the drawings. The object change detection device according to the present embodiment is mounted on a vehicle, and mainly detects a pedestrian as a target, and detects a jump (rapid approach) of the pedestrian to the course of the vehicle. As shown in FIG. 1, the object change detection device 1 of the present embodiment is configured by connecting a camera 12, a display 32, and a speaker 34 to a general-purpose PC (Personal Computer) 20 so that they can communicate with each other. As the general-purpose PC 20, for example, an automobile control computer mounted on a vehicle or a portable electronic computer can be applied.

  The camera 12 captures the vehicle periphery for each frame and generates image data in which the vehicle periphery is projected. The camera 12 is, for example, a CCD camera or a CMOS camera that can capture an image in the visible light region or the infrared region. The camera 12 is mainly arranged so that the front of the vehicle can be photographed, but may be arranged so that the side or the rear of the vehicle can be photographed.

  The general-purpose PC 20 includes an object detection / tracking unit 22, an object vibration information acquisition unit 24, an object change detection threshold calculation unit 26, and an object change detection unit 28 as functional blocks realized by a predetermined program. Yes. The object detection / tracking unit 22 recognizes an object such as a pedestrian using data included in the captured image acquired by the camera 12. The object detection / tracking unit 22 has a shape of the object such as a pedestrian stored in advance for a detection area obtained by surrounding an object that may be an object such as a pedestrian with a rectangular frame. An object is detected by performing pattern matching or the like using data indicating the outline. The object detection / tracking unit 22 tracks the object by moving the detection area surrounded by the frame in the captured image according to the movement of the object such as a pedestrian. The object detection / tracking unit 22 transmits the obtained information as an object partial image signal to the object vibration information acquisition unit 24 and the object change detection unit 28.

  The object vibration information acquisition unit 24 acquires information related to the degree of vibration of the object based on the displacement of the center coordinates of the object such as a pedestrian obtained by the object detection / tracking unit 22 for each frame. The object vibration information acquisition unit 24 transmits information related to the degree of vibration of the object as an object vibration signal to the object change detection threshold value calculation unit 26.

  The object change detection threshold calculation unit 26 uses the object change detection unit 28 based on the information about the degree of vibration of the object acquired by the object vibration information acquisition unit 24 to detect the vehicle of the object such as a pedestrian. A threshold value T for detecting popping out in the traveling direction is calculated. The object change detection threshold calculation unit 26 transmits the threshold T as a change detection threshold signal as the object change detection unit 28.

  The object change detection unit 28 indicates the degree of change of the object such as a pedestrian based on the change of the feature amount distribution representing the image information in the detection area acquired by the object detection / tracking unit 22 for each frame. The object change degree R is calculated. Further, the object change detection unit 28 determines whether the calculated object change degree R exceeds the threshold value T calculated by the object change detection threshold value calculation unit 26 or not. It detects whether or not there is a change such as a jump in the traveling direction.

  When the object change detection unit 28 detects that the object such as a pedestrian has a change such as a jump in the traveling direction of the vehicle, the display 32 displays the jump of the pedestrian or the like to the driver of the vehicle by an image. indicate. When the object change detection unit 28 detects that the object such as a pedestrian has a change such as a jump in the traveling direction of the vehicle, the speaker 34 audibly sends a pedestrian jump or the like to the driver of the vehicle. Annunciating with an alarm sound.

  Hereinafter, the operation of the object change detection device 1 of the present embodiment will be described. In the following processing, a pedestrian is described as an example of an object existing around the vehicle, but the present invention is not limited to this. In addition, the SIFT feature amount is described as an example of the feature amount indicating the shape of the target object, but is not limited thereto. Further, although the KL information amount will be described as an example as a measure for the difference in probability distribution for each frame, it is not limited to this.

As shown in FIG. 2, a captured image around the vehicle is obtained for each frame by the camera 12, and an image signal including image data is input to the object detection / tracking unit 22 (S11). The object detection / tracking unit 22 detects and tracks an object such as a pedestrian using data acquired from the camera 12 (S12). As shown in FIGS. 3 and 4, the object detection / tracking unit 22 displays an object that may be an object O such as a pedestrian in a rectangular frame at times t−1 and t of each frame. The detection area A is set by enclosing it. The target object detection / tracking unit 22 detects the target object O by performing pattern matching or the like on the detection area A using the data indicating the outline of the target object O such as a pedestrian stored in advance. . As shown in FIGS. 3 and 4, for example, the center coordinate c of the object has moved from (x _t−1 , y _t−1 ) to (x _t , y _t ) from time t−1 to time t. In this case, the object detection / tracking unit 22 tracks the object O by moving the detection area A accordingly.

The object vibration information acquisition unit 24 obtains information on the degree of vibration of the object O based on the displacement of the center coordinates c of the object O such as a pedestrian obtained by the object detection / tracking unit 22 for each frame. Obtain (S13). The degree of vibration of the object O can be obtained, for example, by obtaining a covariance matrix S of the center coordinates c of the object O in the captured image, as shown in the following equation (1). About the average value of the center coordinate c, it can be set as the average value from arbitrary n frames before to the present frame, for example.

The object change detection threshold value calculation unit 26 calculates a threshold value T as a function of the covariance matrix S obtained by the object vibration information acquisition unit 24 (S14). Here, the object change detection threshold value calculation unit 26 sets T = T1 if det (S)> s ₀ and sets T2 if det (S) ≦ s ₀ . s ₀ is an arbitrary constant. Here, since it is desired to increase the threshold T when the vibration of the object O is large, T1> T2.

  The object change detection unit 28 determines the degree of change of an object such as a pedestrian based on the change of the feature amount distribution representing the image information in the detection area A acquired by the object detection / tracking unit 22 for each frame. An object change degree R shown is calculated (S15). The object change detection unit 28 acquires a feature amount distribution representing the shape information of the object O in the detection area A. The object change detection unit 28 calculates SIFT feature values as a feature value distribution representing the shape information of the object O. The SIFT feature value is well known in the field of image recognition and the like as a feature value representing a shape. Also, the SIFT feature value can be represented as a probability distribution because it can be represented by a histogram which edge of the image data including the object O is distributed in which direction.

The object change detection unit 28 normalizes the feature quantity distribution and obtains the feature quantity (probability distribution) p (t) as shown in the following equation (2).

The object change detection unit 28 accumulates the feature value p (t) in the general-purpose PC 20 for each frame. As shown in the following equation (3), the object change detection unit 28 determines an object change that is a difference between a feature quantity n frames before from the accumulated feature quantity p (t) and the current feature quantity. The degree R (t, n) is calculated. The object change detection unit 28 calculates the object change degree R (t, n) using the KL information amount as shown in the following equation (4). For example, in FIG. 5, the vertical axis indicates the amount of KL information, and the horizontal axis indicates a frame for each time. The example of FIG. 5 shows a state where the object O, which is a pedestrian, suddenly changes its movement near the last frame of the graph.

  The object change detection unit 28 determines whether or not the calculated object change degree R exceeds the threshold T calculated by the object change detection threshold calculation unit 26 (S16). ). When the object change degree R exceeds the threshold value T (S16), the object change detection unit 28 detects that the object O has a change such as jumping out to the course of the vehicle (S17). When the object change degree R does not exceed the threshold value T (S16), each unit of the general-purpose PC 20 continues the processes of S11 to S16 for the subsequent frames. When the object change detection unit 28 detects that there is a change in the object O (S16, S17), the display 32 and the speaker 34 are used to detect a pedestrian jumping out of the vehicle driver by an image, sound, or alarm, respectively. Attention is paid to the change of the object O (S18).

  According to the present embodiment, the camera 12 captures an image around the vehicle, and the object change detection unit 28 detects a change in the object O detected in the captured image captured by the camera 12, and detects an object change. The threshold value calculation unit 26 changes the condition for the object change detection unit 28 to detect the change of the object O to the side where it is difficult to detect as the vibration of the object O in the captured image increases. Thereby, the object of the detection area A within the frame set to surround the object O in order to detect the object O in the captured image in accordance with the vertical vibration of the object O in the captured image. Even in the case where the background fluctuations other than the object O become large, it is possible to suppress erroneous detection of popping out based on the change in the background. As a result, it is possible to improve the detection accuracy of the change in the object O.

  The object change detection unit 28 detects a change in the object O based on the image information in the detection area A within the frame set to surround the object O in the captured image. For this reason, the fluctuation of the background other than the object O in the detection area A within the frame also increases with the vertical vibration of the object O in the captured image. However, even in such a case, the object change detection threshold calculation unit 26 detects a condition under which the object change detection unit 28 detects a change in the object O as the vibration of the object O in the captured image increases. In order to change to the side where it is hard to be done, the erroneous detection by the change of the background in a frame can be suppressed, and the detection accuracy of the change of the target object O can be improved.

  The object change detection threshold calculation unit 26 sets the threshold T, which is a condition for the object change detection unit 28 to detect a change in the object O, as the vibration of the object O in the captured image increases. The object change detection unit 28 detects the object O when the change in the feature amount distribution representing the image information in the detection area A within the frame exceeds the threshold T set by the object change detection threshold calculation unit 26. Detect changes. For this reason, even when the fluctuation of the background other than the object O in the frame becomes larger due to the vertical vibration of the object O in the captured image, the background is set by the high threshold T. Based on this change, it is possible to suppress erroneous detection of pop-out or the like. As a result, it is possible to improve the detection accuracy of the change of the object.

  Further, the object change detection device 1 of the present embodiment includes a display 32 and a speaker 34 that notify an alarm when a change in the object O is detected by the object change detection unit 28. For this reason, it is possible to notify the change of the object O with higher accuracy while suppressing the alarm from being erroneously notified due to erroneous detection due to the vibration of the object O.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation aspect is possible. The object change detection device according to the present embodiment is also suitable for objects such as pedestrians, other vehicles other than human beings and motorcycles, depending on whether the road environment is a flat road or a road with large unevenness such as a gravel road. This threshold value T can be applied by setting it higher as the road unevenness is larger.

  DESCRIPTION OF SYMBOLS 1 ... Object change detection apparatus, 12 ... Camera, 20 ... General-purpose PC, 22 ... Object detection and tracking part, 24 ... Object vibration information acquisition part, 26 ... Object change detection threshold value calculation part, 28 ... Object change Detection unit, 32 ... display, 34 ... speaker, O ... target, A ... detection region, c ... coordinate center.

Claims (8)

Imaging means for imaging the surroundings of the vehicle;
Detecting means for detecting a change in the object detected in the captured image captured by the imaging means;
Changing means for changing the condition for the detection means to detect a change in the object to a side that is difficult to be detected, as the vibration of the object in the captured image is larger,
An object change detection device comprising:   The target change detection device according to claim 1, wherein the detection unit detects a change in the target based on image information in a frame set to surround the target in the captured image. The change means sets a threshold value that is a condition for the detection means to detect a change in the target object as the vibration of the target object in the captured image increases.
The said detection means detects the change of the said object, when the change of the feature-value distribution showing the image information in the said frame exceeds the said threshold value set by the said change means. Object change detection device.   The alarm device provided with the alarm alerting | reporting means which alert | reports an alarm, when the change of the said object is detected by the said detection means of the object change detection apparatus of any one of Claims 1-3. An imaging process for imaging the surroundings of the vehicle;
A detection step of detecting a change in the object detected in the captured image captured by the imaging step;
A change step of changing a condition for detecting a change of the object in the detection step to a side that is difficult to be detected, as the vibration of the object in the captured image is larger.
An object change detection method comprising:   The object change detection method according to claim 5, wherein the detection step detects a change in the object based on image information in a frame set so as to surround the object in the captured image. In the changing step, the larger the vibration of the object in the captured image, the higher the threshold that is a condition for the detection step to detect a change in the object,
The said detection process detects the change of the said object, when the change of the feature-value distribution showing the image information in the said frame exceeds the said threshold value set by the said change process. Object change detection method.   The alarm method provided with the alarm alerting | reporting process which alert | reports an alarm, when the change of the said object is detected by the said detection process of the target object change detection method of any one of Claims 5-7.

Images