WO2010109831A1

WO2010109831A1 - Drive recorder

Info

Publication number: WO2010109831A1
Application number: PCT/JP2010/001968
Authority: WO
Inventors: 大和宏
Original assignee: コニカミノルタホールディングス株式会社
Priority date: 2009-03-23
Filing date: 2010-03-18
Publication date: 2010-09-30
Also published as: JPWO2010109831A1

Abstract

A drive recorder (1) calculates relative speed to objects (4, 5) from the time-series images acquired by stereo cameras (11, 12), and gives trigger to the recording section to start recording so as to record the time-series images on a recording section in response to the calculated relative speed, or applies marking to the time-series images which are always recorded by the recording section. Hence, the drive recorder (1) with this configuration detects the relative speed from the time-series images to perform the determination of the trigger or the marking, thus making it possible to reduce malfunction and perform more proper trigger or marking in comparison to the determination using a general acceleration sensor.

Description

Drive recorder

The present invention relates to a drive recorder that is suitably mounted on a vehicle, for example, and records an image of a running subject, and particularly relates to a trigger for starting recording and marking of a point of interest in continuous recording.

In recent years, for example, drive recorders capable of recording images at the time of vehicle collision have been put into practical use. This drive recorder is useful for investigating the cause of an accident by recording an image at a predetermined time of several seconds to several tens of seconds before and after the time when a collision occurs. In addition, with the increase in recording media capacity, drive recorders that record continuously and can verify events before the accident, so-called near-miss events, are mainly used in the transportation and transportation industries. It is starting to be used.

By the way, in such a drive recorder that records for a short time, a recording start trigger is generated and recording starts when an acceleration sensor detects vibration of a predetermined level or higher. In the case of continuous recording, when a near-miss event occurs, the driver puts the marking at the point of interest, or the driver takes the recording medium back to the office, and the administrator checks and applies the marking. It is. Therefore, in the case of using the acceleration sensor, the acceleration sensor erroneously detects recording due to vibrations caused by road surface unevenness or crossings, and recording is performed, and there is no free space on the recording medium in the event of an actual accident. There is a problem that a case that cannot be recorded occurs. In the case of continuous recording, there is a problem that marking is troublesome.

Therefore, Patent Document 1 has been proposed in order to solve such a problem. In this prior art, cameras are installed at four locations so as to monitor 360 degrees around the vehicle, while the distance calculation device calculates the distance to the target object, and when the calculated distance is closer than the predetermined distance, the recording start determination is made. It is carried out. Thus, this prior art is a drive recorder that automatically triggers recording without using an acceleration sensor so as not to record an erroneous image due to the road surface unevenness or passing through a railroad crossing. Further, it is shown that the distance calculation device uses a single-lens camera, a twin-lens camera, and an ultrasonic sensor (sonar).

Since this conventional technique simply determines whether or not to perform recording based only on the distance relationship with the target object, if the threshold for determining that recording should be started is severe, recording is performed only to the extent of actual collision. In other words, it is difficult to reliably record a scene such as the near-miss, and on the other hand, if the threshold value is low, it is recorded even in a less dangerous situation, and the capacity of the recording medium is reduced as described above. There is a problem that it is not possible to record at an important time. In addition, when this conventional technique is used for marking continuous recording, if the threshold is severe, leakage of incidents of the near-miss event will occur, and if the threshold is loose, the marking will be full of markings. There is a problem that it is complicated.

JP 2007-334760 A

The present invention has been made in view of the above circumstances, and an object thereof is to provide a drive recorder capable of performing more appropriate triggering and marking.

A drive recorder according to the present invention calculates a relative speed with respect to a target object from a time-series image acquired by a stereo camera, and records the time-series image in a recording unit in response to the calculated relative speed. Accordingly, a recording start trigger is given to the recording unit, or marking is performed on the time-series images that are constantly recorded by the recording unit. For this reason, the drive recorder having such a configuration detects the relative speed from the time-series image and determines the trigger or the marking, so that the malfunction can be reduced compared with the determination using a general acceleration sensor. And more appropriate triggering and marking can be performed.

The above and other objects, features and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings.

1 is a schematic configuration diagram of a drive recorder according to an embodiment of the present invention. It is a block diagram which shows the structure of the controller in the drive recorder shown in FIG. FIG. 2 is a diagram for explaining time-series captured images by a stereo camera in the drive recorder shown in FIG. 1. 4 is a flowchart for explaining a first relative speed calculation method in the drive recorder shown in FIG. 1. It is a figure for demonstrating the calculation method of FIG. 6 is a flowchart for explaining a second method of calculating a relative speed in the drive recorder shown in FIG. 1. It is a figure for demonstrating the calculation method of FIG. It is a figure for demonstrating the calculation method of the said relative velocity. 6 is a flowchart for explaining a third method of calculating a relative speed in the drive recorder shown in FIG. 1. 3 is a flowchart for explaining a first determination method of trigger determination in the drive recorder shown in FIG. 1. 6 is a flowchart for explaining a second determination method of trigger determination in the drive recorder shown in FIG. 1. FIG. 2 is a diagram showing an example of the trigger determination scene in the drive recorder shown in FIG. 1. It is a graph for demonstrating the determination method in the case of performing trigger determination from relative speed in FIG. FIG. 13 is a graph for explaining a determination method when trigger determination is performed from relative acceleration. FIG. 6 is a diagram showing another example of the trigger determination scene in the drive recorder shown in FIG. 1. In FIG. 15, it is a graph for demonstrating the determination method in the case of performing trigger determination from a relative speed. In FIG. 15, it is a graph for demonstrating the determination method in the case of performing trigger determination from a relative acceleration. 6 is a flowchart for explaining a third determination method of trigger determination in the drive recorder shown in FIG. 1. It is a figure which shows the structure of the controller in the drive recorder which concerns on other forms of implementation of this invention. FIG. 20 is a diagram showing an example of a collision possibility determination scene in the drive recorder shown in FIG. 19. FIG. 20 is a diagram showing another example of a collision possibility determination scene in the drive recorder shown in FIG. 19. FIG. 20 is a diagram showing still another example of a collision possibility determination scene in the drive recorder shown in FIG. 19. FIG. 20 is a flowchart for explaining the operation of the drive recorder shown in FIG. 19. FIG.

Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted suitably.

FIG. 1 is a schematic configuration diagram of a drive recorder 1 according to an embodiment of the present invention. The drive recorder 1 is mounted on a vehicle 2 and obtains a two-dimensional input image of a subject, and obtains a three-dimensional image from the stereo images acquired by the

stereo cameras

11 and 12. And a controller 13 that automatically records the images of the

stereo cameras

11 and 12 when the degree of danger is high and the degree of danger is high. The

stereo cameras

11 and 12 output a pair of left and right images (a standard image and a reference image) obtained by photographing the subject at the same timing. In the present embodiment, for the sake of simplicity of explanation, it is assumed that the aberrations of the

stereo cameras

11 and 12 are well corrected and installed parallel to each other. Even if the actual hardware is not in such a condition, it is possible to convert it to an equivalent image by image processing. The stereo images are transmitted from the

stereo cameras

11 and 12 to the controller 13 via a communication line. The communication method of the image data between the

stereo cameras

11 and 12 and the controller 13 is not limited to the wired method, and may be a wireless method.

Here, the drive recorder 1 according to the present embodiment, when recording the images of the

stereo cameras

11 and 12 when the degree of danger is high as described above, uses the target object 4 calculated from the stereo image for the trigger determination. By using the relative speed of 5, there is no need to calculate acceleration information, distance information, object movement information, etc. as in the prior art. Note that the drive recorder 1 not only records a stereo image when the risk level is high in this way, but also always records the stereo image, and when the risk level is high, it is easy to cue playback of the image at that time later. It may be configured to provide markings for In the following description, it is assumed that the drive recorder 1 performs recording by a trigger.

For this reason, the image acquisition unit includes

stereo cameras

11 and 12 as an example, and the controller 13 cuts out an object from a stereo image obtained by the

stereo cameras

11 and 12, as shown in FIG. A relative speed calculation unit 21 that searches only for a region, calculates a relative speed with respect to a target object that is considered to be likely to collide due to being on the lane of the host vehicle, and the calculation result A trigger determination unit 22 that performs the trigger determination in response, and a recording unit 23 that records at least one of the images of the two

stereo cameras

11 and 12 in response to a trigger from the trigger determination unit 22. Composed.

In the drive recorder 1 configured as described above, when calculating the relative speed, first, time-series captured images from the

stereo cameras

11 and 12 are compared. Therefore, as shown in FIG. 3, from a certain time t, images captured by the stereo camera 11 every Δt seconds (one frame) are I1 (i, j, t), I1 (i, j, t + Δt), I1 (i, j, t + 2Δt),..., I1 (i, j, t + nΔt). Similarly, the images captured by the teleo camera 12 are I2 (i, j, t), I2 (i, j, t + Δt). ), I2 (i, j, t + 2Δt),..., I2 (i, j, t + nΔt). Here, i and j represent the positions in the x and y directions of the image pickup elements arranged in a matrix. From the captured image thus obtained, the relative speed calculation unit 21 calculates a relative speed. In the following, three calculation methods are exemplified.

FIG. 4 is a flowchart for explaining a first method for calculating the relative speed by the relative speed calculation unit 21 in the trigger determination, and FIG. 5 is a diagram for explaining the calculation method. In step S1, the corresponding point pr on the reference image I2 (i, j, t) is obtained by the corresponding point search process for the pixel p (i, j) on the base image I1 (i, j, t + Δt) at time t + Δt. (I, j) is calculated, and three-dimensional information P (i, j, t) is calculated by the three-dimensional reconstruction process. The corresponding point search method includes a gradient method which is one of optimization methods, a method of subtracting luminance values as they are (SAD (Sum of Absolute Absolute Difference) method or SSD (Sum of Square Intensity Difference) method), or each method Correlation calculation methods such as a method (NCC (NormalizedNormalCross Correlation) method) that subtracts a local average value from the luminance value of a point and obtains a corresponding point with the similarity of the variance values are used. Among the correlation calculation methods, a POC (corresponding position can be calculated in sub-pixel units based on the similarity of signals obtained by frequency-resolving patterns in a window set for two input images and suppressing amplitude components. By using the phase-only correlation method, stable and highly accurate association is possible.

Next, in step S2, the reference image I1 (i, j) at time t is obtained by performing corresponding point search processing on the pixel p (i, j) on the reference image I1 (i, j, t + Δt) at time t + Δt. , T), the corresponding point pp (i, j) is calculated. Subsequently, in step S3, the correspondence on the reference image I2 (i, j, t) is detected by the corresponding point search process for the pixel pp (i, j) on the reference image I1 (i, j, t) at time t. A point ps (i, j) is calculated, and three-dimensional information Pp (i, j, t) is calculated by a three-dimensional reconstruction process. Then, in the calculated three-dimensional information P (i, j, t) and the three-dimensional information Pp (i, j, t), the target for which the relative speed is to be obtained is determined in step S4, and the

target object

4 , 5 from the position Pp (i, j, t) at time t and the position P (i, j, t) at time t + Δ, the difference obtained as follows in step S5 is the time The relative speed ν (i, j, t) of movement between Δ is obtained.
ν (i, j, t) = P (i, j, t) −Pp (i, j, t)
| Ν (i, j, t) | = {{Px (i, j, t) −Ppx (i, j, t)} 2+ {Py (i, j, t) −Ppy (i, j, t) } 2+ {Pz (i, j, t) -Ppz (i, j, t)} 2} 1/2

Thus, the drive recorder 1 of the present embodiment can calculate the relative speed with high accuracy by calculating the relative speed ν (i, j, t) from the parallax information calculated by the corresponding point search process.

The corresponding point search method in step S3 includes a sub-pixel estimation method for calculating a three-dimensional movement vector using four stereo images at two times T1 and T2 because the reference coordinate position is a decimal point coordinate. Can be used. For such a subpixel estimation method, reference can be made to, for example, Japanese Patent Laid-Open No. 2001-84383. Further, a motion vector generation apparatus and method for selecting an optimal estimation model based on a correlation value between a peak position correlation value and a correlation value around the peak position by the applicant and a model for subpixel estimation, By setting a sub-pixel window at the point of interest, the perimeter monitoring method that directly associates the sub-pixel point of interest, the interpolation processing based on the results of the four corresponding points measured in the past A three-dimensional optical flow calculation method or the like that performs the above can be used.

FIG. 6 is a flowchart for explaining a second method for calculating the relative speed by the relative speed calculation unit 21 in the trigger determination, and FIG. 7 is a diagram for explaining the calculation method. . In the second calculation method, as shown in FIG. 8, the relative speed is calculated only for a predetermined fixed region obtained by dividing an acquired image into regions and applying hatching. In the fixed area, when the relative speed is obtained by the following process, the calculation may be performed using only the representative value to reduce the process. In this way, by setting the calculation area for calculating the corresponding points as a fixed area, there is no need to perform a special calculation process for determining the calculation area, and the relative speed calculation unit 21 performs the corresponding point search process. Only the process of calculating the relative speed from the obtained parallax information may be performed.

It should be noted that in the case of performing computation only on the fixed area, the condition is that the time Δt is as short as a predetermined range, for example, up to 0.1 second, preferably 1/30 second or less. This means that the short time Δt means that the frame rate is large and the amount of change (three-dimensional optical flow), that is, the deviation is small, and this is the movement of the region of the target objects 4 and 5 in the time direction. This means that the distance is also small. Furthermore, there is a high possibility that the target region is included in the fixed region between times t and t + Δt.

First, in step S11, the corresponding point prt on the reference image I2 (i, j, t) is obtained by the corresponding point search process for the pixel pt (i, j) on the reference image I1 (i, j) at time t. (I, j) is calculated, and three-dimensional information P (i, j, t) is calculated by the three-dimensional reconstruction process. Similarly, in step S12, the pixel pt + Δt (i, j) on the base image I1 (i, j, t + Δt) at time t + Δt is subjected to the corresponding point search process on the reference image I2 (i, j, t). Corresponding point prt + Δt (i, j) is calculated, and three-dimensional information P (i, j, t + Δt) is calculated by the three-dimensional reconstruction process. Subsequently, in step S13, the target objects 4 and 5 are extracted. In step S14, the three-dimensional information P (i, j, t) and P (i, j, t + Δt) in the region of the target objects 4 and 5 are extracted. From the difference, the relative velocity ν (i, j, t) is obtained as follows.
ν (i, j, t) = P (i, j, t) −P (i, j, t + Δt)
| Ν (i, j, t) | = {{Px (i, j, t) −Px (i, j, t + Δt)} 2+ {Py (i, j, t) −Py (i, j, t + Δt) } 2+ {Pz (i, j, t) −Pz (i, j, t + Δt)} 2} 1/2

The advantage of this method is that there is no need to perform corresponding point search by tracking in the time direction unlike the first calculation method, and for each time t, t + Δt, t + 2Δt,. (Step S11 uses the previous value), and the processing can be performed at high speed. As the corresponding point search method, the method described above can be used.

The relative speed used for the actual trigger determination described later is the difference of the average value of the three-dimensional information P (i, j, T) calculated for each pixel as described above, The histogram is used for the three-dimensional information P (i, j, t) calculated at time t, which is the difference between the most frequent ones from the histogram of the three-dimensional information P (i, j, T) calculated every time. Any of the three-dimensional information P (i, j, t + Δt) selected and calculated at time t + Δt may be obtained using any of the average values of the results of the distance change within the threshold.

FIG. 9 is a flowchart for explaining a third method of calculating the relative speed by the relative speed calculation unit 21 in the trigger determination. In the third calculation method, the first or second calculation method is used in step S22. However, in step S21, the target objects 4 and 5 are extracted in advance from the acquired image, and the relative speed of only the extracted target object is calculated. For extraction of the target objects 4 and 5, a method such as pattern recognition from a two-dimensional image is used. Therefore, even if the first or second calculation method is used, the object extraction in the steps S4 and S13 is not performed. In this method, the calculation load of the preprocessing in step S21 increases, but it is only necessary to calculate the relative speed of only the area of the target objects 4 and 5, so the calculation load of the process in the subsequent step S22 is reduced. As a result, the processing can be performed at a higher speed than when the entire region is processed.

There are the following methods for extracting the target object in step S21, and any of them may be used. First, based on the distance information and the road model, an image in which the road surface is removed, a three-dimensional movement vector of an object above the road surface is calculated, and a stationary object and a moving object are determined based on the three-dimensional information. There are a processing apparatus and an image processing method. For such an image processing apparatus and method, reference can be made, for example, to JP-A-2006-134035. Then, after obtaining the optical flow of each attention point by obtaining corresponding points between multiple images taken at different timings, the vanishing point of each optical flow is obtained, and the vanishing point is determined by the stationary object There is a method of extracting an object that does not match the vanishing point of the optical flow as a moving object. Then, the parallax d is calculated from the stereo image by the corresponding point search process, the road surface parameter is estimated from the relationship between the parallax d and the y-axis direction on the image, and only the area excluding the estimated road surface is the target area. There is a motion analysis device. For such an analysis apparatus, reference can be made to, for example, Japanese Patent Application Laid-Open No. 2009-181492. Then, there is a speed-up processing method of the periphery monitoring method that analyzes the amount of movement of the two-dimensional optical flow calculated from the time series image and extracts a region having a motion different from the prediction as a target region.

Preferably, the types of the target objects 4 and 5 such as whether they are people or cars are recognized on the extracted

target objects

4 and 5 based on the size of the target objects 4 and 5. May be.

Hereinafter, the trigger determination method of this embodiment will be described in detail. The trigger determination unit 22 performs trigger determination based on the relative speed ν (i, j, t) obtained as described above by the relative speed calculation unit 21. First, in the first determination method, as shown in FIG. 10, when the trigger determination unit 22 takes in the relative speed ν (i, j, t) from the relative speed calculation unit 21 in step S31, in step S32. Compared with the threshold value TH1, if it is larger than the threshold value TH1 in the minus direction, that is, if the target objects 4 and 5 are rapidly approached, it is determined that there is a possibility of collision and the recording unit 23 starts recording. If the trigger is given and is smaller than the threshold value TH1, it is determined that the possibility of collision is small, and the process returns to step S31.

On the other hand, in the second determination method, as shown in FIG. 11, the trigger determination unit 22 receives the time-series relative speeds ν (i, j, t), ν (i, j) from the relative speed calculation unit 21 in step S41. , T + Δt), the relative acceleration a is obtained by the following equation, compared with the threshold value TH2 in step S42, and when it is larger than the threshold value TH2 in the minus direction, that is, the change in acceleration is large and the target objects 4 and 5 are rapidly approached. If it is determined that there is a possibility of collision, a recording start trigger is given to the recording unit 23, and if it is smaller than the threshold value TH2, it is determined that there is little possibility of collision and the process returns to step S41. Any method other than the above may be used as a method of calculating the relative acceleration from the relative velocity.
a = {ν (i, j, t + Δt) −ν (i, j, t)} / t + Δt−t
= Δν (i, j, t) / Δt

Accordingly, as shown in FIG. 12, for example, the preceding vehicle (target object 4) is braked suddenly while the host vehicle (vehicle 2) and the preceding vehicle (target object 4) are following at 60 km / h at a constant speed. Assuming that the vehicle is stopped and stopped (0 km / h), the relative speed ν (i, j, t) falls as shown in FIG. 13, and the relative acceleration a falls greatly negatively as shown in FIG. . However, the relative speed of the preceding vehicle (target object 4) toward the host vehicle (vehicle 2) is negative, and the relative speed of going away is positive. If the threshold TH1 is set to -50 km / h, for example, and the relative speed ν (i, j, t) is greatly reduced beyond that, and the threshold TH2 is set to -20 km / h / sec, for example. When the relative acceleration a is greatly reduced beyond that, a trigger determination is performed.

Here, when the trigger determination is performed by comparing the relative speed ν (i, j, t) with the threshold value TH1 by comparing FIG. 13 and FIG. 14, the relative speed is set to an arbitrary value from an arbitrary speed. When the comparison is made by comparing the relative acceleration a with the threshold value TH2, the change occurs with a certain width from the relatively stable median value (0). Easy to set up.

On the other hand, as shown in FIG. 15, for example, the preceding vehicle (target object 4) is traveling at 60 km / h, the own vehicle (vehicle 2) catches up at 100 km / h, and the inter-vehicle distance is reduced. Assuming that the vehicle (vehicle 2) suddenly brakes and stops (0 km / h), the relative speed ν (i, j, t) is as shown in FIG. 16, and the relative acceleration a is as shown in FIG. As shown, both increase significantly positively. In this case, since it is on the safe side, the trigger is not applied in the determination with the threshold values TH1 and TH2. However, since it was a dangerous situation immediately before, the relative speed ν (i, j, t) and the relative acceleration a were obtained as absolute values, and the magnitude relation of comparison with the thresholds TH1 and TH2 in steps S32 and S42. By reversing (inequality sign), even in such a situation, it can be set as a trigger target.

In the third determination method, as shown in FIG. 18, the trigger determination unit 22 takes in the relative speed ν (i, j, t) from the relative speed calculation unit 21 in step S51, and the relative speed ν. The three-dimensional information P (i, j, t) of the target objects 4 and 5 obtained when (i, j, t) is obtained is taken, and the distance D (i, j, t), and the collision prediction time S (i, j, t) is obtained from the distance D (i, j, t) and the relative velocity ν (i, j, t) by the following equation. Then, in step S52, the collision prediction time S (i, j, t) is compared with the threshold value TH3. If it is smaller than the threshold value TH3, it is determined that there is a possibility of collision, and a trigger is given to the recording unit 23. If it is equal to or greater than TH3, it is determined that the possibility of collision is small, and the process returns to step S51.
D (i, j, t) = {P (i, j, t) 2} 1/2
S (i, j, t) = D (i, j, t) / ν (i, j, t)

In this way, the trigger determination unit 22 considers not only the relative speed ν (i, j, t) but also the distance D (i, j, t) with respect to the target objects 4 and 5, and the distance D (i, j , T) and the relative speed ν (i, j, t) are calculated from the collision predicted time S (i, j, t), and the predicted collision time S (i, j, t) is compared with the threshold value TH3. By performing the trigger determination, the drive recorder 1 of the present embodiment can perform the determination with high accuracy.

Furthermore, in the drive recorder 1a using the fourth determination method, as shown in FIG. 19, the controller 13a applies to the target object (target region of the trigger determination unit) determined to be the trigger target by the trigger determination unit 22. On the other hand, the possibility of collision is further analyzed by the collision possibility analysis unit 24, and when the possibility of collision is high, a recording start trigger is given to the recording unit 23.

In the first analysis method of the collision possibility analysis unit 24, the relative velocity calculation unit 21 calculates the three-dimensional information P (i, j, t), Pp (i) of the time-series images calculated as shown in FIG. , J, t), the vector / P (i, j, t) (/ represents that it is a vector) is calculated, and the direction and length of the vector / P (i, j, t) are calculated. The collision is determined from the speed of the host vehicle (vehicle 2) and the distance D (i, j, t) to the target objects 4 and 5. As the collision determination method, the above-described method or the like is used. For example, assuming that the vehicle (vehicle 2) is a cube, the intersection P (X, Y, Z) between the plane constituting the cube and the three-dimensional optical flow (the vector / P (i, j, t)) There is a method of making a determination by calculating.

In this way, the collision possibility analysis unit 24 further analyzes even the collision possibility of the region determined as the trigger target from the relative velocity ν (i, j, t) and the relative acceleration a by the trigger determination unit 22, and the analysis By giving a trigger according to the result, malfunctions can be reduced by analyzing the region of the target objects 4 and 5 in more detail, such as whether the vehicle is on the lane of the host vehicle (vehicle 2). Further, the collision possibility analysis unit 24 uses the three-dimensional information P (i, j, t), Pp (i, j), which is position information of the region determined as the trigger target by the trigger determination unit 22, as the collision possibility. , T), the direction and length of the three-dimensional movement vector / P (i, j, t), the speed of the host vehicle (vehicle 2), and the distance D (i, j, t) to the target objects 4 and 5 Therefore, it is possible to make a determination in consideration of the movement of the target objects 4 and 5.

In that case, as shown in FIG. 20, a cube 2b that is one size larger is defined in the cube 2a of the host vehicle (vehicle 2), a plane that forms the

cubes

2a and 2b, and a vector / P (i, j, The collision possibility analysis unit 24 may be configured to perform the determination at different levels at the intersection P (X, Y, Z) with t). For example, the target object 9a in the case of intersecting with the own vehicle area of the cube 2a is determined to be a collision danger target having a high possibility of collision, and the area of the cube 2b is a near-miss area and intersects only with the near-hat area (cube 2b). In this case, the target object 9b is determined to be a near-miss target with a high degree of danger although the possibility of collision is low. The trigger determination may be arbitrarily selected as to whether only the case of intersecting with the own vehicle area (cube 2a) or the case of intersecting with the near-miss area (cube 2b) is included.

Furthermore, as shown in FIG. 21, the size of the near miss area may be changed according to the relative speed ν (i, j, t). That is, as shown in FIG. 21A, when the relative speed ν (i, j, t) with respect to the target object 9b is large, the size of the near-miss area is increased as indicated by reference numeral 2b ′. As shown in (b), when the relative velocity ν (i, j, t) is small, the default size is set. As described above, the collision possibility analysis unit 24 increases the extraction region of the target object 9b as the relative velocity ν (i, j, t) increases (cube 2b → 2b ′), so that the analysis can be performed with higher accuracy. It can be performed.

Further, as shown in FIG. 21 (a), the size of the near-miss region (cube 2b ') may be increased on the moving direction side of the target object 9b. In the example of FIG. 21 (a), the target object 9b approaches the host vehicle (vehicle 2) and also moves from the left to the right, and therefore, the near-miss region (cube 2b ′). Is formed large on the right side. In this way, the collision possibility analysis unit 24 increases the extraction area (cube 2b ′) of the target objects 9a and 9b on the moving direction side of the target object 9b, and in a scene that jumps out of the measurement. More accurate analysis can be performed.

The collision possibility analysis unit 24 extracts the target objects 9a and 9b when the relative velocity calculation unit 21 determines that the person area can be extracted by recognizing the target objects 9a and 9b. A margin may be provided so that the area (

cubes

2b, 2b ′) is slightly larger than the own vehicle area (cube 2a).

In the second analysis method of the collision possibility analysis unit 24, the relative velocity calculation unit 21 causes the pixel pp (i, j) on the reference image I1 (i, j, t) at time t as shown in FIG. On the other hand, the corresponding point ps (i, j) on the reference image I2 (i, j, t) is calculated by the corresponding point search process, and obtained from these pixels pp (i, j) and ps (i, j). Corresponding to the reference image I2 (i, j, t) by the corresponding point search process for the optical flow OFt to be performed and the pixel p (i, j) on the standard image I1 (i, j, t + Δt) at time t + Δt. The point pr (i, j) is calculated, and using the optical flow OFt + Δt obtained from the pixels p (i, j) and pr (i, j), as shown in FIG. ) Of the target object 9c is equal to the vehicle speed V1. In a fixed state, when the angle θt, θt + Δt with respect to the traveling direction F of the host vehicle (vehicle 2) is detected, and the change of the angles θt, θt + Δt with respect to time (θt−θt + Δt) is within a predetermined threshold value, It is analyzed that there is a possibility of collision.

In this way, the collision possibility analysis unit 24 detects the angles θt and θt + Δt with respect to the traveling direction F of the host vehicle (vehicle 2) with respect to the area determined as the trigger target by the trigger determination unit 22. If the angles θt and θt + Δt do not change, it is possible to collide with simple parameters such as the angles θt and θt + Δt with respect to the traveling direction F of the host vehicle (vehicle 2) by determining as the region where there is a possibility of a collision if the vehicle runs as it is. The presence or absence of sex can be determined.

Information recorded by the recording unit 23 includes at least one of the base image I1 or the reference image I2, the collision time, the vehicle speed at the time of the collision, the acceleration, the steering angle of the steering wheel, and 3 Dimensional information and the like can be considered, but the parameter obtained from the optical flow can also be obtained from a time-series recorded image after the fact. Therefore, the controller 13 does not particularly use the optical flow when performing the above-described trigger determination. In some cases, it may not be recorded. Moreover, the parameter used by the above-mentioned trigger determination may be recorded, and the three-dimensional information effective for accident analysis may be recorded.

In addition, the recording unit 23 is configured to record a rank corresponding to the level of the near-miss in addition to the possibility of collision when the collision possibility analysis unit 24 determines that the scene is a near-miss scene. Also good. In that case, the collision possibility analysis unit 24 includes a plurality of different sizes in the near-miss area, and can determine the rank depending on which one corresponds. By storing the ranks together in this way, it is possible to easily select and extract a scene according to the level. As a result, the drive recorder according to the present embodiment performs high-level post-processing such as extracting only a high-level scene and creating an overhead image or a sketch that can be used for accident proof, (All near-miss scenes) can be extracted in chronological order and used to reflect on the day's driving, and scenes up to the required rank can be easily extracted.

FIG. 23 is a flowchart for explaining the operation of the drive recorder 1a having the collision possibility analysis unit 24. In step S0, time-series stereo images I1 (i, j, t) and I2 (i, j, t) are acquired by the

stereo cameras

11 and 12, as shown in FIG. Subsequently, in step S1, the three-dimensional information P is obtained from the corresponding point search process of the reference image I1 (i, j, t + Δt) and the reference image I2 (i, j, t) at the time t + Δt to the three-dimensional reconstruction process. (I, j, t) is calculated. Next, in step S2, time-series association is performed by corresponding point search processing between the reference image I1 (i, j, t + Δt) at the time t + Δt and the reference image I1 (i, j, t) at the time t. Done. Subsequently, in step S3, as in step S1, the three-dimensional reconstruction process is performed from the corresponding point search process of the reference image I1 (i, j, t) and the reference image I2 (i, j, t) at time t. Is used to calculate the three-dimensional information Pp (i, j, t). Then, in the calculated three-dimensional information P (i, j, t) and the three-dimensional information Pp (i, j, t), the target object is extracted in step S4, and the target objects 4 and 5 are extracted. In step S5, the relative speed ν (i, j, t) is obtained from the position Pp (i, j, t) at time t and the position P (i, j, t) at time t + Δ. .

Thereafter, the process proceeds to the determination by the trigger determination unit 22, and in step S41, the time-series relative speeds ν (i, j, t) and ν (i, j, t + Δt) are taken in from the relative speed calculation unit 21 and are relative. The acceleration a is obtained and compared with the threshold value TH2 in step S42. If no trigger determination is made, the process returns to step S0 after waiting for time Δt, that is, one frame period in step S60. On the other hand, when the trigger determination is made in step S42, the process further proceeds to step S61, where the collision possibility analysis unit 24 determines the possibility of collision, and in step S62, if there is no possibility. Returns from step S60 to step S0, and if possible, the recording process of step S63 is performed. After the recording process, it is determined in step S64 whether or not the predicted collision time has exceeded a predetermined time. If not, the process returns from step S60 to step S0, and the process is continued. Is terminated. Thus, the drive recorder 1a according to the present embodiment can leave images at least before and after the collision, and can store information that is effective for analysis of accidents and near-miss events.

Thus, since the trigger determination is performed by detecting the relative speeds ν (i, j, t) and ν (i, j, t + Δt) from the images of the

stereo cameras

11 and 12, the drive recorder 1a of the present embodiment. Compared with the determination using a general acceleration sensor, it is possible to reduce malfunctions, and does not require complicated processing. It is possible to record reliably even when the preceding vehicle suddenly brakes. Further, when the drive recorder 1a of the present embodiment uses the

stereo cameras

11 and 12 that can acquire time-series three-dimensional information of the target objects 4 and 5 that are extremely effective for accident analysis, the

stereo camera

11 , 12 makes it possible to eliminate the need for a separate configuration when performing the trigger determination of the acceleration sensor or the like.

Here, the free running distance, the braking distance, and the turning angle of the steering wheel vary depending on the speed, while the relative speed calculated from the stationary object can be replaced with the vehicle speed of the own vehicle (vehicle 2). By performing the collision determination according to the vehicle speed (changing the size of the cube 2b), the determination accuracy can be improved.

This specification discloses various modes of technology as described above, and the main technologies are summarized below.

A drive recorder according to an aspect includes a stereo camera that acquires a time-series image, a recording unit that records the time-series image acquired by the stereo camera, and the time-series image acquired by the stereo camera. A relative velocity calculation unit for calculating a relative velocity with respect to the target object, and a trigger for starting the recording in response to the relative velocity calculated by the relative velocity calculation unit, or recording constantly by the recording unit A trigger determination unit for marking the time-series images.

In another aspect, preferably, the drive recorder is mounted on a vehicle, and when the time series image acquired by the image acquisition unit is recorded in the recording unit, the trigger determination unit Is a drive recorder that applies a trigger for starting the recording to the recording unit or marks the time-series images that are always recorded by the recording unit, wherein the image acquisition unit includes a stereo A relative speed calculation unit configured to calculate a relative speed with respect to a target object from an image acquired by the stereo camera, wherein the trigger determination unit responds to the relative speed calculated by the relative speed calculation unit. Then, the trigger is given to the recording unit, or the marking is performed on the time-series image constantly recorded by the recording unit.

According to this configuration, the relative speed calculation unit calculates the relative speed with respect to the target object such as being on the lane from the time-series image acquired by the stereo camera, and the trigger determination unit calculates the relative speed. When the relative speed calculated by the relative speed calculation unit is equal to or greater than a predetermined threshold, the recording is triggered by the recording unit, or the marking is performed on a time-series image constantly recorded by the recording unit. Applied.

Therefore, since the drive recorder having such a configuration detects the relative speed from the time-series image and determines the trigger or the marking, it can reduce malfunction as compared with the determination using a general acceleration sensor. It is possible to record reliably in the case of a so-called near-miss scene, for example, when the preceding vehicle steps on the brakes suddenly, when there is a high possibility of a collision with a simple method that does not require complicated processing. it can. Further, the drive recorder having such a configuration can acquire time-series three-dimensional information of a target object that is extremely effective for accident analysis by using a stereo camera. Further, in the drive recorder having such a configuration, when not all the areas are searched but the area to be the target object is extracted in advance and the search is performed, the search process for the extra area is performed. There is no need.

According to another aspect, in the above-described drive recorder, the relative speed calculation unit further calculates a relative acceleration based on the calculated relative speed, and the trigger determination unit responds to the relative acceleration. Then, a trigger for starting the recording is given, or marking is performed on the time-series images constantly recorded by the recording unit.

According to this configuration, by using the relative acceleration for the determination of the trigger or the marking, the relative acceleration changes from a relatively stable median value to a certain range. Easy.

According to another aspect, the above-described drive recorder further includes a collision possibility analysis unit that analyzes a collision possibility with respect to a target area of the trigger determination unit, and the trigger determination unit is configured to respond to the analysis result. Then, a trigger for starting the recording is given, or marking is performed on the time-series images constantly recorded by the recording unit.

In the drive recorder having such a configuration, after the determination based on the relative speed and the relative acceleration is performed by the trigger determination unit, the collision possibility analysis unit further includes, for example, a vehicle Malfunctions can be reduced by analyzing the region of the target object in more detail, such as whether it is in a lane.

According to another aspect, in the above-described drive recorder, the relative speed calculation unit further calculates a distance to the target object, and calculates a collision prediction time from a relationship between the calculated distance and the relative speed. The trigger determination unit gives a trigger for starting the recording when the predicted collision time is within a predetermined threshold value, or marks the time-series images that are constantly recorded by the recording unit. is there.

The drive recorder having such a configuration can make a determination with high accuracy by considering not only the relative speed but also the distance to the target object.

In another aspect, in the above-described drive recorder, the relative speed calculation unit calculates the relative speed from the disparity information calculated by the corresponding point search process for the stereo image acquired by the stereo camera. Is.

The drive recorder having such a configuration can calculate the relative speed with high accuracy by calculating the relative speed by the corresponding point search process.

In another aspect, in the above-described drive recorder, the relative speed calculation unit sets a calculation area for calculating the corresponding point as a fixed area on the screen.

According to this configuration, by setting the calculation area for calculating the corresponding point as a fixed area on the screen, it is not necessary to perform a special calculation process for determining the calculation area, and the relative speed calculation unit Only the process of calculating the relative speed from the obtained parallax information may be performed in the search process.

According to another aspect, in the above-described drive recorder, the collision possibility analysis unit determines the collision possibility based on position information of a target area of the trigger determination unit, direction and length of a three-dimensional movement vector. It is judged from.

According to this configuration, the collision possibility analysis unit performs the determination using the position information of the region determined as the trigger target by the trigger determination unit and the direction and length of the three-dimensional movement vector. This drive recorder can make a determination in consideration of the movement of the target object.

In another aspect, in the above-described drive recorder, the collision possibility analysis unit enlarges the extraction area of the target object as the relative speed increases.

In the drive recorder having such a configuration, the collision possibility analysis unit can perform the analysis with higher accuracy by changing the extraction area of the target object according to the relative speed.

In another aspect, in the above-described drive recorder, the collision possibility analysis unit enlarges the extraction area of the target object on the moving direction side of the target object.

In the drive recorder having such a configuration, the collision possibility analysis unit changes the extraction area of the target object in accordance with the moving direction of the target object, so that, for example, in a scene that pops out from the measurement method, it is more accurate. High analysis can be performed.

In another aspect, in the above-described drive recorder, the stereo camera is mounted on a vehicle, and the collision possibility analysis unit has determined that the collision possibility is a trigger target by the trigger determination unit. An angle with respect to the traveling direction of the host vehicle is detected for the region, and when the change of the angle with respect to time is within a predetermined threshold value, it is analyzed that there is a possibility of collision.

According to this configuration, the collision possibility analysis unit analyzes the area of the target object in more detail as described above in order to reduce malfunctions. If the angle with respect to the traveling direction of the host vehicle does not change, it is determined that there is a possibility of a collision if the vehicle runs as it is. Therefore, the drive recorder having such a configuration can determine whether or not there is a possibility of collision with a simple parameter such as an angle with respect to the traveling direction of the host vehicle.

According to another aspect, in the above-described drive recorder, the collision possibility analysis unit performs rank classification according to the analysis result of the collision possibility and further stores the rank in the recording unit. is there.

The drive recorder having such a configuration can easily extract scenes up to a required rank when used for reflection of drive results.

Further, in another aspect, in the above-described drive recorder, the trigger determination unit causes the recording unit to trigger a recording end when at least the predicted collision time calculated by the relative speed calculation unit is exceeded. An end marking is given to the time-series image that is given or always recorded by the recording unit.

The drive recorder having such a configuration can leave images at least during the period before and after the collision, and can store information useful for analyzing accidents and near-miss events.

This application is based on Japanese Patent Application No. 2009-70467 filed on Mar. 23, 2009, the contents of which are included in the present application.

In order to express the present invention, the present invention has been properly and fully described through the embodiments with reference to the drawings. However, those skilled in the art can easily change and / or improve the above-described embodiments. It should be recognized that this is possible. Accordingly, unless the modifications or improvements implemented by those skilled in the art are at a level that departs from the scope of the claims recited in the claims, the modifications or improvements are not covered by the claims. It is interpreted that it is included in

According to the present invention, a drive recorder can be provided.

Claims

A stereo camera that acquires time-series images;
A recording unit for recording the time-series images acquired by the stereo camera;
A relative speed calculation unit that calculates a relative speed with respect to a target object from the time-series image acquired by the stereo camera;
In response to the relative speed calculated by the relative speed calculation unit, a trigger determination unit that gives a trigger for starting the recording or performs marking on the time-series image constantly recorded by the recording unit is provided. A drive recorder characterized by this.
The relative speed calculation unit further calculates a relative acceleration based on the calculated relative speed,
2. The trigger determination unit gives a trigger for starting the recording in response to the relative acceleration, or marks the time-series image constantly recorded by the recording unit. Drive recorder as described in.
A collision possibility analysis unit for analyzing the possibility of collision with respect to the target area of the trigger determination unit;
The trigger determination unit gives a trigger to start the recording according to the analysis result, or marks the time-series image constantly recorded by the recording unit. The drive recorder according to claim 2.
The relative speed calculation unit further calculates a distance to the target object, calculates a collision prediction time from a relationship between the calculated distance and the relative speed,
The trigger determination unit gives a trigger to start the recording when the predicted collision time is within a predetermined threshold value, or marks the time-series image constantly recorded by the recording unit. The drive recorder according to claim 1.
5. The relative speed calculation unit calculates the relative speed from disparity information calculated by corresponding point search processing for a stereo image acquired by the stereo camera. The drive recorder according to item 1.
The drive recorder according to claim 5, wherein the relative speed calculation unit sets a calculation area for calculating the corresponding point as a fixed area on a screen.
The drive according to claim 3, wherein the collision possibility analysis unit determines the collision possibility from position information of a target area of the trigger determination unit and a direction and a length of a three-dimensional movement vector. Recorder.
The drive recorder according to claim 3 or 7, wherein the collision possibility analysis unit increases an extraction area of the target object as the relative speed increases.
The drive recorder according to claim 3 or 7, wherein the collision possibility analysis unit increases an extraction area of the target object on a moving direction side of the target object.
The stereo camera is mounted on a vehicle,
The collision possibility analysis unit detects an angle of the collision possibility with respect to a target area of the trigger determination unit with respect to a traveling direction of the own vehicle, and a change of the angle with respect to time is within a predetermined threshold. The drive recorder according to claim 3, wherein the drive recorder is analyzed as having a possibility of collision.
The drive recorder according to claim 3 or 10, wherein the collision possibility analysis unit performs rank division according to the analysis result of the collision possibility and further stores the rank in the recording unit. .
The trigger determination unit gives the recording end trigger to the recording unit at least when the predicted collision time calculated by the relative speed calculation unit is exceeded, or the time-series image constantly recorded by the recording unit The drive recorder according to claim 4, wherein end marking is performed on the drive recorder.