JP2017162281A - Moving body detection system and data consistency determination method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving body detection system and data consistency determination method that can prevent wrong operations due to recognition of two or more moving bodies as a single body at a low load and at a low cost.SOLUTION: A moving body detection system comprises: imaging means (11) picking up an image in a traveling direction of an own vehicle; image processing means (12) for extracting moving bodies from the image, and outputting a location, width and speed of each moving body; and means (21) for predicting a future location of each moving body on the basis of the location, width and speed of each moving body. The moving body detection system comprises data consistency determination means (22) that includes: storage means (20) storing time-sequentially the location and width of each moving body; and first comparison and determination means comparing the width in a current frame of each moving body with the width in previous one or more frames stored in the storage means, and determining that the moving body is formed of a plurality of moving bodies when a change in the width is equal to or greater than a threshold.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a mobile body detection system that detects a mobile body and predicts its future position, and a data consistency determination method thereof.

  When it is judged that the possibility of a collision is high by recognizing a moving object such as a pedestrian or a bicycle that appears in the field of view of an in-vehicle camera for the purpose of safe driving support or automatic driving in a car, predicting its future position Various moving body detection systems that activate avoidance processing such as braking control and warning have been developed (see Patent Documents 1 and 2).

  In such a moving body detection system, two or more moving bodies, for example, two pedestrians may overlap and be recognized as one person at substantially the same position. The purpose of this system is not to detect the number of moving objects, so the misidentification of the number of moving objects is not a problem, but if the movement of each moving object is different, the future based on the speed and position information of the moving object There is a possibility that the position is not properly predicted and malfunction of the avoidance process (invoking in an unnecessary scene) may occur.

  Patent Document 1 discloses a method of detecting an overlap of two objects from a difference between left and right parallax using a stereo camera. However, this is a method that can be applied only when there is a difference in the depth direction, and cannot be applied when the difference in depth is small.

  Patent Document 2 discloses a method for targeting still images and using the width of pedestrian candidates for estimating the number of pedestrians at the image recognition stage. However, since a threshold is simply set for the horizontal width of a still image (one frame image in the case of a moving image) and one person and two or more persons are identified, it cannot be identified with a width smaller than the width defined as “one person”. Moreover, in consideration of measurement errors, the threshold value must be set to a value larger than the average width for one pedestrian, and the identifiable range is further limited.

  Naturally, there is room for improvement in detection accuracy by increasing the time required for detection determination (number of image frames), but in the case of safe driving support and mobile object detection and collision avoidance in automatic driving, the responsiveness associated with the above is required. The decrease is not preferable, and a simple and highly responsive practical measure is desired from the viewpoint of load and cost on the apparatus.

Japanese Patent Laid-Open No. 2003-150938 JP 2006-10652 A

  The present invention has been made in view of the above circumstances, and its object is to detect a moving body that can prevent malfunction caused by recognizing two or more moving bodies as a single unit at low load and low cost. To provide a system and a data consistency determination method.

  In order to solve the above-mentioned problems, the present inventor has intensively studied and, as a result, when two or more moving bodies are recognized as single, (i) two or more moving bodies at substantially the same position are regarded as a single moving body. There are a first case of recognition (FIG. 10) and (ii) a second case of recognition of two or more moving bodies at substantially the same position as one of the different moving bodies for each frame (FIG. 11). I understood that.

  In the first case, for example, as shown in FIGS. 10A and 10B, data detected for the pedestrian 62 on the front side in the traveling direction in the frames 1 and 2 that are in chronological order. Since the upper center of gravity position 60p is on the left side in the figure, the moving speed of the center of gravity position during this period is calculated from the difference between the frames 1 and 2 with a larger value than the actual speed of the pedestrian 62, and the predicted speed is also large. Become.

  On the other hand, in the second case, as shown in FIGS. 11 (a) and 11 (b), in the frames 1 and 2 that are in chronological order, the center of gravity position 70p on the detected data is the left pedestrian. By changing from 72 to the pedestrian 71 on the right side, the moving speed of the center of gravity position during this period is calculated from the difference between the frames 1 and 2 with a larger value than the actual speed of the pedestrian 71, and the predicted speed also increases.

  When the predicted speed is calculated to be large as described above, there is a possibility that the avoidance process is activated even though the possibility of collision is low. On the other hand, when the positional relationship between the two pedestrians is opposite in each figure, the predicted speed is calculated to be small, and the avoidance process may not be activated.

  However, in the first case, the width 60w on the detected data changes. In the second case, the width 70w on the detected data does not change, but the speed changes discontinuously. Therefore, by performing data consistency determination assuming each case, the inventors have arrived at the present invention by obtaining the knowledge that an error can be detected without losing responsiveness and malfunction can be prevented.

That is, the present invention provides an image pickup means (11) for picking up an image of the traveling direction of the host vehicle, and an image processing means (12) for extracting the moving body from the image and outputting the position, width and speed of each moving body. And a mobile body detection system comprising means (21) for predicting the future position of each mobile body based on the position, width, and speed of each mobile body,
Storage means (20) for storing the position and width of each mobile object in time series, and comparing the width of each mobile object in the current frame with the width in one or more immediately preceding frames stored in the storage means. If the change in the width is equal to or greater than the threshold, the data consistency determining means (22) including the first comparison determining means for determining that the moving body is composed of a plurality of moving bodies is provided.

  In the above configuration, the width of each moving body in the current frame is compared with the width in one or more immediately preceding frames that have already been recorded, and the change is detected, so that it is affected by individual differences in the width of each moving body. If the rate of change or absolute value is equal to or greater than the threshold value, there is no immediate data consistency, that is, it can be determined that the moving object data includes a plurality of moving objects, and the width is simply given a threshold value. Therefore, there is an advantage that a mobile body detection system that can detect with higher accuracy than the case of only the above and has excellent response can be configured at low cost.

  In the present invention, the data consistency determination means (22) detects a speed distribution peak in a predetermined number of frames immediately before the storage means for storing the speed of each moving body in time series and the storage means. If the peak is 2 or more, it is preferable to further include a second comparison / determination unit that determines that the moving body is composed of a plurality of moving bodies.

  In the above configuration, when a plurality of peaks are detected in the velocity distribution in a predetermined number of frames immediately before each moving body, the frequency is low due to the instantaneous movement of the detection target (center of gravity position) between two or more moving bodies. Assuming that high-speed velocity peaks and discontinuous velocity changes have occurred, it is possible to immediately determine that the mobile data does not have data consistency, and that the mobile data contains multiple mobile objects, making it more accurate and responsive. There is an advantage that it is possible to construct a moving body detection system excellent in the above.

  In the present invention, when the position of each moving body in the current frame and the position in the immediately preceding frame are not substantially changed, the first comparison / determination means has a width change smaller than the threshold value. If the threshold is 2 or more, it is preferable that the mobile body is configured to be determined to be composed of a plurality of mobile bodies.

  In the above configuration, when a change in the width is detected even though the position of each moving body has not changed, it is highly possible that the moving body includes a plurality of moving bodies. It can be determined that there is no data consistency and the moving body data includes a plurality of moving bodies, and the responsiveness in this case can be further improved.

  In the mobile body detection system of the present invention described above, the means (21) for predicting the future position of the mobile body, the data consistency judgment means (22), and the respective comparison judgment means constituting it are respectively It is assumed that the computer is configured by a computer (CPU, memory, I / F) capable of executing a program recording a prescribed procedure.

The present invention captures an image of the traveling direction of the host vehicle, extracts a moving body from the image by image recognition, and predicts a future position of the moving body based on the position, width, and speed of the moving body. A data consistency determination method in a body detection system,
Storing the position and width of each moving body in time series (210);
Comparing the width of each moving body in the current frame with the width of one or more immediately preceding stored frames and determining that the moving body is composed of a plurality of moving bodies when the change in width is equal to or greater than a threshold value ( 221), and the data consistency determination method.

  In the method of the present invention described above, the storing step (210) includes storing the speed of each moving object in time series, and detects a peak of a velocity distribution in a predetermined number of frames immediately before being stored. In the case of two or more, it is preferable that the method further includes a step (233) of determining that the moving body includes a plurality of moving bodies.

  In the method of the present invention, when the position of each moving body in the current frame and the position in the immediately preceding frame are not substantially changed, if the change in the width is equal to or greater than a second threshold smaller than the threshold, It is preferable that the method further includes a step (222) of determining that the moving body includes a plurality of moving bodies.

  As described above, the mobile object detection system and the data consistency determination method according to the present invention can prevent malfunction caused by recognizing two or more mobile objects as a single unit at a low load and at a low cost, and can be appropriately avoided. It is advantageous in taking measures.

It is a block diagram which shows the moving body detection system which concerns on this invention embodiment. It is a flowchart which shows operation | movement of the mobile body detection system which concerns on this invention embodiment. It is a flowchart which shows the data consistency determination process in the mobile body detection system which concerns on this invention embodiment. (A) is one moving body, (b) is a histogram showing the velocity distribution when two moving bodies are included. It is a schematic diagram which shows the recognition image of 2 frames which are in a time series in the case of recognizing two mobile bodies independently. 6 is a graph showing changes in position and width over time of the moving body of FIG. 5. 6 is a graph showing the relationship between the position and width of the moving body in FIG. 5. It is the graph which recognized the histogram which shows the velocity distribution in the case of containing two moving bodies as one moving body, and fitted it with the function of superposition of a normal distribution. FIG. 9 is a graph obtained by fitting with a function of superimposing a normal distribution including two peaks on the histogram of FIG. 8. It is a schematic diagram which shows the example of the recognition image of 2 frames which are in the time series at the time of recognizing two mobile bodies independently. It is a schematic diagram which shows the example of the recognition image of 2 frames which are in a time series in the case of recognizing two moving bodies as any one different moving body for every flame | frame.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
As shown in FIG. 1, the moving body detection system according to the present invention includes a stereo camera 11 and an image processing apparatus 1 including a recognition processing unit 12, a data storage unit 20, a moving body position prediction unit 21, and data matching. The control unit 2 mainly includes a sex determination unit 22 and a control amount calculation unit 23.

  The stereo camera 11 captures an image in the traveling direction of the host vehicle, and the captured image is acquired in the VRAM of the recognition processing unit 12. The recognition processing unit 12 extracts moving objects from the acquired image by recognition processing, and outputs the position, width, and speed of each moving object to the controller 2 at a predetermined frame rate. The controller 2 predicts the future position of each moving body in the procedure as shown in FIG. 2 and determines the possibility of collision with the own vehicle.

(Basic control process)
First, the controller 2 acquires the position (distance, center of gravity position), width (horizontal width), and speed (lateral speed) of each moving object output from the image processing apparatus 1 (100), and the data storage unit 20 The position, width, and speed of each moving object over the past n frames are stored in time series. The mobile body position prediction unit 21 calculates the future position of each mobile body based on the position, width, and speed of each mobile body (111), and based on the vehicle speed and acceleration information of the host vehicle obtained from the vehicle speed sensor 3 The future position of is calculated (112). Vehicle information such as a rudder angle sensor and a yaw rate sensor may be used in combination.

  In parallel with the mobile object position prediction unit 21 calculating the future position of each mobile object and the host vehicle, the data consistency determination unit 22 applies the past n frames of each mobile object stored in the data storage unit 20. With reference to the position, width, and speed data, consistency with the data of the current frame is determined (120, 121). A specific determination procedure will be described later.

  If there is no problem in consistency, the possibility of collision is determined (140), and if there is a problem in consistency, error determination or emergency control determination is performed without determining the possibility of collision (130). ). It is also possible to determine the possibility of collision first and then determine the consistency.

  If it is determined that there is a possibility of collision with any moving body, the control amount calculation unit 23 calculates the braking control amount necessary to avoid the collision, and the collision between the host vehicle and the moving body occurs. When it is determined whether or not a margin time (TimeToCollision; TTC) is equal to or less than a predetermined value (142), and if it is equal to or less than the predetermined value, the automatic brake 6 is operated by the control processing device 4 (160) and is not less than the predetermined value. Activates the alarm device 5 (150). When it is determined that there is no possibility of collision, and after the alarm device 5 is activated, the data storage unit 20 is updated to data of the past n frames including the current frame, and the process proceeds to processing of the next frame.

(Data integrity judgment process)
Next, a specific data consistency determination procedure will be described with reference to FIG.
As described above, when two or more moving objects are recognized as single, (i) a first case where two or more moving objects at substantially the same position are recognized as a single moving object; and (ii) approximately Since there is a second case in which two or more moving bodies at the same position are recognized as any one of the different moving bodies for each frame, the data consistency determination is performed assuming these cases. Although the following description shows a case where two inspections are performed in parallel, they can be sequentially performed.

(I) When Recognizing Two or More Moving Objects as Single Moving Objects First, the data consistency determination unit 22 first determines the position (distance, center of gravity position), width (horizontal width), speed (horizontal width) of each moving object in the current frame. (Direction speed) is acquired (200), and the position, width, and speed data of each moving object over the past n frames are read from the data storage unit 20 (210), and the width of each moving object in the current frame is determined in the past n frames. If the change in the width is equal to or greater than the threshold value compared to the width, it is determined that the moving body is composed of a plurality of moving bodies (step 221).

  For example, as shown in FIG. 5 (a), two pedestrians 51 and 52 moving to the left side in the figure at substantially the same position in a certain frame 1 are single pedestrians (center of gravity position 50p). As shown in FIG. 5B, a case is assumed in which one pedestrian 52 preceding in the subsequent frame 2 is reversed and turns to the right side in the figure.

  As shown in FIG. 6, the trajectory of the center of gravity of the pedestrians 51 and 52 during this period is the same direction for the pedestrians 51p and 52p (the left direction in FIG. ), One pedestrian 52p is reversed near the second frame, and is accelerated after intersecting the pedestrian 51p near the fourth frame.

  When these two pedestrians are recognized as one person, in 4 to 10 frames, the trajectory of the center of gravity position 50p is recognized with a smaller amount of movement than the trajectory of one pedestrian 52p, and thus the speed is recognized to be low. However, the predicted speed is estimated to be small and the predicted arrival position may be estimated in the foreground, but the width 50w of the moving body suddenly increases after 4 frames when two pedestrians 51 and 52 intersect. When the change in width is greater than or equal to the threshold, for example, (1) when the amount of change in width is greater than or equal to the threshold, (2) when the rate of change (slope) in width is greater than or equal to the threshold, (3) When the mobile body (50w) increases continuously over m frames (m is 2 to 3), (4) when an inflection point (peak) occurs in the width change curve, the moving body (50w) It can be determined that a plurality of moving bodies (51, 52) are included.

  Next, when the position of each moving body in the current frame and the position in the immediately preceding frame are not substantially changed, if the change in width is equal to or larger than a second threshold value smaller than the threshold value, the moving body is It is determined that it is composed of a plurality of moving bodies (step 222).

  For example, in the case shown in FIGS. 5 (a) and 5 (b) described above, if a correlation 50pw between the center of gravity and the width of the pedestrians 51 and 52 is taken, a position -4 is obtained as shown in FIG. Can be determined that the moving body (50pw) includes a plurality of moving bodies (51, 52) even if the change in the width is slight.

  When it is determined in the above steps (221) and (222) that a plurality of moving bodies are included, it is determined whether or not TTC is equal to or smaller than a predetermined value (step 225), and TTC is equal to or smaller than a predetermined value. In the case, emergency control determination is performed (226).

  If it is determined that the urgency is high, the time change amount of the edge (end coordinate of the currently recognized width) instead of the center of gravity is calculated as the speed, and the currently recognized width is divided into 1/3. Control coordinates are determined by adopting the coordinates to be used as the center of gravity positions of the two moving bodies, and if there is a possibility of collision, the automatic brake 6 is implemented. In this case, it is desirable to change the threshold value for control determination as an emergency process. For example, in the case of automatic braking, when the latest recognition result is not on the course, only the warning may be used.

  On the other hand, if the TTC is not less than or equal to the predetermined value (not urgent) in step 225, an error is returned to the image processing apparatus 1 (recognition processing unit 12) (228). At this time, a threshold value set for the recognition processing unit 12 (such as a threshold value for determining edge detection) is corrected. In this case, the control determination based on the current position is performed, but the control determination considering the moving body position prediction is not performed.

  If it is determined in the above steps (221) and (222) that there is no problem in data consistency, normal control determination is performed (224).

(Ii) When Recognizing Two or More Moving Objects as Any Different for Each Frame First, a histogram (speed distribution) of speeds in the past n frames of each moving object stored in the data storage unit 20 is created (231). ). Next, the number of peaks in the histogram is counted (232), and when two or more peaks are detected, it is determined that the moving body is composed of a plurality of moving bodies (233).

  For example, the solid line 40 in FIG. 4A shows a histogram of one pedestrian who is almost stationary, and has a normal distribution centered on one peak near a speed of 0.5 km / h. . A broken line 40 'in the figure shows a histogram of one moving pedestrian, and shows a speed distribution that is biased to the left and right of one peak near a speed of 4.1 km / h.

  On the other hand, in FIG. 4B, in the histogram of the past n frames of the moving object detected as one person, in addition to the peak 41 centered around the speed of 0 km / h, the vicinity of the speed of 5 km / h is shown. Since the center peak 42 appears, as shown in FIG. 11, the position of the center of gravity on the data detected in the time-series contextual frame moves between two pedestrians. It can be seen that the velocity (42) has appeared.

  Therefore, when there are two or more peaks, fitting is performed using a function in which normal distributions are superposed (234), and only the data to which each peak belongs is used, and recalculation is performed for each moving object. The position and speed are obtained (235).

As a method for determining the peak, for example, when the frequency of the nth representative point of speed in the histogram is P (n),
{P (n-1) -P (n)} * {P (n + 1) -P (n)} <-1
And,
| P (n-1) -P (n) |> C (constant value)
| P (n + 1) −P (n) |> C (constant value)
In this case, it is obtained by counting the number of peaks, assuming that the point is a peak. This method is an example, and other methods may be used.

The function form to fit is for two peaks:
A ₁ * exp (− (x−μ ₁ ) ² / 2δ ₁ ² ) + A ₂ * exp (− (x−μ ₂ ) ² / 2δ ₂ ² )
And the variables are A ₁ , A ₂ , μ ₁ , μ ₂ , δ ₁ , δ ₂ .

The range of “data belonging” is the range of μ ₁ ± δ ₁ and μ ₂ ± δ ₂ (half-value width) with respect to the peaks at the center points μ ₁ and μ ₂ , respectively, and the portion where the two peaks overlap is not used. This range is an example and may be different. If a plurality of peaks are detected, an error and a detection position are returned to the image processing apparatus 1 (recognition processing unit 12) (236).

An example of selection and recalculation of the “belonging data” will be given. For example, consider a case where the position and speed from the left end of the screen are acquired as measurement data, and the speed has two peaks. When the centers of the two peaks are 4 km / h and 6 km / h, respectively, and the half-value width is 0.5 km / h, the measurement data is (3 m, 4.2 km / h), (5 m, 6.3 km). / h), (6 m, 7 km / h), (3.5 m, 4 km / h), (5.8 m, 6.1 km / h), the data belonging to peak 1 is 4 km ± 0.5, Since the data belonging to peak 2 is 6 km ± 0.5,
-Data belonging to peak 1: (3 m, 4.2 km / h), (3.5 m, 4 km / h)
・ Data belonging to peak 2: (5 m, 6.3 km / h), (5.8 m, 6.1 km / h)
And recalculate the future position for each different object.

  In the determination of whether or not to perform an emergency stop brake of a car, if the target is within 1 second from recognition to brake operation, the difference in mileage between 6 km / h and 4 km / h pedestrians is about 56 cm / sec. If the recognition method has a resolution of about 10 cm, the data consistency determination process according to (i) and (ii) can be performed.

  In addition, when the moving body data includes three or more moving bodies (pedestrians), in the method of detecting the change in the width of (i) above, the width and the position of the center of gravity are determined by two persons at both ends, so that collision is possible. If it is not urgent in the sex determination, there is no functional problem even if only “error” is returned, and the emergency process may be the same as in the case of two persons.

  Next, a specific example of fitting to the histogram of speeds in the past n frames of each moving object will be described with reference to FIGS.

  First, when a histogram as shown by dots in FIG. 8 is obtained, when fitting as a single moving body 50d, a normal distribution near the pedestrian on the right side in the figure is obtained, but the deviation is large ( Especially, the left part) is an error judgment.

  In this case, it may be determined simply by the magnitude of the error square value, but the ratio of the error square value of only the positive point and the error square value of only the negative point from the pedestrian position x obtained by fitting. Can also be determined. This is because, when the deviation is large only on one side of the normal distribution, it is determined that there is an error factor on one side, that is, there is another pedestrian.

  Next, as shown in FIG. 9, assuming that there are two pedestrians 51d and 52d and performing fitting, the respective peak positions can be obtained as 51d: 0.9 and 52d: 5.3. In this case, it is possible to predict the positions of the pedestrians 51d and 52d by performing recalculation using only the values that fall within the half-value width of the respective normal distributions (ranges indicated by double arrows in the figure).

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, Based on the technical idea of this invention, various deformation | transformation and a change are further possible.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Control controller 3 Vehicle speed sensor 4 Control processing apparatus 5 Alarm apparatus 6 Automatic brake 11 Stereo camera 12 Recognition processing part 20 Data storage part 21 Mobile body position prediction part 22 Data consistency determination part 23 Control amount calculation part 51, 52 Pedestrian (moving body)
50p, 51p, 52p position (center of gravity position)
50w width (width)

Claims (6)

Imaging means for capturing an image of the traveling direction of the host vehicle;
Image processing means for extracting moving bodies from the image and outputting the position, width, and speed of each moving body;
Means for predicting the future position of each mobile body based on the position, width and speed of each mobile body;
In the mobile object detection system with
Storage means for storing the position and width of each moving body in time series;
When the width of each moving body in the current frame is compared with the width in one or more immediately preceding frames stored in the storage means, and the change in width is equal to or greater than a threshold, the moving body is composed of a plurality of moving bodies. First comparison determination means for determining;
A moving body detection system comprising data consistency determination means including The data consistency determination means includes
Storage means for storing the speed of each moving body in time series;
A second comparison / determination unit that detects a peak of velocity distribution in a predetermined number of frames immediately before stored in the storage unit, and determines that the mobile unit is composed of a plurality of mobile units if the peak is 2 or more; The moving body detection system according to claim 1, further comprising:   The first comparison / determination means, when the position of each moving body in the current frame and the position in the immediately preceding frame are not substantially changed, the change in the width is equal to or greater than a second threshold smaller than the threshold. If so, the mobile body detection system according to claim 1, wherein the mobile body detection system is configured to determine that the mobile body includes a plurality of mobile bodies. In a moving body detection system that captures an image of the traveling direction of the host vehicle, extracts a moving body from the image by image recognition, and predicts the future position of the moving body based on the position, width, and speed of the moving body. A method for determining data consistency,
Storing the position and width of each moving body in time series;
Comparing the width of each moving object in the current frame with the width of one or more immediately preceding stored frames, and determining that the moving object is composed of a plurality of moving objects when the change in width is equal to or greater than a threshold; , Including the data consistency determination method. The step of storing includes storing the speed of each moving body in time series,
5. The data consistency according to claim 4, further comprising a step of detecting a peak of velocity distribution in a predetermined number of frames immediately before being stored, and determining that the moving body is composed of a plurality of moving bodies when the peak is 2 or more. Judgment method.   When the position of each moving body in the current frame and the position in the immediately preceding frame are not substantially changed, a plurality of the moving bodies are provided if the change in the width is equal to or larger than a second threshold value smaller than the threshold value. The data consistency determination method according to claim 4, further comprising a step of determining that the mobile body is composed of a plurality of moving bodies.

Images