JP2017142661A - Vehicle periphery monitoring system and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately detect moving bodies by reducing a possibility of erroneously detecting a partial area of a background as a moving body.SOLUTION: A feature point extraction section 6c extracts feature points of an edge image. A distance calculation section 6d calculates a distance between feature points in vertical direction. A condition determination section 6e determines whether or not there are feature points satisfying a first condition that a first prescribed number of more pairs whose distances of the pair between the feature points in the vertical direction are identical with each other, are present in the vertical direction, and satisfying a second condition that a second prescribed number or more groups satisfying the first condition are present in a horizontal direction. A feature point exclusion section 6f excludes feature points satisfying both of the first condition and second condition, from the edge image as exclusion objects. An optical flow calculation section 6g calculates an optical flow using the edge image from which the feature points as the exclusion objects are excluded.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a vehicle periphery monitoring system and a computer program.

  As a technique for detecting an object such as a pedestrian or another vehicle around the vehicle as a moving body, there is a technique for calculating an optical flow (that is, a movement vector of the object). In the method of calculating the optical flow, the similarity of the feature points is obtained from the frames before and after the image taken by the camera, the positions are compared, and an object having the same shape and the position changing in the frames before and after is moved. Detect as. However, depending on the shooting environment, it may be difficult to distinguish between a stationary object and a moving object. For example, Patent Document 1 discloses a method for distinguishing a background, a stationary object, and a moving object by calculating an optical flow of a normal image captured by a camera and calculating an optical flow of an overhead image obtained by converting the viewpoint of the normal image. Is disclosed.

JP 2011-48520 A

  However, in the method disclosed in Patent Document 1, if the background includes an isomorphous continuous pattern such as a brick wall, a number of similar feature points are extracted in the preceding and following frames, and the similar feature points are included. There is a possibility that the area is erroneously detected as a moving body. If a partial area of the background is erroneously detected as a moving object, there is a problem that accuracy of detecting a moving object such as a pedestrian or another vehicle decreases.

  The present invention has been made in view of the above-described circumstances, and the object thereof is to reduce the possibility of misdetecting a partial region of the background as a moving object, and to appropriately detect the moving object. An object is to provide a vehicle periphery monitoring system and a computer program.

  According to the first aspect of the present invention, the image acquisition unit (6a) acquires an image photographed by the photographing unit (3) for photographing the periphery of the vehicle. The edge image acquisition unit (6b) acquires an edge image from the image. The feature point extraction unit (6c) extracts feature points of the edge image. The distance calculation unit (6d) calculates the distance between the feature points in the first predetermined direction. The condition determining unit (6e) satisfies a first condition in which a pair in which the distance between feature points in the first predetermined direction is the same as the other satisfies the first condition in which the first predetermined number is greater than or equal to the first predetermined direction. It is determined whether or not there is a feature point satisfying the second condition in which the group satisfying the condition of 1 satisfies a second predetermined number or more in the second predetermined direction. The feature point exclusion unit (6f) excludes feature points that satisfy both the first condition and the second condition from the edge image as exclusion targets. The optical flow calculation unit (6g) calculates an optical flow using the edge image from which the feature points to be excluded are excluded.

  A feature point that satisfies both the first condition and the second condition is excluded from the edge image as an exclusion target, and the optical flow is calculated using the edge image from which the feature point to be excluded is excluded. The background contains an isomorphic continuous pattern such as a brick wall, and even if many similar feature points are extracted in the previous and subsequent frames, the optical flow is calculated by excluding the feature points to be excluded from the edge image. The possibility of erroneous detection of a partial area of the background as a moving object can be reduced. Thereby, moving bodies, such as a pedestrian and another vehicle, can be detected appropriately, for example.

Functional block diagram showing an embodiment of the present invention flowchart Figure showing the transition until the feature points to be excluded are excluded (Part 1) The figure which shows the aspect which calculates the optical flow The figure which shows the conventional technique when the vehicle has stopped The figure which shows the method of this invention when the vehicle has stopped The figure which shows the conventional technique when the vehicle is moving The figure which shows the method of this invention when the vehicle is moving Diagram showing a technique that does not leave the feature points of the remaining object Diagram showing the technique for remaining feature points of the target Figure showing the transition until exclusion of feature points to be excluded (Part 2) Figure showing the transition until excluding feature points to be excluded (Part 3) Figure showing the transition until excluding feature points to be excluded (Part 4) Fig. 5 shows the transition until the feature points to be excluded are excluded.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The vehicle periphery monitoring system 1 includes a vehicle device 2, a camera 3 (corresponding to a photographing unit), a display device 4, and a notification device 5 (corresponding to a notification unit). The camera 3 is a camera that is disposed at the rear part of the vehicle body and has a field of view (that is, an angle of view) in the rear of the vehicle as the periphery of the vehicle. For example, the camera 3 starts the photographing operation on the condition that the shift position is moved to the retracted position. That is, the camera 3 functions as an in-vehicle device that provides driving assistance when the vehicle moves backward.

  When the camera 3 performs a photographing operation, the camera 3 outputs an image signal including the photographed image to the vehicle device 2. In addition, the output of the image signal from the camera 3 to the vehicle device 2 may be performed by data communication by wired connection or data communication by wireless connection. For example, USB (Universal Serial Bus) may be used for data communication by wired connection. For data communication by wireless connection, for example, Bluetooth (registered trademark), BLE (Bluetooth Low Energy), WiFi (Wireless Fidelity) (registered trademark), or the like may be used. Further, the camera 3 may continuously output the image signal to the vehicular device 2 or may intermittently output the image signal at a predetermined cycle (for example, a cycle of several milliseconds).

  The display device 4 is a display arranged in the vehicle interior. When a display command signal is input from the vehicle device 2, the display device 4 displays an image of the rear of the vehicle taken by the camera 3. The notification device 5 is a speaker arranged in the vehicle interior, and outputs a warning sound when a notification command signal is input from the vehicle device 2. The output of the display command signal from the vehicle apparatus 2 to the display device 4 and the output of the notification command signal to the notification device 5 may be performed by data communication by wired connection or data communication by wireless connection.

  The vehicle device 2 includes a control unit 6. The control unit 6 is configured by a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an I / O (Input / Output). The control unit 6 executes processing corresponding to the computer program by executing the computer program stored in the non-transitional tangible recording medium, and controls the overall operation of the vehicle device 2.

  The control unit 6 includes, as functions related to the present invention, an image acquisition unit 6a, an edge image acquisition unit 6b, a feature point extraction unit 6c, a distance calculation unit 6d, a condition determination unit 6e, and a feature point exclusion unit 6f. And an optical flow calculation unit 6g and a notification command unit 6h. Each of these parts 6a-6h is comprised by the computer program which the control part 6 performs, and is implement | achieved by the software.

  The image acquisition unit 6 a performs signal processing on the image signal input from the camera 3 and acquires an image behind the vehicle captured by the camera 3. The edge image acquisition unit 6b detects a discontinuously changing portion in the image acquired by the image acquisition unit 6a and acquires an edge image. The feature point extraction unit 6c extracts a discontinuously changing portion in the edge image as a feature point. For example, a Harris corner method is used as a method for extracting feature points. The distance calculation unit 6d calculates the distance between feature points in the vertical direction (corresponding to the first predetermined direction). The condition determining unit 6e satisfies a first condition in which a pair having the same distance between feature points in the vertical direction is equal to or more than the first predetermined number in the vertical direction, and a group satisfying the first condition is horizontal. It is determined whether or not there is a feature point satisfying the second condition that is equal to or greater than the second predetermined number in the direction (corresponding to the second predetermined direction).

  The feature point excluding unit 6f satisfies both the first condition and the second condition when the condition determining unit 6e determines that there is a feature point that satisfies both the first condition and the second condition. Of the feature points, the feature points existing at the extreme ends of the top, bottom, left and right are determined as the remaining objects. Then, the feature point exclusion unit 6f determines other feature points excluding the determined remaining object as an exclusion object, and excludes the determined exclusion target feature point from the edge image.

  The optical flow calculation unit 6g compares the edge images from which the feature points to be excluded are excluded, calculates the optical flow (that is, the movement vector of the object), and whether there is a moving object such as a pedestrian or another vehicle, for example. Determine whether or not. As a method for calculating the optical flow, for example, a block matching method or a gradient method is used. When it is determined that a moving body is present as a result of calculation by the optical flow calculation unit 6g, the notification command unit 6h causes the notification device 5 to output a notification command signal and cause the notification device 5 to perform a notification operation. Inform the driver of the presence of the body.

  Next, the operation of the above configuration will be described with reference to FIGS. The control unit 6 performs a moving body detection process in connection with the present invention. When the moving body detection process is started, the control unit 6 inputs an image signal from the camera 3 (S1), acquires the image of the Nth frame by performing signal processing on the input image signal (S2, the image acquisition procedure). Equivalent to). The control unit 6 acquires an edge image from the acquired Nth frame image (S3, corresponding to the edge image acquisition procedure), and extracts feature points of the Nth frame edge image (S4, feature point extraction procedure). Equivalent to

  Next, the control unit 6 calculates the distance between the feature points in the vertical direction (S5, corresponding to the distance calculation procedure), and the pair in which the distance between the feature points in the vertical direction is the same as the others is the first in the vertical direction. It is determined whether or not there is a feature point satisfying the second condition that satisfies the first condition that is equal to or greater than the predetermined number and satisfies the first condition that satisfies the first condition in the horizontal direction. (S6 corresponds to a condition determination procedure). In this case, the controller 6 determines that the distance between the feature points in the vertical direction is within a predetermined range (that is, within the range from the lower limit value to the upper limit value), and the distance between the feature points in the vertical direction is the same as the other distance. Is identified as a pair. The control unit 6 may set the first predetermined number or the second predetermined number as a fixed value or a variable value. When the first predetermined number or the second predetermined number is a variable value, the control unit 6 may use, for example, a predetermined ratio with respect to the total number of extracted feature points as the first predetermined number or the second predetermined number.

  When the control unit 6 determines that there is a feature point that satisfies both the first condition and the second condition (S6: YES), among the feature points that satisfy both the first condition and the second condition, Feature points existing at the top, bottom, left, and right end portions are determined as remaining objects (S7). The control unit 6 determines other feature points excluding the determined remaining target as exclusion targets (S8), and excludes the determined exclusion target feature points from the Nth frame edge image (S9, feature point exclusion). Corresponding to the procedure).

  This will be specifically described with reference to FIG. Assuming the case where the rear part of the vehicle body faces the brick wall, the camera 3 captures the brick wall as shown in FIG. In this case, if the shape and color of the individual bricks constituting the brick wall are the same, the camera 3 captures the same continuous pattern. When the control unit 6 acquires an image from the image signal input from the camera 3 and acquires an edge image from the acquired image, the control unit 6 extracts the feature points of the acquired edge image as shown in FIG. . In this case, since the image has an isomorphic continuous pattern, the feature points are regularly arranged in a grid pattern. Therefore, the feature points satisfy a first condition in which pairs having the same distance (d1, d2, d3,..., Dn) between feature points in the vertical direction satisfy a first predetermined number or more in the vertical direction, and A group that satisfies the first condition satisfies a second condition that is equal to or greater than a second predetermined number in the horizontal direction. In FIG. 3B, the portions indicated by a1, a2,... Are a pair, and the portions indicated by b1, b2,. When the control unit 6 determines that substantially all feature points satisfy both the first condition and the second condition, the control unit 6 determines whether the first condition and the second condition are as shown in FIG. Of the feature points satisfying both, the feature points excluding the feature points existing at the top, bottom, left and right end portions (feature points indicated by m1 to m4 in FIG. 3) are excluded from the edge image.

  Next, the control unit 6 compares the position of the feature point of the edge image of the Nth frame with the position of the feature point of the edge image of the (N−1) th frame acquired by the moving body detection process before this. An optical flow is calculated (S10, corresponding to the optical flow calculation procedure), and it is determined whether or not a moving object is present (S11). That is, if the control unit 6 determines that there is a feature point that satisfies both the first condition and the second condition, the Nth frame edge image excluding the feature point to be excluded, and N− The edge image of the first frame is used. If the feature point to be excluded is also excluded from the edge image of the (N−1) th frame, the control unit 6 compares the edge images from which the feature points to be excluded are excluded. If the feature point to be excluded is not excluded from the edge image of the (N-1) th frame, the control unit 6 excludes the feature point to be excluded and the edge from which the feature point to be excluded is not excluded. Compare the image.

  On the other hand, when the control unit 6 determines that there is no feature point that satisfies both the first condition and the second condition (S6: NO), the Nth frame does not exclude the feature point to be excluded. And the edge image of the (N-1) th frame are used. If the feature point to be excluded is not excluded from the edge image of the (N-1) th frame, the control unit 6 compares the edge images that do not exclude the feature point to be excluded. If the feature point to be excluded is excluded from the edge image of the (N-1) th frame, the control unit 6 does not exclude the feature point to be excluded and the edge from which the feature point to be excluded is excluded Compare the image.

  If the control unit 6 determines that a moving body is present as a result of calculating the optical flow (S11: YES), the control unit 6 causes the notification device 5 to output a notification command signal and cause the notification device 5 to perform a notification operation ( S12), the moving body detection process is terminated. That is, as illustrated in FIG. 4, when the control unit 6 determines that, for example, the pedestrian P exists as a moving object as the optical flow calculation result, the control unit 6 causes the notification device 5 to perform a notification operation. By performing the above-described series of processing, the control unit 6 calculates the optical flow by excluding the feature points to be excluded from the edge image even when the background includes an isomorphic continuous pattern, thereby obtaining the background flow. The possibility of erroneous detection of some areas as moving objects is reduced. On the other hand, if the control unit 6 determines that there is no moving body as a result of calculating the optical flow (S11: NO), the control unit 6 ends the moving body detection process without causing the notification device 5 to output a notification command signal.

  The control unit 6 performs the above-described series of processes both when the vehicle is stopped and when it is moving, thereby reducing the possibility of erroneously detecting a partial region of the background as a moving object. Is possible. As shown in FIGS. 5 and 6, if the vehicle M is stopped with the rear part of the vehicle body facing the brick wall W and the distance between the vehicle M and the brick wall W remains “D1”, the camera is not changed. The image of the (N-1) th frame and the image of the Nth frame taken in step 3 are the same, and the feature points of the edge image of the (N-1) th frame and the feature points of the edge image of the Nth frame have the same arrangement. Become. Therefore, in the conventional method that does not exclude the feature points to be excluded, as shown in FIG. 5, the optical flow is calculated in a state in which there are regions including similar feature points in the previous and subsequent frames. There is a high possibility of erroneous detection of a region including a moving object. On the other hand, in the method of the present invention that excludes feature points to be excluded, as shown in FIG. 6, the optical flow is calculated in a state where there are no regions including similar feature points in the previous and subsequent frames. There is no possibility of false detection.

  Further, as shown in FIGS. 7 and 8, the vehicle M is moving the rear part of the vehicle body toward the brick wall W, and the distance between the vehicle M and the brick wall W changes shortly from “D1” to “D2”. If this is the case, the N-1th frame image and the Nth frame image captured by the camera 3 are different from the N-1th frame edge image feature point and the Nth frame edge image feature point. Are arranged differently, but the continuity of the feature points is maintained. Therefore, in the conventional method that does not exclude the feature points to be excluded, as shown in FIG. 7, the optical flow is calculated in a state in which there are regions including similar feature points in the previous and subsequent frames. There is a high possibility of erroneous detection of a region including a moving object. On the other hand, in the method of the present invention that excludes feature points to be excluded, as shown in FIG. 8, the optical flow is calculated in a state where there are no regions including similar feature points in the previous and subsequent frames. There is no possibility of false detection.

  In addition, the control unit 6 can detect the moving object as a background without erroneous detection even if the moving object includes the same shape continuous pattern by remaining the feature points to be remained. . As shown in FIGS. 9 and 10, for example, a case is assumed where a truck N in which a pattern of an isomorphous continuous pattern is provided on the side wall surface of the container passes behind the vehicle M. The photographing field of view of the camera 3 is indicated by broken lines A1 and A2. As shown in FIG. 9, in the method that does not leave the remaining feature points, all feature points that satisfy both the first condition and the second condition are excluded as exclusion targets. It cannot be detected. On the other hand, as shown in FIG. 10, in the method of remaining the feature points of the remaining target, a part remains without excluding all the feature points that satisfy both the first condition and the second condition as exclusion targets. Since it remains as an object, the track N can be detected as a moving object by detecting a change in the position of the feature points of the remaining object (feature points indicated by m1 to m4 in FIG. 10).

  Moreover, although the above demonstrated the brick wall as the background which comprises an isomorphous continuous pattern, the fence as shown in FIG. 11, the fence as shown in FIG. 12, the goods of the same shape as shown in FIG. 13 are displayed. The present invention can also be applied to a case where a product shelf or a building or the like to which a window having the same shape as shown in FIG. Also in this case, the control unit 6 excludes from the edge image feature points excluding the feature points existing at the top, bottom, left and right end portions from among the feature points satisfying both the first condition and the second condition, Calculate the flow.

As described above, according to the present embodiment, the following effects can be obtained.
In the vehicle periphery monitoring system 1, a group in which pairs having the same distance between feature points in the vertical direction are equal to or more than a first predetermined number in the vertical direction is set as the first condition, and a group that satisfies the first condition A condition that is equal to or greater than the second predetermined number in the horizontal direction was set as the second condition. Then, feature points that satisfy both the first condition and the second condition are excluded from the edge image as an exclusion target, and the optical flow is calculated using the edge image that excludes the feature point of the exclusion target. . Even if the background contains an isomorphic continuous pattern such as a brick wall and many similar feature points are extracted in the previous and next frames, the possibility of misdetecting a part of the background as a moving object is reduced. can do. Thereby, moving bodies, such as a pedestrian and another vehicle, can be detected appropriately, for example.

  Further, some of the feature points that satisfy both the first condition and the second condition are left as excluded objects in the edge image without being excluded from the edge image. Thereby, even when the moving body includes the same shape continuous pattern, the moving body can be appropriately detected without being erroneously detected as the background.

The present invention is not limited to those exemplified in the above-described embodiment, and can be arbitrarily modified or expanded without departing from the scope thereof.
The configuration in which the vertical direction is the first predetermined direction and the horizontal direction is the second predetermined direction is illustrated, but the horizontal direction may be the first predetermined direction and the vertical direction may be the second predetermined direction. That is, the distance between the feature points in the horizontal direction is calculated, a pair in which the distance between the feature points in the horizontal direction is the same as the other satisfies the first condition that the first predetermined number or more in the horizontal direction is satisfied, and It may be configured to determine whether or not there is a feature point that satisfies the second condition in which a group satisfying the condition of 1 satisfies a second predetermined number or more in the vertical direction.
Although the structure which determines as a residual object the feature point which exists in the extreme end of the top, bottom, left and right among the feature points satisfying both the first condition and the second condition is illustrated, it is different from the top, bottom, left and right end parts. The configuration may be such that the feature points existing in the region are determined as the remaining objects. In addition, the configuration that is determined as the remaining object may be omitted.
Although the structure which image | photographs the vehicle back was illustrated, the structure which detects the moving body of the vehicle front and vehicle side by image | photographing the vehicle front and the vehicle side may be sufficient.

  In the drawings, 1 is a vehicle periphery monitoring system, 3 is a camera (imaging unit), 5 is a notification device (notification unit), 6 is a control unit, 6a is an image acquisition unit, 6b is an edge image acquisition unit, and 6c is feature point extraction. 6d is a distance calculation unit, 6e is a condition determination unit, 6f is a feature point exclusion unit, 6g is an optical flow calculation unit, and 6h is a notification command unit.

Claims (7)

A photographing unit (3) for photographing the periphery of the vehicle;
An image acquisition unit (6a) for acquiring an image captured by the imaging unit;
An edge image acquisition unit (6b) for acquiring an edge image from the image;
A feature point extraction unit (6c) for extracting feature points of the edge image;
A distance calculation unit (6d) for calculating a distance between feature points in the first predetermined direction;
A group satisfying a first condition in which a pair having the same distance between feature points in the first predetermined direction is equal to or more than the first predetermined number in the first predetermined direction, and satisfying the first condition. A condition determination unit (6e) that determines whether or not there is a feature point that satisfies the second predetermined number that is equal to or greater than the second predetermined number in the second predetermined direction;
A feature point exclusion unit (6f) that excludes feature points that satisfy both the first condition and the second condition from the edge image as exclusion targets;
A vehicle periphery monitoring system comprising: an optical flow calculation unit (6g) that calculates an optical flow using the edge image from which the feature points to be excluded are excluded. In the vehicle periphery monitoring system according to claim 1,
The distance calculation unit calculates a distance between feature points in the vertical direction of the edge image as the first predetermined direction;
The condition determination unit includes a feature point that satisfies the second condition that the group satisfying the first condition is equal to or greater than the second predetermined number in the horizontal direction of the edge image as the second predetermined direction. A vehicle periphery monitoring system that determines whether or not. In the vehicle periphery monitoring system according to claim 1 or 2,
The feature point exclusion unit determines at least one feature point among remaining feature points satisfying both the first condition and the second condition as a remaining object, and other feature points excluding the determined remaining object Is determined as an exclusion target, and the feature point of the determined exclusion target is excluded from the edge image. In the vehicle periphery monitoring system according to claim 3,
The said feature point exclusion part is a vehicle periphery monitoring system which determines the feature point which exists in the extreme end of the top, bottom, left and right among the feature points satisfying both the first condition and the second condition as a remaining object. In the vehicle periphery monitoring system according to any one of claims 1 to 4,
The vehicle periphery monitoring system provided with the alerting | reporting instruction | indication part (6h) which alert | reports the calculation result by the said optical flow calculation part by an alerting | reporting part (5). In the vehicle periphery monitoring system according to any one of claims 1 to 5,
The said imaging | photography part is a vehicle periphery monitoring system which image | photographs the vehicle back. In the control unit (6) of the vehicle periphery monitoring system (1) having an image capturing unit (3) for capturing the periphery of the vehicle,
An image acquisition procedure for acquiring an image captured by the imaging unit;
An edge image acquisition procedure for acquiring an edge image from the image;
A feature point extraction procedure for extracting feature points of the edge image;
A distance calculation procedure for calculating a distance between feature points in a first predetermined direction;
A group satisfying a first condition in which a pair having the same distance between feature points in the first predetermined direction is equal to or more than the first predetermined number in the first predetermined direction, and satisfying the first condition. A condition determination procedure for determining whether or not there is a feature point that satisfies the second predetermined number that is equal to or greater than the second predetermined number in the second predetermined direction;
A feature point exclusion procedure for excluding feature points that satisfy both the first condition and the second condition from the edge image as exclusion targets;
A computer program for executing an optical flow calculation procedure for calculating an optical flow using the edge image from which the feature points to be excluded are excluded.

Images