JP2009003761A - Height difference recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a height difference recognition system capable of properly recognizing height differences in the periphery of a vehicle. <P>SOLUTION: In a vehicle peripheral image providing device 100 applying the height difference recognition system, a control part 102 calculates a motion vector ratio being a ratio of the length of a nearby feature point motion vector showing a motion per unit time of a feature point within a nearby range of a vehicle 200 in an image to the length of a remote feature point motion vector showing a motion per unit time of a feature point within a remote range of the vehicle 200 in the image. Furthermore, the control part 102 compares this motion vector ratio with a motion vector ratio obtained on the assumption that a height difference does not exist between the nearby feature point and the remote feature point, to determine whether a height difference exists between the nearby feature point and the remote feature point or not. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a height difference recognition system that recognizes a height difference around a vehicle.

In recent years, a system for monitoring the periphery of a vehicle has been proposed in order to ensure the safety of the vehicle. For example, in the vehicle surrounding monitoring device described in Patent Document 1, the periphery of the vehicle is photographed by a camera mounted on the vehicle, and the boundary of a three-dimensional obstacle such as a height difference is recognized based on the color of the obtained image. To do. When the boundary of the three-dimensional obstacle is within a predetermined distance from the vehicle, the possibility of contact is warned.
JP 2004-302609 A

  However, in the vehicle surrounding monitoring device described in Patent Document 1, since a height difference or the like is recognized based on only the color, when there is a portion having a color different from other portions on the road surface, If there is a difference in height between the different parts of the color, it may happen that it is mistakenly recognized.

  An object of the present invention is to solve the above-described problems, and to provide a height difference recognition system that can appropriately recognize a height difference around a vehicle.

  According to the present invention, a height difference recognition system for recognizing a height difference around a vehicle corresponds to the image data generated by the image capturing means for capturing the image of the periphery of the vehicle and generating image data. Feature point extraction for extracting a first feature point within a first range near the vehicle and a second feature point within a second range farther than the first range based on the image Means, a first motion vector indicating the movement of the first feature point extracted by the feature point extraction means, and the movement of the second feature point accompanying the movement of the vehicle. Presence or absence of the height difference based on a difference between the first motion vector and the second motion vector acquired by the motion vector acquisition means; Height difference judgment means for judging Characterized in that it has a.

  According to this configuration, the difference between the first motion vector indicating the motion of the first feature point near the vehicle in the image and the second motion vector indicating the motion of the second feature point far from the vehicle is Based on the difference between the case where there is a height difference between the one feature point and the second feature point and the case where there is no height difference, the height is determined based on the difference between the first motion vector and the second motion vector. By determining the presence / absence of a difference, it is possible to prevent erroneous recognition as in the case of recognizing a height difference based on only a conventional color, and to recognize the height difference appropriately.

  Further, the elevation difference recognition system according to the present invention is configured such that the first difference when the elevation difference is assumed not to exist based on a distance between the first feature point and the second feature point in the image. Reference value acquisition means for acquiring a reference value indicating a difference between the motion vector of the feature point and the motion vector of the second feature point, and the height difference determination means includes the first motion vector and the first motion vector. When the difference between the two motion vectors is different from the reference value acquired by the reference value acquisition means, it is determined that the height difference exists between the first feature point and the second feature point. You may do it.

  According to this configuration, the difference between the first motion vector and the second motion vector when it is assumed that there is no height difference is obtained as a reference value, and the reference value and the actual first motion vector By comparing the difference with the second motion vector and determining whether or not there is a height difference, the height difference can be appropriately recognized. The reference value may be a single value or a predetermined range.

  From the same point of view, in the elevation difference recognition system according to the present invention, the difference between the motion vector of the first feature point and the motion vector of the second feature point is the length of the motion vector of the second feature point. It may be a ratio of the motion vector length of the first feature point to.

  From the same point of view, in the elevation difference recognition system according to the present invention, the elevation difference determination unit is configured such that the difference between the first motion vector and the second motion vector is acquired by the reference value acquisition unit. If the difference is larger than the above, it may be determined that there is a height difference that decreases from the vicinity of the vehicle toward the far side between the first feature point and the second feature point.

  Further, from the same viewpoint, the elevation difference recognition system according to the present invention is characterized in that the elevation difference determination unit is configured to acquire the difference between the first motion vector and the second motion vector by the reference value acquisition unit. When the value is smaller than the value, it may be determined that there is a height difference that increases from the vicinity of the vehicle toward the far side between the first feature point and the second feature point.

  Further, the elevation difference recognition system according to the present invention may include an alarm unit that issues an alarm when the elevation difference determination unit determines that the elevation difference exists.

  According to this configuration, when there is a height difference around the vehicle, it is possible to call attention to the passenger of the vehicle.

  According to the present invention, it is possible to determine whether there is a height difference based on a difference between a motion vector indicating the motion of a feature point near the vehicle in the image and a motion vector indicating the motion of a feature point far from the vehicle. It becomes possible to recognize the difference appropriately.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. FIG. 1 is a diagram showing a configuration of a vehicle periphery image providing apparatus to which an elevation difference recognition system according to an embodiment of the present invention is applied.

  1 is mounted on a vehicle, and includes a camera 152, 154, 156, 158 and 160, a control unit 102, a memory 104, a display 106, and a speaker 108. Composed.

  Cameras 152 to 160 are attached to the outside of the vehicle 200 and shoot around the vehicle to generate image data. FIG. 2 is a diagram illustrating attachment positions of the cameras 152 to 160 in the vehicle. In FIG. 2, the camera 152 is attached to the left side of the vehicle 200, the camera 154 is attached to the right side, the camera 156 is attached to the front left side, the camera 158 is attached to the front right side, and the camera 160 is attached to the rear center. The cameras 152 to 160 have a wide-angle or fish-eye lens (not shown) and have a wide angle of view, and the entire periphery of the vehicle 200 can be photographed by these cameras 152 to 160.

  Again, returning to FIG. The control unit 102 controls the entire vehicle surrounding image providing apparatus 100. Specifically, when the image data from the cameras 152 to 160 is input, the control unit 102 generates, for example, a bird's-eye view image from above the vehicle by the image conversion process, and displays the image on the display 106. By viewing the image displayed on the display 106, the driver can recognize the presence of an obstacle around the vehicle, and can compensate for the blind spot from the driver's seat. However, it is difficult to recognize the height difference around the vehicle only from an overhead image from above the vehicle. Therefore, the control unit 102 performs processing for recognizing the height difference around the vehicle. Details of height difference recognition will be described below.

  FIG. 3 is a diagram for explaining the principle of height difference recognition in the present embodiment. 3A shows a case where there is no difference in height between the feature point 301 near the camera 152 and the distant feature point 401. FIG. 3B shows the relationship between the near feature point 301 and the distant feature point 402. FIG. 3C, there is an elevation difference that increases from the vicinity of the camera 152 toward the far side between the near feature point 301 and the far feature point 403 in FIG. Indicates the case where it exists. The distant feature point 401 in FIG. 3A, the distant feature point 402 in FIG. 3B, and the distant feature point 403 in FIG. Exists. Accordingly, the distant feature points 401, 402, and 403 are present at the same position in the coordinate system of an image obtained by photographing with the camera 152.

  In the height difference recognition according to the present embodiment, even when the difference in the position of the feature point cannot be recognized from the image as described above, the difference in the position is recognized, and further, the difference in height due to the difference in the position is recognized. Recognize the existence of

  Specifically, as illustrated in FIG. 4, the control unit 102 extracts a feature point 301 in the vicinity of the camera 152, that is, in the vicinity of the vehicle 200, from an image obtained by photographing with the camera 152 at time 0. A feature point far from the vehicle (a feature point 401 in FIG. 3A, a feature point 402 in FIG. 3B, and a feature point 403 in FIG. 3C) is extracted. Furthermore, the control unit 102 extracts each of the near feature point 301, the distant feature point 401, and the like from the image at time t after the elapse of the unit time t accompanying the movement of the vehicle 200, and as shown in FIG. The motion vector 311 indicating the motion per unit time t of the neighboring feature point 301 and the motion vectors 411, 412 and 413 indicating the motion per unit time t of the distant feature points 401, 402 and 403 are acquired.

  Here, the distant feature point 403 shown in FIG. 3C at the closest position from the vehicle 200, the distant feature point 402 shown in FIG. 3B at the farthest position, and the distant feature point shown in FIG. 3A at the middle position. Since the feature point 401 exists, the distance per unit time t in the image has the largest distance feature point 403, the smallest distance feature point 402, and the distance feature point 401 becomes an intermediate movement amount. Therefore, as shown in FIG. 5, the motion vector 413 of the distant feature point 403 is the longest, the motion vector 412 of the distant feature point 402 is the shortest, and the motion vector 411 of the distant feature point 401 is intermediate. It becomes length.

  As described above, in the image at time 0, even if the distant feature points 401, 402, and 403 exist at the same position, the lengths of these motion vectors differ depending on whether there is a difference in height, and further, the distant feature Since the length of the motion vector such as the point 401 depends on the speed of the vehicle 200, it is not possible to determine whether there is a height difference only by the length of the motion vector such as the far feature point 401. In view of the fact that it is necessary to consider the relative relationship between the motion vector of the neighboring feature point 301 and the motion vector of the neighboring feature point 301, the control unit 102 determines the motion vector of the neighboring feature point 301, Recognize the existence of elevation differences based on the difference between the two.

  6 and 7 are flowcharts showing the operation of height difference recognition. In the following, it is assumed that the vehicle 200 is traveling. In the following, a case where the height difference is recognized using an image obtained by photographing with the camera 152 will be described. However, the height difference is recognized using images obtained by photographing with the cameras 154 to 160. The same applies to the case.

  The control unit 102 instructs the camera 152 to perform shooting. In response to this instruction, the camera 152 captures the periphery of the vehicle 200 and generates image data (S101). The generated image data is output to the control unit 102. Note that the process of S101 is continued while the processes after S102 are being performed.

  The control unit 102 forms an image corresponding to the input image data, and divides the image into an image corresponding to a range near the vehicle 200 and an image corresponding to a range far from the vehicle 200. For example, the control unit 102 sets the lower half of the image as an image corresponding to the near range, and sets the upper half as an image corresponding to the far range. Next, the control unit 102, based on the pixel value indicating the color of the pixels constituting the image, the feature point in the image corresponding to the neighborhood range (neighboring feature point) and the feature point in the image corresponding to the far range (distant location) (Feature point) are extracted (S102). The positions of the near feature points and the far feature points are specified by the coordinates in the image coordinate system.

  Next, the control unit 102 acquires the distance between the near feature point and the far feature point in the image (S103). The distance is represented by, for example, the number of pixels between the near feature point and the far feature point.

  Next, the control unit 102 determines the movement of the distant feature points on the image per unit time t accompanying the movement of the vehicle 200 when it is assumed that there is no height difference between the near feature points and the distant feature points. A predetermined range centered on the ratio of the length of a motion vector indicating the motion of a neighboring feature point to the length of the indicated motion vector (motion vector ratio) is set as a reference range for the motion vector ratio (S104).

  Specifically, the memory 104 stores a table as shown in FIG. The table shown in FIG. 8 shows the correspondence between the distance between the feature points in the image and the feature points in the distance (feature point distance) and the reference range of the motion vector ratio. The control unit 102 refers to this table. Thus, the reference range of the motion vector ratio corresponding to the feature point distance acquired in S103 is acquired and set. Here, the reference range of the motion vector ratio is determined according to the feature point distance. As the feature point distance is longer, it is assumed that there is no difference in height between neighboring feature points and distant feature points. In this case, the motion vector ratio becomes large, that is, the motion vector ratio takes into consideration that it depends on the feature point distance.

  Next, the control unit 102 determines whether or not the unit time t has elapsed since the extraction of the nearby feature points and the distant feature points in S102 (S105). When the unit time t has elapsed, the process proceeds to the operation shown in FIG. 7, an image corresponding to the latest image data input from the camera 152 is formed, and the near feature point and the far feature point extracted in S102 Are extracted from the image. As in the case of S102, the positions of these near feature points and far feature points are specified by coordinates in the coordinate system of the image. Further, the control unit 102 determines the interval between the unit time t based on the positions of the extracted nearby feature points and the distant feature points on the image and the positions of the nearby feature points and the distant feature points extracted in S102. A nearby feature point motion vector indicating the motion of the nearby feature point on the image and a distant feature point motion vector indicating the motion of the distant feature point are acquired (S106).

  Next, the control unit 102, based on the acquired nearby feature point motion vector and the far feature point motion vector indicating the motion of the far feature point, A motion vector ratio as a ratio is calculated (S107). Further, the control unit 102 determines whether or not the motion vector ratio calculated in S107 is within the reference range set in S104 (S108).

  When the motion vector ratio calculated in S107 is within the reference range set in S104, the motion vector ratio is a motion when it is assumed that there is no height difference between the nearby feature point and the far feature point. This means that the value is close to the vector ratio, and the near feature point and the far feature point are in a positional relationship as shown in FIG. For this reason, when the motion vector ratio is within the reference range, the control unit 102 recognizes that there is no height difference between the nearby feature point and the distant feature point (S109). Thereafter, the operations after the extraction of the neighboring feature points and the far feature points (S102) in FIG. 6 are repeated again.

  On the other hand, if the motion vector ratio calculated in S107 is outside the reference range set in S104, the control unit 102 further determines whether the motion vector ratio calculated in S107 is larger than the reference range set in S104. It is determined whether or not (S110).

  When the motion vector ratio calculated in S107 is larger than the reference range set in S104, the neighboring feature points and the distant feature points are in a positional relationship as shown in FIG. For this reason, when the motion vector ratio is larger than the reference range, the control unit 102 recognizes that there is a height difference that decreases from the vicinity to the distant feature point between the nearby feature point and the distant feature point (S111). On the other hand, when the motion vector ratio calculated in S107 is smaller than the reference range set in S104, the neighboring feature points and the distant feature points are in a positional relationship as shown in FIG. For this reason, when the motion vector ratio is larger than the reference range, the control unit 102 recognizes that there is a height difference that increases from the vicinity toward the distance between the nearby feature point and the distant feature point (S112).

  When the height difference is recognized in S111 or S112, the control unit 102 causes the display 106 to display a warning screen indicating that there is a height difference and to sound a warning sound from the speaker 108 (S113). Thereafter, the operations after the extraction of the neighboring feature points and the far feature points (S102) in FIG. 6 are repeated again.

  As described above, in the present embodiment, the distance of the feature point in the near range of the vehicle 200 to the length of the feature point motion vector indicating the motion of the feature point in the far range of the vehicle 200 in the image per time t. When the motion vector ratio, which is the ratio of the lengths of the motion vector lengths of neighboring feature points indicating the motion of the motion vector, is calculated and it is assumed that there is no height difference between the motion vector ratio and the neighboring feature points and the far feature points. By comparing the motion vector ratio and determining whether there is a height difference between the nearby feature point and the far feature point, it is possible to appropriately recognize the height difference.

  In the above-described embodiment, the ratio of the length of the neighborhood feature point motion vector to the length of the distance feature point motion vector is calculated as the difference between the neighborhood feature point motion vector and the far feature point motion vector. A vector difference between the motion vector and the distant feature point motion vector may be calculated, and the presence / absence of a height difference may be determined based on the vector difference.

  In this case, in place of the table shown in FIG. 8, the memory 104 does not have a difference in height between the feature point distance and the neighboring feature point and the far feature point, and the vehicle 200 is a predetermined reference. Associating a predetermined range (reference range) centered on the vector difference between the motion vector of a distant feature point on the image per unit time t and the motion vector of a nearby feature point per unit time t, assuming that it has moved at a speed Remember the table. Then, after the processing of S101 to S103 in FIG. 6, instead of the processing in S104, the control unit 102 acquires a reference range corresponding to the calculated feature point distance with reference to the table stored in the memory 104. Set.

  Further, after the processing of S105 and S106, instead of the processing of S107 to S113, the control unit 102 acquires the speed of the vehicle 200 and calculates the vector difference between the nearby feature point motion vector and the far feature point motion vector. To do. Then, the control unit 102 divides the reference speed described above by the speed of the acquired vehicle 200 and multiplies the divided value by the calculated vector difference. Further, the control unit 102 determines whether or not the vector difference obtained by this calculation is within the reference range, and if it is within the reference range, there is a height difference between the neighboring feature points and the distant feature points. Recognize not. In addition, when the vector difference obtained by the calculation is larger than the reference range, the control unit 102 recognizes that there is a height difference that decreases from the neighborhood to the distance between the neighboring feature point and the far feature point. If the vector difference obtained by the above is smaller than the reference range, the display 106 recognizes that there is a height difference that increases from the neighborhood to the distance between the nearby feature point and the far feature point, and the display 106 has a height difference. An alarm screen to that effect is displayed and an alarm sound is emitted from the speaker 108.

  As described above, the height difference recognition system according to the present invention can appropriately recognize the height difference around the vehicle, and is useful as a height difference recognition system.

It is a figure which shows the structure of the vehicle periphery image provision apparatus to which the elevation difference recognition system which concerns on embodiment of this invention is applied. It is a figure which shows the attachment position of the camera in a vehicle. It is a figure explaining the principle of elevation difference recognition. It is a figure which shows an example of the near feature point and distant feature point in an image. It is a figure which shows an example of the near feature point in a picture, a distant feature point, and a motion vector. It is a 1st flowchart which shows the operation | movement of elevation difference recognition. It is a 2nd flowchart which shows the operation | movement of elevation difference recognition. It is a figure which shows an example of the table which matched feature point distance and the motion vector ratio reference | standard range.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Vehicle periphery image provision apparatus 102 Control part 104 Memory 106 Display 108 Speaker 152,154,156,158,160 Camera 200 Vehicle

Claims (6)

A height difference recognition system for recognizing a height difference around a vehicle,
Photographing means for photographing the periphery of the vehicle and generating image data;
Based on an image corresponding to the image data generated by the photographing means, a first feature point in the first range near the vehicle and a second range farther than the first range Feature point extracting means for extracting the second feature point of
A first motion vector indicating the movement of the first feature point extracted by the feature point extracting means with the movement of the vehicle, and a second motion vector indicating the movement of the second feature point with the movement of the vehicle. Motion vector acquisition means for acquiring a motion vector of
A height difference judging means for judging the presence or absence of the height difference based on a difference between the first motion vector and the second motion vector obtained by the motion vector obtaining means. Difference recognition system. Based on the distance between the first feature point and the second feature point in the image, the motion vector of the first feature point and the second feature when the height difference is assumed not to exist. Reference value acquisition means for acquiring a reference value indicating a difference from the motion vector of the point,
The height difference determining means is configured to determine whether the difference between the first motion vector and the second motion vector is different from the reference value acquired by the reference value acquiring means. 2. The height difference recognition system according to claim 1, wherein it is determined that the height difference exists between two feature points.   The difference between the motion vector of the first feature point and the motion vector of the second feature point is the length of the motion vector of the first feature point with respect to the length of the motion vector of the second feature point. The height difference recognition system according to claim 2, wherein the difference is a ratio.   When the difference between the first motion vector and the second motion vector is greater than the reference value acquired by the reference value acquisition unit, the height difference determination unit is configured to detect the first feature point and the height difference determination unit. 4. The height difference recognition system according to claim 2, wherein it is determined that there is a height difference that decreases from the vicinity of the vehicle toward the far side with the second feature point.   When the difference between the first motion vector and the second motion vector is smaller than the reference value acquired by the reference value acquisition unit, the height difference determination unit is 5. The height difference recognition system according to claim 2, wherein it is determined that there is a height difference that increases from the vicinity of the vehicle toward the far side between the second feature point and the second feature point.   6. The height difference recognition system according to claim 1, further comprising warning means for issuing a warning when the height difference determining means determines that the height difference exists.

Images