JP4828333B2 - Vehicle perimeter monitoring system - Google Patents

Description

The present invention relates to a vehicle periphery monitoring system .

There has been proposed a technique for determining the presence or absence of an object such as a human who may come into contact with a vehicle based on a surrounding image of the vehicle captured by an in-vehicle infrared camera (see, for example, Patent Document 1).
JP 2001-6096 A

However, if the person and the building overlap, the human (indicated by a broken line) x is mixed into the background building as shown in FIG. 4 in the grayscale image of the surrounding image of the vehicle. In other words, it may be difficult to recognize this human existence.

Therefore, the present invention provides a vehicle periphery monitoring system capable of recognizing the presence of a target object with high accuracy even when the target object such as a human is mixed in the background in an image obtained by an imaging device. And

A vehicle periphery monitoring system according to a first aspect of the present invention is a system that monitors the periphery of a vehicle based on an image obtained by an imaging device mounted on the vehicle, and the luminance locally peaks in the image. was recognized as a peak point, and a line segment composed of a peak point group with continuity, a first processing section recognizes a side-by-side pair of longitudinal of the line segment as a target partial candidate of the target object Measuring the horizontal interval and vertical size in real space of each of the pair of target portion candidates recognized by the first processing unit, and measuring the measured horizontal interval and vertical size, The spacing and the ratio of the sizes, or the spacing and the size and the ratio are the horizontal spacing of both legs as a plurality of target portions of a human being as a target object and Whether the pair of target part candidates falls under both human legs depending on whether the size in the straight direction, the interval and the ratio of the sizes, or the requirements according to the interval, the size, and the ratio are met And a second processing unit for determining whether or not.

According to the vehicle periphery monitoring system of the first invention, the positions of a plurality of target portions in the target object from among “target portion candidates” that may correspond to the target portion such as an arm or a leg of a target object such as a human being. In view of the relationship and the like, a thing that is highly likely to correspond to the target part is recognized as the target part. As a result, even when a target object such as a human is included in the background in the image, the plurality of target portions of the target object, and hence the presence of the target object, can be recognized with high accuracy.
Further, the inventor of the present application has found that both human legs can be recognized with high accuracy even in the case where a human is mixed into a background building or the like in an image obtained by the imaging apparatus. Therefore, according to the vehicle periphery monitoring system of the first aspect of the present invention, the horizontal interval between the human legs among the target part candidates that may correspond to both legs as the plurality of target parts of the target object that the person has. In view of the above, a part that is highly likely to fall into both human legs is determined as a target part. As a result of this determination, even if a person is mixed into the background in the image, both the legs of the person, and thus the presence of the person, can be recognized with high accuracy.
“Point” means an arbitrary unit element constituting an image such as a sub-pixel in addition to a pixel.
The vehicle periphery monitoring system according to a second aspect of the present invention is the vehicle periphery monitoring system according to the first aspect, wherein the first processing unit sets the line segment as a primary target portion candidate, and among the plurality of primary target portion candidates, Recognizing a plurality of primary target part candidates satisfying requirements according to one or both of the shape and arrangement of the target parts of the target object in the image as secondary target part candidates corresponding to the target part candidates It is characterized by doing.

According to the vehicle periphery monitoring system of the second aspect of the present invention, the arrangement in the image obtained by the imaging device among the plurality of primary target part candidates depends on one or both of the shape and the arrangement of the target part in the target object. Only a plurality of primary target part candidates that satisfy the above requirements can be recognized as secondary target part candidates. As a result, it is not necessary to execute processing for determining whether or not the target part candidate that is unlikely to correspond to the target part of the target object in view of the shape and arrangement is applicable to the target part. The economy can be planned.

The vehicle periphery monitoring system according to a third aspect of the invention is the vehicle periphery monitoring system according to the first aspect of the invention, based on the time-series measurement positions of the target part candidates recognized by the second processing unit as corresponding to the target part. A third processing unit for evaluating the possibility of contact between the vehicle and the target object having the target portion is further provided.

According to the vehicle periphery monitoring system of the third aspect of the present invention , even when the target object is mixed in the background in the image obtained by the imaging device as described above, the presence of the target object or the plurality of target portions is recognized with high accuracy. Therefore, the possibility that the vehicle and the target object come into contact with each other can be evaluated with high accuracy.

The vehicle periphery monitoring system according to a fourth aspect is the vehicle periphery monitoring system according to the third aspect , wherein the third processing unit is configured to evaluate the target object or the target based on the evaluation of the possibility of contact between the vehicle and the target object. Information that informs or emphasizes the presence of a part is output to an information output device mounted on the vehicle.

According to the vehicle periphery monitoring system of the fourth aspect of the present invention, the possibility of contact between the vehicle and the target object can be evaluated with high accuracy as described above. From the viewpoint of avoiding contact between the vehicle and the target object, the target object Or the information which emphasizes the some object part etc. can be output appropriately. Here, the output of information means that the information is output in any form that allows the driver of the vehicle to recognize the information through the five senses, such as vision, hearing, and touch. This allows the driver of the vehicle to recognize the presence of the target object and appropriately control the behavior of the vehicle such as vehicle deceleration and steering in order to avoid contact between the vehicle and the target object.

The vehicle periphery monitoring system according to a fifth aspect is the vehicle periphery monitoring system according to the third aspect , wherein the third processing unit is configured to evaluate the target object or the target based on the evaluation of the possibility of contact between the vehicle and the target object. Information that informs or emphasizes the presence of a part is output to an information output device mounted on the vehicle.

According to the vehicle periphery monitoring system of the fifth aspect of the invention , the in-vehicle device is controlled based on the possibility of contact between the vehicle and the target object evaluated with high accuracy as described above. Thereby, from the viewpoint of avoiding contact between the vehicle and the target object, the behavior of the vehicle such as deceleration or steering of the vehicle can be appropriately controlled.

An embodiment of a vehicle periphery monitoring system of the present invention will be described with reference to the drawings.

The configuration of the vehicle periphery monitoring system of the present invention will be described with reference to FIG. As shown in FIG. 1, the vehicle 1 includes a vehicle periphery monitoring system 10, a pair of infrared cameras (imaging devices) 102 disposed substantially symmetrically at the center in the vehicle width direction at the front portion of the vehicle 1, A HUD 104 is mounted on the front window so as not to obstruct the driver's view. As shown in FIG. 2, various sensors such as a yaw rate sensor 122 and a speed sensor 124 are mounted on the vehicle 1.

The vehicle periphery monitoring system 10 is a system that monitors the periphery of a vehicle based on an image obtained by a pair of in-vehicle imaging devices. The vehicle periphery monitoring system 10 is stored in a memory and an ECU or a computer (configured by a CPU, ROM, RAM, I / O, etc.) as hardware mounted on the vehicle 1, and has various functions in the computer. And a vehicle periphery monitoring program of the present invention as software for providing The vehicle periphery monitoring program may be stored in the memory (ROM) of the in-vehicle computer from the beginning, but a part or all of the program is stored in the server (at any timing such as when a request from the in-vehicle computer is received). It constitutes a construction system of a vehicle periphery monitoring system.) It may be distributed or broadcast from 20 to the in-vehicle computer via a network or satellite and stored in its memory (RAM) or the like.

As shown in FIG. 2, the vehicle periphery monitoring system 10 includes a first processing unit 11, a second processing unit 12, and a third processing unit 13. The first processing unit 11 recognizes a point where the luminance locally peaks in the image obtained by the infrared camera 102 as a “peak point”, and a line segment composed of a continuous peak point group is “ Recognize as “target part candidate”. The second processing unit 12 measures the position and size in real space of the target portion candidate recognized by the first processing unit 11. In addition, the second processing unit 12 determines whether the arrangement of the plurality of target portion candidates and the size of the plurality of target portion candidates satisfy the requirements according to the arrangement and size of the plurality of target portions in the target object. It is determined whether a plurality of target part candidates correspond to the plurality of target parts. The third processing unit 13 evaluates the possibility of contact between the vehicle 1 and the target object having the target part based on the time-series measurement position of the target part candidate recognized as the target part by the second processing unit 12. To do. In addition, the third processing unit 13 displays, on the HUD (information output device) 104, an image that informs or emphasizes the presence of the target object or a plurality of target parts in accordance with the evaluation of the possibility of contact between the vehicle 1 and the target object. Control the operation of the in-vehicle device such as displaying it.

Functions of the vehicle 1 and the vehicle periphery monitoring system 10 configured as described above will be described with reference to FIGS.

First, the first processing unit 11 executes “first processing” (S11 in FIG. 3).

The first processing unit 11 recognizes a point where the luminance locally peaks in a grayscale image obtained by A / D conversion of the infrared image captured by the pair of infrared cameras 102 as a “peak point” (FIG. 3 / S111). For example, the gray scale image shown in FIG. 4 is obtained, and the luminance is measured along the scanning line in a region surrounded by a frame in the image. In this case, for example, as shown in FIG. 5, a point where the luminance is peaked (circled) along the scanning lines of heights Y ₁ , Y ₂ , Y ₃ in this image area is the peak. Recognized as a point. In addition, when the deviation of the brightness | luminance of the continuous point along a scanning line is below a very small threshold value, the said continuous point can be recognized as a peak point.

In addition, the first processing unit 11 recognizes each line segment constituted by the continuous peak point group as a “primary target portion candidate” (FIG. 3 / S112). For example, when a predetermined number or more of peak points whose distance is less than or equal to a threshold value are connected, the connected peak points of the predetermined number or more are recognized as a continuous peak point group, and as a primary target portion candidate. Thereby, in the image region shown in FIG. 5, for example, a plurality of primary target portion candidates c ₁ as shown in FIG. 6 are recognized. Also, the image is averaged, peak points are extracted from the averaged image, the extracted peak points are subjected to 8 neighborhood processing, and the peak points subjected to 8 neighborhood processing are connected by a predetermined number or more. In this case, the connected peak points may be extracted as a peak point group (line segment).

Further, the first processing unit 11 selects a pair of primary target part candidates (or a part of the primary target part candidates) satisfying the following (Requirement 11) and (Requirement 12) among the primary candidates of the target object as “2”. It is recognized as “next target portion candidate” (FIG. 3 / S113).
(Requirement 11)
The shape of the pair of primary target portion candidates in the image is both vertically long (for example, the ratio of the length in the vertical direction and the length in the horizontal direction of the circumscribed rectangle in each image of the pair of primary target portion candidates is predetermined. Be in range)
(Requirement 12)
A pair of primary target part candidates are located side by side (for example, the vertical extent of the pair of primary target part candidates in the image overlaps a predetermined point or more)
(Requirement 11) and (Requirement 12) are set according to the shape and arrangement of both legs as a plurality of target portions possessed by a person as a target object. That is, (Requirement 11) is set according to the fact that the human leg is vertically long. Further, (Requirement 12) is set in response to a normal human being having two legs side by side. Thereby, for example, a pair of primary target portion candidates c ₁ surrounded by a broken line in FIG. 6 are recognized as secondary target portion candidates (hereinafter, referred to as “target portion candidates” as appropriate) c ₂ . Note that a pair of primary target portion candidates that satisfy the following (Requirement 13) may be recognized as secondary target portion candidates instead of (Requirement 11) and (Requirement 12) or in addition.
(Requirement 13)
The fact that the Y coordinate of the lower end of the pair of primary target part candidates is below the threshold from the ground surface (Requirement 13) is generally set according to whether both human legs are in contact with or near the ground. It is.

Next, the second processing unit 12 executes “second processing” (S12 in FIG. 3).

The second processing unit 12 uses the parallax of the pair of infrared cameras 102 to measure the position of the target portion candidate recognized by the first processing unit 11 in the real space (FIG. 3 / S121). At this time, the second processing unit 12 corrects the positional deviation in the image due to the turning of the vehicle 1 based on the outputs of the yaw rate sensor 122 and the speed sensor 124. The measurement position of the target portion candidate subjected to the turning angle correction is stored in the memory in time series. As shown in FIG. 1, the real space position is the origin O at the midpoint of the mounting position of the pair of infrared cameras 102, and the horizontal, vertical and front-rear directions are the X, Y and Z axes, respectively. Means a position in the coordinate system. Since the method for measuring the real space position and the method for correcting the turning angle are described in, for example, Patent Document 1, detailed description thereof is omitted in this specification.

Further, the second processing unit 12 measures the size of the target portion candidate in the real space (FIG. 3 / S122). For example, an interval between two real space positions corresponding to the upper end point and lower end point of the target portion candidate in the image is calculated, and this interval is measured as the size of the secondary candidate of the target object.

Further, the second processing unit 12 determines whether the correlation between the positions of the plurality of target portion candidates and the sizes of the plurality of target portion candidates satisfy the following (Requirement 21) and (Requirement 22). It is determined whether or not the plurality of target portion candidates correspond to both human legs (a plurality of target portions of the target object) (S123 in FIG. 3).
(Requirement 21)
The horizontal interval between the pair of target portion candidates is within a predetermined range (for example, the horizontal interval between the circumscribed rectangles of the pair of target portion candidates is 0.1 to 0.5 [m]).
(Requirement 22)
Each size of the target portion candidate is within a predetermined range (for example, the vertical size of the circumscribed rectangle of the pair of target portion candidates is 0.5 to 1.0 [m])
If the second processing unit 12 determines that (requirement 21) and (requirement 22) are satisfied (FIG. 3 / S123... YES), the second processing unit 12 recognizes the pair of target part candidates as a plurality of target parts ( FIG. 3 / S124). Thereby, for example, the target portion candidate c ₂ shown in FIG. 6 is recognized as both human legs. Requirement 21 and requirement 22 are set according to the arrangement and size of both legs as a plurality of target portions possessed by a person as a target object. In addition, instead of (Requirement 21) and (Requirement 22), whether or not the plurality of target portion candidates correspond to the target object is determined according to whether or not the following (Requirement 23) is satisfied. May be.
(Requirement 23)
The ratio between the horizontal interval between the pair of target portion candidates and the average value of one or both sizes of the pair of target portion candidates is within a predetermined range (for example, the circumscribed rectangle of the pair of target portion candidates The ratio between the horizontal interval and the average value of the size of one or both of the circumscribed rectangles of the pair of control portion candidates is in the range of 0.2 to 0.5)

Subsequently, the third processing unit 13 executes “third processing” (S13 in FIG. 3).

The third processing unit 13 reads the time-series measurement position of the target part candidate recognized as the target part by the second processing unit 12 from the memory, and uses the vehicle 1 as a reference based on the read time-series measurement position. The relative speed (including the size and direction) of the target object (or a plurality of target parts thereof) is calculated (FIG. 3 / S131).

Further, the third processing unit 13 determines (or evaluates) the possibility of contact between the vehicle 1 and the target object (FIG. 3 / S132). For example, as shown in FIG. 4, the product of the relative speed v _s of the object x such as a two-wheeled vehicle with respect to the vehicle 1 and the margin time T (v _s × T) rather than the triangular area A _{0 that} can be monitored by the infrared camera 102. ) Is defined as a lower triangular area (alarm determination area). Among the triangular regions, a first region (approaching determination region) A ₁ having a width in the X direction centered on the Z axis of α + 2β (α: vehicle width, β: margin width) and the first region A ₁ The left and right second areas (intrusion determination areas) A _2L and A _2R are defined. When the object x is in the first area A ₁ or when the object x is in the second area A ₂ on the left and right, it may enter the first area A ₁ in view of the relative velocity vector v _s. When predicted, it is determined that there is a high possibility that the object x and the vehicle 1 are in contact with each other.

If the third processing unit 13 determines that there is a high possibility that the vehicle 1 and the target object are in contact with each other (FIG. 3 / S132... YES), this target object is the human x (or a plurality of target parts thereof). In order to emphasize the presence of both legs), as shown in FIG. 8, the frame f surrounding the person x is displayed on the HUD 104 as “first information” with its color orange (FIG. 3 / S133). Note that the first information may be output from a speaker (not shown) such as “beep”.

On the other hand, when the third processing unit 13 determines that the possibility that the vehicle 1 and the target object are in contact with each other is low (FIG. 3 / S132... NO), the third processing unit 13 conservatively emphasizes the presence of the person x as the target object. As shown in FIG. 8, the frame f surrounding the person x is displayed on the HUD 104 as “second information” with the color thereof set to yellow (FIG. 3 / S134). Note that a voice shorter than (or smaller than) the first information such as “beep” as the second information may be output from a speaker (not shown).

According to the vehicle periphery monitoring system 10 of the present invention that exhibits the above-described function, a general human being can be selected from among target part candidates that may correspond to both legs as a plurality of target parts that a person as a target object has. In view of the arrangement and size of both legs, the object that is highly likely to fall under both human legs is recognized as a target object (FIGS. 3 / S123, S124, and FIG. 6). As a result, as shown in FIG. 4, even when the person x is mixed in the background in the grayscale image, the presence of the person x or both legs can be recognized with high accuracy.

In addition, only a plurality of primary target part candidates c ₁ satisfying (requirement 11) and (requirement 12) according to the shape and arrangement of both human legs as a plurality of target parts. can be recognized as a target partial candidate (secondary target partial candidate) c _2. This eliminates the need to execute the second process (FIG. 3 / S12) for determining whether or not the target part candidate that is unlikely to fall under both human legs falls under both human legs. The economy can be planned.

Further, since the presence of a human (target object) or both legs (a plurality of target portions) can be recognized with high accuracy, the possibility that the vehicle 1 and the target object come into contact can be evaluated with high accuracy (FIG. 3 / S132). Further, since the possibility of contact between the vehicle 1 and the target object can be evaluated with high accuracy, from the viewpoint of avoiding contact between the vehicle 1 and the target object, the frame f () that emphasizes the target object or a plurality of target portions thereof. The first information or the second information) can be appropriately displayed on the HUD 104 (FIG. 8). This allows the driver of the vehicle 1 to recognize the presence of a person in front and appropriately control the behavior of the vehicle 1 such as deceleration and steering in order to avoid contact between the vehicle 1 and this person.

In the above embodiment, the human leg is recognized as the target part. However, in another embodiment, other human body parts such as the human arms based on the same idea as the case where both human legs are recognized. May be recognized as the target portion.

In the embodiment, the third processing unit 13 displays the first or second information that emphasizes the presence of the target object on the HUD 104 based on the evaluation of the possibility of contact between the vehicle 1 and the target object. As an embodiment, the third processing unit 13 may control the behavior of the vehicle 1 based on the evaluation or may be controlled by an in-vehicle device such as a steering system or a brake system. According to the vehicle periphery monitoring system 10 having the above configuration, the in-vehicle device is controlled based on the possibility of contact between the vehicle 1 and the target object evaluated with high accuracy as described above. From the viewpoint of avoiding this contact, the behavior of the vehicle 1 such as deceleration and steering can be appropriately controlled.

Configuration diagram of vehicle, vehicle periphery monitoring system and server of the present invention Configuration explanatory diagram of the vehicle periphery monitoring system of the present invention Explanatory drawing about the function of the vehicle periphery monitoring system of the present invention Explanatory drawing about the function of the vehicle periphery monitoring system of the present invention Explanatory drawing about the function of the vehicle periphery monitoring system of the present invention Explanatory drawing about the function of the vehicle periphery monitoring system of the present invention Explanatory drawing about the function of the vehicle periphery monitoring system of the present invention Explanatory drawing about the function of the vehicle periphery monitoring system of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Vehicle periphery monitoring system, 11 ... 1st process part, 12 ... 2nd process part, 13 ... 3rd process part, 20 ... Server, 102 ... Infrared camera (imaging device), 104 ... HUD (information Output device (vehicle equipment)), 122 ... yaw rate sensor, 124 ... speed sensor

Claims (5)

A system for monitoring the periphery of a vehicle based on an image obtained by an imaging device mounted on the vehicle,
The luminance in the image is recognized as a peak point a point which indicates a locally peaks and a line segment composed of a peak point group with continuity, the object side-by-side pair of longitudinal of the line segment A first processing unit that is recognized as a target portion candidate of
The horizontal interval and vertical size in the real space of each of the pair of target portion candidates recognized by the first processing unit are measured, and the measured horizontal interval and vertical size, the interval And the ratio of the size, or the interval and the size, and the ratio are the horizontal distance and the vertical size of both legs as a plurality of target parts of a human being as a target object, the ratio of the distance and the size, Or a second processing unit that determines whether or not the pair of target portion candidates corresponds to both human legs according to whether or not the requirements according to the interval, the size, and the ratio are satisfied. A vehicle periphery monitoring system characterized by this. In the vehicle periphery monitoring system according to claim 1,
The first processing unit sets the line segment as a primary target portion candidate, and one or both of shapes and arrangements of the target portions of the target object in the image among the plurality of primary target portion candidates. A vehicle periphery monitoring system that recognizes a plurality of primary target part candidates that satisfy the requirements according to the above as secondary target part candidates corresponding to the target part candidates. In the vehicle periphery monitoring system according to claim 1,
A third method for evaluating the possibility of contact between the vehicle and the target object having the target portion based on a time-series measurement position of the target portion candidate recognized as corresponding to the target portion by the second processing unit. A vehicle periphery monitoring system, further comprising a processing unit. In the vehicle periphery monitoring system according to claim 3 ,
An information output device in which the third processing unit is mounted on the vehicle with information that informs or emphasizes the presence of the target object or the target portion based on the evaluation of the possibility of contact between the vehicle and the target object. A vehicle periphery monitoring system characterized in that the vehicle periphery is output. In the vehicle periphery monitoring system according to claim 3 ,
The vehicle periphery monitoring characterized in that the third processing unit controls the behavior of the vehicle or causes a device mounted on the vehicle to control based on the evaluation of the possibility of contact between the vehicle and the target object. system.

Images