JP4173896B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle from an image acquired through an imaging unit such as a camera mounted on the vehicle.

  2. Description of the Related Art In recent years, a pedestrian who has a high possibility of a collision by mounting an image pickup unit such as a CCD camera on a vehicle, picking up an image of the periphery, and detecting the position of an object such as a pedestrian existing around the vehicle from the picked-up image A technique for determining a target object such as information and presenting information to a driver is known (see, for example, Patent Document 1).

  In the vehicle periphery monitoring device of Patent Document 1, a stereo camera composed of two infrared cameras is mounted on the vehicle, and is present in the vicinity of the vehicle based on image shift (parallax) obtained from the two infrared cameras. The relative position of the target object to the vehicle is detected as position data. Then, the movement vector of the object with respect to the vehicle is calculated based on the time-series data of the position of the object, and the object is highly likely to collide with the vehicle based on the position data of the object and the movement vector. Is determined.

  At this time, in the vehicle periphery monitoring device, an approach determination corresponding to a roadway portion on which the vehicle travels in an alarm determination region within a predetermined distance in front of the vehicle (region determined according to the relative speed of the object with respect to the vehicle). The area AR1 and the entry determination areas AR2 and AR3, which are areas laterally outside the approach determination area AR1, are set. Then, when an object is present in the approach determination area AR1, it is determined that the object is a pedestrian or the like on the roadway and has a high possibility of collision with the vehicle. Further, when there is an object in the entry determination areas AR2 and AR3, the object is a roadside pedestrian or the like, an approach collision determination is performed based on the movement vector of the object, and the object and The possibility of a collision with the vehicle is determined. As a result, it is determined that the possibility of a collision is low for a pedestrian who normally walks on the roadside, and a collision is determined to be high for a pedestrian who moves to cross the road from the roadside. Appropriate information is presented to the person.

On the other hand, for example, a roadside pedestrian may suddenly jump out onto the road. In such a case, it is desirable to present information to the driver and control vehicle behavior as quickly as possible. However, in the vehicle periphery monitoring device disclosed in Patent Document 1, the possibility of a collision of an object existing in the approach determination area is determined based on a movement vector calculated from a plurality of time-series position data. It takes time to determine that the property is high. Therefore, when a pedestrian on the road side suddenly jumps out to the roadway, there is an inconvenience that information presentation to the driver and vehicle behavior control cannot be performed quickly.
JP 2001-6096 A

  The present invention eliminates such inconvenience, quickly determines a pedestrian as a target to avoid contact with the vehicle from images around the vehicle, and performs information presentation to the driver and control of vehicle behavior. An object of the present invention is to provide a vehicle periphery monitoring device capable of performing the above.

In order to achieve such an object, a vehicle periphery monitoring device according to the present invention is a vehicle periphery monitoring device that monitors the periphery of a vehicle from an image acquired via an image capturing unit mounted on the vehicle. The object extraction means for extracting objects existing around the vehicle from the acquired image, the pedestrian extraction means for extracting pedestrians from the objects extracted by the object extraction means, A determination algorithm including a posture determination unit that determines a posture related to the inclination of the trunk axis of the pedestrian extracted by the pedestrian extraction unit and at least a first determination process related to the posture of the pedestrian determined by the posture determination unit is executed. By doing so, avoidance target determination means for determining whether or not the object extracted by the object extraction means is an avoidance target that should avoid contact with the vehicle, Depending on the judgment result of the avoidance judging section even without, characterized in that it comprises a vehicle equipment control means for controlling the equipment of the vehicle (first invention).

  In the vehicle periphery monitoring device of the present invention, the object extracting means extracts an object existing around the vehicle from an image acquired via the imaging means, and the pedestrian extracting means extracts the object. Of them are extracted. Here, for example, when a pedestrian on the road side suddenly jumps out to the roadway, it is necessary to perform information presentation such as warning to the driver, control of vehicle behavior, etc. as quickly as possible. For this reason, it is desired to quickly predict the crossing of a pedestrian road in advance. At this time, it is generally considered that a pedestrian trying to cross a road and other pedestrians have different postures related to body orientation, inclination, and the like at the time when the pedestrian exists on the roadside. For example, when the pedestrian starts moving or jumps out onto the roadway, the trunk axis is tilted with respect to the traveling direction, whereas when the pedestrian is stopped or on the road When the movement is continued along, the inclination of the trunk axis is sufficiently small.

Therefore, the posture determination unit determines a posture related to the inclination of the trunk axis of the pedestrian from the image acquired through the imaging unit. Based on the posture of the pedestrian thus determined, the behavior of the pedestrian such as a crossing of the road can be predicted quickly in advance. And since the determination algorithm which the said avoidance object determination means performs includes the 1st determination process regarding the said pedestrian's attitude | position, this avoidance object determination means is an avoidance object which should avoid a contact with a vehicle. When determining whether or not, it is possible to quickly determine the determination result. Therefore, by controlling the vehicle device according to the determination result of the avoidance target determination unit by the vehicle device control unit, it is possible to quickly control the vehicle device corresponding to the avoidance target. .

  Specifically, in the vehicle periphery monitoring device of the present invention, the device controlled by the vehicle device control means preferably includes a device capable of issuing an alarm to the driver of the vehicle (second invention). .

  According to this, since it is possible to quickly determine whether or not the object is an avoidance target by the avoidance target determination means, the vehicle equipment control means alerts the driver in response to the avoidance target. Can be issued quickly.

  Alternatively, in the vehicle periphery monitoring device of the present invention, it is preferable that the device controlled by the vehicle device control means includes a device capable of operating the traveling behavior of the vehicle (third invention).

  According to this, since it is possible to quickly determine whether or not the object is an avoidance target by the avoidance target determination means, the vehicle device control means determines the traveling behavior of the vehicle corresponding to the avoidance target. It can be operated quickly.

  Further, in the vehicle periphery monitoring device according to any one of the first to third aspects of the invention, the relative position detection means for sequentially detecting the relative position of the object extracted by the object extraction means with respect to the vehicle, and the determination of the posture determination means Based on a time series of the relative position of the object detected by the relative position detection means at a time interval longer than the processing execution cycle, a moving direction feature amount representing the moving direction of the object relative to the vehicle is calculated. And a moving direction feature amount calculating means that performs the determination algorithm executed by the avoidance target determining means together with the first determination processing, the moving direction of the object calculated by the moving direction feature amount calculating means. Including a second determination process relating to a feature amount, wherein the first determination process is a process of determining whether or not the posture of the pedestrian satisfies a predetermined first requirement. The process is a process of determining whether or not the moving direction feature quantity of the object satisfies a predetermined second requirement, and the avoidance target determining means is a second determination process related to the moving direction feature quantity of the target object. When the determination result satisfies the second requirement, the determination result of the second determination process does not satisfy the second requirement, and the determination result of the first determination process regarding the posture of the pedestrian satisfies the first requirement. When satisfying, it is preferable to determine that the object is an avoidance target (fourth invention).

  According to this, since the determination algorithm executed by the avoidance target determination unit includes the second determination process related to the moving direction feature amount, it is possible to determine a determination result with high reliability. However, since the calculation of the moving direction feature amount is performed at a time interval longer than the execution cycle of the determination process of the posture determination unit, it takes time to determine the determination result of the avoidance target determination unit. On the other hand, the determination algorithm executed by the avoidance target determination means includes a first determination process related to the posture of the pedestrian, so that the determination result can be quickly determined. Therefore, the avoidance target determination unit satisfies the second requirement when the determination result of the second determination process related to the moving direction feature quantity of the object satisfies the second requirement, and the determination result of the second determination process satisfies the second requirement. In addition, when the determination result of the first determination process regarding the posture of the pedestrian satisfies the first requirement, it is determined that the object is an avoidance target.

  Accordingly, when a highly reliable determination result based on the moving direction feature amount is obtained, the avoidance target determination unit determines whether the target object is an avoidance target based on the determination result. For example, even when there is not enough time to calculate the moving direction feature amount, it is possible to determine whether the target object is an avoidance target based on a quick determination result based on the pedestrian's posture. is there. Therefore, it can be determined more reliably whether the target object is an avoidance target that should avoid contact with the vehicle.

  The first requirement is, for example, a requirement that the moving direction of the object is a direction toward the host vehicle. The second requirement is, for example, a requirement that the object is a pedestrian and that the posture of the pedestrian is a forward leaning posture toward the center of the lane in which the host vehicle is traveling.

  Furthermore, in the vehicle periphery monitoring device according to the fourth aspect of the present invention, the vehicle equipment control means includes a case where the determination result of the second determination process relating to the moving direction feature quantity of the object satisfies the first requirement, and the second determination When the determination result of the process does not satisfy the first requirement, and the determination result of the first determination process regarding the posture of the pedestrian satisfies the second requirement, the vehicle devices are controlled in different forms. It is preferable (5th invention).

  That is, when the determination result of the second determination process related to the moving direction feature amount of the object satisfies the first requirement, the determination result of the second determination process does not satisfy the first requirement, and the pedestrian The reliability of the final determination result by the avoidance target determination means, the urgency of avoidance, and the like are different from the case where the determination result of the first determination process related to the posture satisfies the second requirement. Therefore, the vehicle equipment control means can control the vehicle equipment in a different form for each case, thereby more appropriately performing information presentation to the driver and vehicle behavior control.

  For example, when the device controlled by the vehicle device control means is a device capable of issuing a warning to the driver of the vehicle, the vehicle device control means uses a different warning method for each case, An alarm may be issued to the driver.

Further, in the vehicle periphery monitoring device according to any one of the first to fifth aspects of the invention, the posture determination means determines the symmetry of the pedestrian on the image in the left-right direction of the vehicle with respect to the trunk axis of the pedestrian. It is preferably a means for determining the degree of symmetry, which is a feature amount representing an attitude related to inclination (sixth invention).

  That is, in general, the posture of a pedestrian trying to cross a road is a forward leaning posture toward the center of the lane in which the vehicle travels. At this time, on the image acquired through the imaging means mounted on the vehicle, the side opposite to the portion near the center side of the lane in the region corresponding to the pedestrian, with the head of the pedestrian as the center In this part, the degree of symmetry is considered to be low. That is, the symmetry of the pedestrian in the left-right direction of the vehicle on the image is closely related to whether or not the pedestrian is about to cross the road (whether or not it is a forward leaning posture). This is a feature quantity related to. Therefore, by using the degree of symmetry, for example, it can be determined whether or not the pedestrian's posture is a forward leaning posture toward the center of the lane on which the vehicle travels. Is predicted quickly. Therefore, the avoidance target determination means determines whether or not the pedestrian should be avoided based on the degree of symmetry determined by the posture determination means in the first determination process. Thus, it is possible to quickly predict the behavior of a pedestrian such as a crossing of a road and quickly determine whether or not the pedestrian is an object to avoid contact with a vehicle.

Alternatively, in the vehicle periphery monitoring device according to any one of the first to fifth aspects of the invention, the posture determination means uses the inclination of the trunk axis of the pedestrian as a feature amount representing the posture related to the inclination of the trunk axis of the pedestrian , A means for determining the degree of the inclination is preferred (seventh invention).

  That is, the inclination of the trunk axis (the direction and angle of the trunk axis) of the pedestrian is a feature amount that represents the posture related to the inclination of the trunk axis of the pedestrian. It can be determined whether or not the posture is a forward leaning posture toward the center of the lane in which the vehicle travels. At this time, in general, the posture of the pedestrian trying to cross the road is the forward leaning posture toward the center of the lane in which the vehicle travels (the posture in which the trunk axis is tilted forward to the center of the lane in which the vehicle travels Therefore, the pedestrian's behavior such as crossing the road is quickly predicted from the degree of inclination of the trunk axis. Therefore, the avoidance target determination unit determines whether or not the pedestrian is an avoidance target based on the degree of inclination of the trunk axis determined by the posture determination unit in the first determination process. Thus, it is possible to quickly predict the behavior of a pedestrian such as crossing a road and quickly determine whether or not the pedestrian is an object to avoid contact with a vehicle.

  An embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a functional block diagram of the vehicle periphery monitoring device according to the present embodiment, FIG. 2 is an explanatory view of an attachment mode of the vehicle periphery monitoring device shown in FIG. 1 to the vehicle, and FIG. It is a flowchart which shows the target object detection and alarm operation | movement in the vehicle periphery monitoring apparatus. FIG. 4 is an exemplary view of a processed image in the object detection / alarm operation of FIG.

  With reference to FIGS. 1 and 2, the vehicle periphery monitoring device of the present embodiment includes an image processing unit 1 which is an electronic unit including a CPU (Central Processing Unit). The image processing unit 1 includes two infrared cameras 2R and 2L mounted on the host vehicle 10, a yaw rate sensor 3 that detects the yaw rate of the host vehicle 10, and a vehicle speed that detects the traveling speed (vehicle speed) of the host vehicle 10. The sensor 4 and the brake sensor 5 for detecting the operation of the brake of the host vehicle 10 are connected.

  In addition, the image processing unit 1 displays an image acquired via the infrared camera 2R, 2L and a speaker 6 that is provided in the vehicle periphery monitoring device for issuing a warning by voice and has a possibility of a collision. A display device 7 is connected to allow the driver to recognize a high object. The display device 7 includes, for example, a HUD (head-up display) 7 a that displays information such as an image on the front window of the host vehicle 10. The HUD 7 a is provided so that the screen is displayed at a position that does not obstruct the driver's forward view of the front window of the host vehicle 10.

  The infrared cameras 2R and 2L are cameras that can detect far-infrared rays, and have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases. The infrared cameras 2R and 2L correspond to the imaging means of the present invention.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are disposed on the right side and the left side of the front portion of the host vehicle 10 at positions that are substantially symmetrical with respect to the center in the vehicle width direction of the host vehicle 10. The infrared cameras 2R and 2L are fixed to the host vehicle 10 so that the optical axes of the two two infrared cameras 2R and 2L are parallel to each other and the heights from both road surfaces are equal.

  Although not shown in detail, the image processing unit 1 is configured by an electronic circuit including an A / D conversion circuit, a CPU, a RAM, a ROM, an image memory, and the like, and includes infrared cameras 2R and 2L, a yaw rate sensor 3, and a vehicle speed sensor 4. The output (analog signal) of the brake sensor 5 is converted into a digital signal via the A / D conversion circuit and input. Then, based on the input data, the image processing unit 1 determines whether or not a process for detecting an object such as a pedestrian or a predetermined requirement for the detected object is satisfied, and the requirement is satisfied. In such a case, a process for issuing a warning to the driver via the speaker 6 or the display device 7 is executed.

  More specifically, the image processing unit 1 functions as a function of object extraction means 11 for extracting an object existing around the host vehicle 10 from the acquired image, and a pedestrian among the extracted objects. Pedestrian extraction means 12 to be extracted, posture determination means 13 to determine the posture of the extracted pedestrian, and avoidance target to determine whether the object is an avoidance target that should avoid contact with the host vehicle 10 A determination unit 14 and a vehicle device control unit 15 that controls the device of the host vehicle 10 according to the determination result of the avoidance target determination unit 14 are provided. In addition, the image processing unit 1 has, as its function, a relative position detection unit 16 that sequentially detects a relative position of the object with respect to the host vehicle 10, and a movement direction feature value calculation unit 17 that calculates a movement direction feature value of the object. Is provided.

  The object extraction means 11 extracts an object existing around the host vehicle 10 from images acquired via the infrared cameras 2R and 2L. Specifically, the object extracting means 11 is a predetermined reference image among images acquired via the infrared cameras 2R and 2L (in this embodiment, it is assumed that the image is acquired via the infrared camera 2R). Then, binarization processing is performed to extract a binarized region where the luminance value of the pixel in the reference image is equal to or greater than a predetermined threshold. Furthermore, the object extraction means 11 creates run length data from the binarized area, and extracts the object by a labeling process or the like based on the run length data.

  The relative position detection means 16 searches for an object corresponding to the object extracted by the object extraction means 11 on the reference image on the image acquired via the infrared camera 2L, and two infrared cameras 2R, 2L. Based on the displacement (parallax) of the object on the image obtained from the above, the distance of the object to the host vehicle 10 is detected, converted into real space coordinates, and the relative position of the object to the vehicle 10 is detected. As a specific method for detecting the relative position of the object based on the image, for example, a method as described in Patent Document 1 described above can be used.

  Based on the time series of the relative position of the object detected by the relative position detection means 16, the movement direction feature value calculation means 17 calculates a movement direction feature value representing the movement direction of the object relative to the host vehicle 10. Specifically, the movement direction feature quantity calculating means 17 calculates a movement vector of the target object with respect to the host vehicle 10 from the time series data of the relative position of the target object with respect to the host vehicle 10 as the movement direction feature quantity. Note that time series data for a predetermined period is required to calculate the movement vector.

  The pedestrian extraction unit 12 extracts a pedestrian from the objects extracted by the object extraction unit 11. As a method of extracting a pedestrian, for example, the characteristics indicating the shape of the object such as the aspect ratio of the object, the ratio of the area of the object and the area of the circumscribed rectangle of the object, and the size of the object A method for determining whether or not the object is a pedestrian is used based on characteristics such as luminance dispersion on the gray scale image.

  The posture determination unit 13 uses the inclination of the trunk axis of the pedestrian extracted by the pedestrian extraction unit 12 as a feature amount representing the posture of the pedestrian, and determines the degree of the inclination. As a method of calculating the trunk axis of the pedestrian, for example, first, run length data is created from a binarized region in which the luminance value of the pixel in the acquired image is equal to or greater than a predetermined threshold. Then, a method is used in which point sequence data composed of the coordinates of the midpoint pixels of each line of the generated run length data is calculated, and an approximate straight line approximating the point sequence data is used as the trunk axis of the pedestrian. Note that the discrimination processing by the posture discrimination means 13 is executed sequentially (every arithmetic processing cycle).

  The avoidance target determination means 14 determines whether or not the object extracted by the object extraction means 11 is an avoidance target that should avoid contact with the host vehicle 10. At this time, the determination algorithm executed by the avoidance target determination unit 14 includes a first determination process related to the posture of the pedestrian determined by the posture determination unit 13 and the movement of the target calculated by the movement direction feature amount calculation unit 17. And a second determination process related to the direction feature amount.

  The vehicle equipment control means 15 issues a warning to the driver of the host vehicle 10 about the object determined as the avoidance target by the avoidance target determination means 14. Specifically, the vehicle device control means 15 outputs voice guidance through the speaker 6 or outputs an image obtained by, for example, the infrared camera 2R to the display device 7, and is an avoidance target on the image. An object such as a pedestrian is emphasized and displayed to the driver of the host vehicle 10. The speaker 6 and the display device 7 correspond to devices controlled by the vehicle device control means according to the present invention.

  Next, the overall operation (object detection / alarm operation) of the vehicle periphery monitoring device of the present embodiment will be described with reference to the flowchart shown in FIG. Referring to FIG. 3, the image processing unit 1 repeats the processing of STEP001 to STEP020 for each predetermined calculation processing cycle to execute the object detection / alarm operation. First, the image processing unit 1 acquires an infrared image that is an output signal of the infrared cameras 2R and 2L (STEP001), performs A / D conversion (STEP002), and stores a grayscale image in an image memory (STEP003). The right image is obtained by the infrared camera 2R, and the left image is obtained by the infrared camera 2L. Since the right image and the left image are displayed with the positions in the horizontal direction (x direction) shifted on the image of the same object, the distance to the object can be calculated based on this shift (parallax). it can.

  Next, the image processing unit 1 binarizes the image signal of a reference image (in this embodiment, an image obtained by the infrared camera 2R) of the grayscale image (STEP 004). That is, a process is performed in which a region where the luminance value of the image signal of the reference image is brighter than the threshold value Ith is set to “1” (white) and a dark region is set to “0” (black). The threshold value Ith is a value that is experimentally determined in advance.

  FIG. 4A illustrates an image obtained by binarizing an image obtained by the infrared camera 2R. In FIG. 4A, the hatched area is black, and the area surrounded by the thick solid line is white. A region surrounded by a thick solid line is a region of an object that has a high luminance level (at a high temperature) and is displayed as white on the screen in the image obtained from the infrared camera 2. In the example shown in FIG. 4A, there is a pedestrian on the left side in front of the host vehicle 10, and there is another vehicle in front of the host vehicle 10, and the pedestrian is about to cross the road. It is an example.

  Next, the image processing unit 1 creates run-length data from an area (binarized area) that has become “white” in the binarization process (STEP 005). The generated run-length data represents a binarized area as a set of lines that are one-dimensional connected pixels in the horizontal direction of the image, and each run constituting the binarized area is represented by the coordinates of the start point and the start point. This is indicated by the length to the end point (number of pixels).

  Next, the image processing unit 1 extracts the target object by labeling the target object (STEP 006) based on the generated run length data (STEP 007). That is, among the lines represented by the run length data, a run having a portion overlapping in the vertical direction (y direction) of the image is regarded as one object, and a label (identifier) is attached, thereby connecting connected regions in the image. Is extracted as an object.

  By the above-described processing of STEP 005 to 007, the binarized areas shown in FIG. 4A are extracted as objects (binarized objects) T1 to T6 as illustrated in FIG. 4B. At this time, for example, the object of the label T1 is indicated by n pieces of run-length data L1 to Ln. The extracted objects (binarized objects) include, for example, other vehicles, artificial structures such as electric poles and vending machines, in addition to pedestrians around the road. In the example of FIG. 4B, the target object T1 is a binarized target object corresponding to a pedestrian existing on the left side in front of the host vehicle 10, and the target objects T2 to T6 are positioned in front of the host vehicle 10. It is a binarized object corresponding to another vehicle that exists.

  Next, the image processing unit 1 calculates the area S, the center of gravity G, and the aspect ratio ASPECT of the circumscribed rectangle of the target object (STEP008). Specifically, the area S of the object T1 is obtained by integrating the length of the line indicated by each run-length data Li (i = 1,..., N) with respect to the n pieces of run-length data of the object T1. To calculate. The coordinates of the center of gravity G of the object T1 are the length of the line indicated by each run-length data Li and the coordinates (x [i], y [i]) of the midpoint of each line of the run-length data Li. Multiplying these values and multiplying them by n run-length data of the object T1 is calculated by dividing by the area S. Further, the aspect ratio ASPECT of the object T1 is calculated as a ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object T1.

  Next, the image processing unit 1 performs tracking of the object for the time, that is, recognizes the same object for each calculation cycle of the image processing unit 1 (STEP 009). In the inter-time tracking process, the object A is extracted by the process of STEP007 at the time (discrete system time) k in a certain calculation process cycle, and the object B is extracted by the process of STEP007 at the time k + 1 of the next calculation process period. Then, the identity of the objects A and B is determined. This identity determination is performed based on, for example, the center of gravity G, the area S, the aspect ratio ASPECT of the objects A and B, and the like. When it is determined that the objects A and B are the same, the label of the object B extracted at time k + 1 is changed to the same label as the label of the object A. This inter-time tracking process is executed for a binarized reference image.

  Next, the image processing unit 1 reads the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3 and integrates the yaw rate YR over time, thereby calculating the turning angle θr of the host vehicle 10. Calculate (STEP010).

  On the other hand, in parallel with the processing of STEP 009 and 010, in STEP 011 to 014, processing for calculating the distance z between the object and the host vehicle 10 (the distance in the front-rear direction of the host vehicle 10) is performed. Since this calculation takes a longer time than STEP 009,010, it is executed in a cycle longer than STEP 009,010 (for example, a cycle about three times the execution cycle of STEP001 to 010).

  First, the image processing unit 1 selects one of the objects to be tracked by the binarized image of the reference image, so that the search image R1 (the entire circumscribed quadrangular region surrounding the object) is selected from the reference image. Are used as search images) (STEP 011).

  Next, the image processing unit 1 selects an image corresponding to the search image R1 (hereinafter referred to as “corresponding image”) from among the reference images (the right image and the left image obtained from the infrared cameras 2R and 2L). ) Is set, a correlation operation is performed, and a corresponding image is extracted (STEP 012). Specifically, the search area R2 is set in the reference image according to each vertex coordinate of the search image R1, and the coordinates (x0, y0) are set as the base point (upper left vertex of the area) in the search area R2. A local region R3 having the same shape as the search image R1 is set. Then, by changing the coordinates (x0, y0) of the base point and moving the local region R3 within the search region R2, the absolute difference sum of luminance values (SAD) indicating the degree of correlation between the local region R3 and the search image R1 , Sum of Absolute Difference) C (x0, y0) is calculated by the following equation (1).

Here, the absolute difference sum C (x0, y0) is a local area based on the luminance value IR of the pixel at the coordinates (m, n) in the search image R1 and the coordinates (x0, y0) in the search area R2. The absolute value of the difference from the luminance value IL of the pixel at the coordinate (x0 + m, y0 + n) in R3 is taken, and all the pixels (m = 0,..., M) in the search image R1 and local region R3 of the absolute value of this difference. M−1, n = 0,..., N−1) is obtained as a total value. Note that the smaller the value of the absolute difference sum C (x0, y0), the higher the degree of correlation between the search image R1 and the local region R3. Thus, the coordinates (x0, y0) of the base point at which the absolute difference sum C (x0, y0) is minimum are obtained, and the local region R3 at this position is extracted as the corresponding image R4. This correlation calculation is performed using a grayscale image instead of a binarized image. Through the processing in STEPs 011 to 012, the search image R1 in the standard image and the corresponding image R4 in the reference image are extracted.

  Next, the image processing unit 1 calculates the parallax Δd (number of pixels) based on the centroid position of the search image R1 and the centroid position of the corresponding image R4 (STEP013). Then, the image processing unit 1 calculates the distance z between the host vehicle 10 and the object by the following equation (2) using the calculated parallax Δd (STEP014).

z = B × F / (Δd × p) (2)
Note that B is the base line length (interval of the optical axis) of the infrared cameras 2R and 2L, F is the focal length F of the infrared cameras 2R and 2L, and p is the pixel interval.

  After the processing of STEP010 and STEP014, the image processing unit 1 next converts the coordinates (x, y) and the distance z in the image into real space coordinates, and the position of each object in the real space. The real space position (relative position with respect to the host vehicle 10) is calculated (STEP015). Here, the real space position is, as shown in FIG. 2, a real space coordinate system (XYZ coordinate system) set with the midpoint of the attachment position of the infrared cameras 2R and 2L (the position fixed to the host vehicle 10) as the origin. ) At position (X, Y, Z). The X direction and Y direction of the real space coordinate system are the left-right direction (vehicle width direction) and the up-down direction of the host vehicle 10, respectively. These X direction and Y direction are the x direction (horizontal direction) of the right image and the left image. Direction) and y direction (longitudinal direction). The Z direction in the real space coordinate system is the front-rear direction of the host vehicle 10. The real space position (X, Y, Z) is calculated by the following equations (3), (4), and (5).

X = x × z × p / f (3)
Y = y × z × p / f (4)
Z = z (5)
Next, the image processing unit 1 uses the turning angle θr calculated in STEP 010 in order to compensate for the influence of the change in the turning angle of the host vehicle 10 and improve the accuracy of the real space position of the object. The real space position of is corrected. (STEP016). In the turning angle correction, when the host vehicle 10 turns, for example, to the left by the turning angle θr during the period from time k to (k + 1), the range of the image is in the x direction on the images obtained by the infrared cameras 2R and 2L. Since it deviates, this is a process for correcting this. In the following description, “the real space position of the object” means the real space position of the object subjected to the turning angle correction.

  Next, the image processing unit 1 obtains a movement vector of the object relative to the host vehicle 10 (STEP017). Specifically, a straight line that approximates time-series data in a predetermined period (a period from the current time to a predetermined time) of the real space position of the same object is obtained, and the straight line at the time before the predetermined time is obtained. A vector that moves from the position of the object (coordinate Pv1 = (Xv1, Yv1, Zv1)) to the position of the object on the straight line (coordinate Pv0 = (Xv0, Yv0, Zv0)) at the current time. Asking. In addition, the method described in the above-mentioned patent document 1 is used for the specific calculation process of the approximate straight line. Further, in this process, since the calculation processing cycle for a predetermined period has not elapsed since the object was detected, the time series data of the real space position of the object is insufficient and the movement vector cannot be calculated. In the case, proceed to STEP018 as it is.

  Next, the image processing unit 1 performs an avoidance determination process for determining whether or not the detected object is an avoidance object (an object that should avoid contact with the host vehicle 10) (STEP018). Details of the avoidance determination process will be described later. If it is determined in STEP 018 that the detected object is not an avoidance target (the determination result in STEP 018 is NO), the process returns to STEP 001 and the above-described processing is repeated. If it is determined in STEP 018 that the detected object is an avoidance target (YES in STEP 018), the process proceeds to an alarm output determination process in STEP 019.

  In STEP019, the image processing unit 1 performs an alarm output determination process for determining whether or not to output an actual alarm regarding the object. In this alarm output determination process, it is confirmed from the output BR of the brake sensor 5 that the driver has operated the brake of the host vehicle 10, and the deceleration acceleration of the host vehicle 10 (the acceleration in the vehicle speed decreasing direction is corrected). Is greater than a predetermined threshold (> 0), it is determined that no alarm is output. Further, when the driver does not perform a braking operation or when the deceleration acceleration of the host vehicle 10 is equal to or less than a predetermined threshold even if the braking operation is performed, it is determined that an alarm is output.

  When the image processing unit 1 determines that an alarm is to be output (the determination result in STEP019 is YES), the image processing unit 1 performs an alarm output process for issuing an alarm by the speaker 6 and the display device 7 to the driver of the host vehicle 10. Execute (STEP020), return to STEP001, and repeat the above process. Details of the alarm output process will be described later. When it is determined in STEP019 that no alarm is output (when it is determined that no alarm is output for all objects), the determination result in STEP019 is NO. In this case, the process returns to STEP001 as it is, Repeat the process.

  The above is the object detection / alarm operation in the image processing unit 1 of the vehicle periphery monitoring device of the present embodiment. As a result, an object such as a pedestrian in front of the host vehicle 10 is detected from the infrared image around the host vehicle 10 and a signal indicating the traveling state of the host vehicle 10, and the driver about the target object to be avoided. An alarm is issued.

  Next, the avoidance determination process in STEP018 of the flowchart shown in FIG. 3 will be described in detail with reference to the flowchart shown in FIG. FIG. 5 is a flowchart showing the avoidance determination process of the present embodiment. The avoidance determination process is detected by the following collision determination process, determination process whether or not the vehicle is in the approach determination area, approach collision determination process (by movement vector or posture), pedestrian determination process, and artificial structure determination process This is a process for determining the possibility of collision between the target object and the host vehicle 10 and the type of the target object, and determining whether or not the target object is an avoidance target.

  Referring to FIG. 5, first, the image processing unit 1 performs a collision determination process as one of processes for determining the degree of possibility that the target object collides with the host vehicle 10 (STEP 101). In the collision determination process, the relative velocity Vs in the Z direction between the target object and the host vehicle 10 is obtained, and both of them collide within the margin time T on the assumption that both move within the height H while maintaining the relative velocity Vs. This is a process for determining whether or not there is a possibility of the failure. Specifically, when the coordinate value (distance) Zv0 in the current Z direction of the object is Vs × T or less and the coordinate value (height) Yv0 in the current Y direction of the object is H or less. It is determined that there is a possibility that the object and the host vehicle 10 collide.

  Here, with reference to FIG. 6, FIG. 6 is a view of the road on which the host vehicle 10 travels as viewed from above, and shows an area section ahead of the host vehicle 10. As shown in FIG. 6, if an area that can be monitored by the infrared cameras 2R and 2L is an outer triangular area AR0 indicated by a thick solid line, an area closer to the host vehicle 10 than Z1 (= Vs × T) in the area AR0. Is a region where there is a possibility of collision between the object and the host vehicle 10. The height H is a predetermined height that defines a range in the height direction, and is set to, for example, about twice the vehicle height of the host vehicle 10.

  If the determination result in STEP 101 is NO (there is no possibility that the target object and the host vehicle 10 will collide within the margin time T), the target 10 and the vehicle 10 are not allowed to collide by steering or braking operation of the vehicle 10. This is a situation that can be avoided. In this case, proceeding to STEP 108, the image processing unit 1 determines that the object is not an avoidance target, and ends the avoidance determination process.

  When the determination result in STEP 101 is YES (the object and the host vehicle 10 may collide within the margin time T), the process proceeds to STEP 102 and the degree of possibility that the object contacts the host vehicle 10 is determined. As one of the determination processes, the image processing unit 1 performs a determination process as to whether or not the object exists in the approach determination area AR1. As shown in FIG. 6, the approach determination area AR1 is a width (α + 2β) in which the margin β is added to both sides of the vehicle width α of the own vehicle 10 in the area closer to the own vehicle 10 than Z1 in the above-described area AR0. ) Is an area AR1. The approach determination area AR1 also has a predetermined height H.

  When the determination result in STEP 102 is YES (the object exists in the approach determination area AR1), the object collides with the host vehicle 10 when the object remains in the current real space position. There is a high possibility of doing. In this case, proceeding to STEP 103, the image processing unit 1 performs a pedestrian determination process for determining whether or not the object is a pedestrian. The pedestrian determination process is a process for determining whether or not the object is a pedestrian from features such as the shape and size of the object and luminance dispersion on the grayscale image. As a specific method of the pedestrian determination process, for example, a method described in Japanese Patent Application Laid-Open No. 2003-284057 by the present applicant can be used.

  If the determination result in STEP 103 is YES (the object may be a pedestrian), the process proceeds to STEP 104, and in order to further improve the reliability of the determination of the possibility that the object is a pedestrian, 1 performs an artificial structure determination process for determining whether or not the object is an artificial structure. Artificial structure determination processing is performed when an image of a target object is detected as a feature that is not a pedestrian, for example, matches the shape of a pre-registered artificial structure. This is a process of determining that the object is an object and excluding it from the alarm target.

  If the determination result in STEP 104 is NO (the object is not an artificial structure), the process proceeds to STEP 107, where the image processing unit 1 determines that the object is an avoidance target and ends the avoidance determination process. Therefore, when it is determined that the object is present in the approach determination area AR1, the object is likely to be a pedestrian, and is not an artificial structure, the object is determined to be an avoidance target. Is done.

  If the determination result in STEP 103 is NO (the object is not a pedestrian) or if the determination result in STEP 104 is YES (the object is an artificial structure), the process proceeds to STEP 108. The image processing unit 1 determines that the object is not an avoidance target, and ends the avoidance determination process.

  On the other hand, if the determination result in STEP 102 is NO (the object does not exist in the approach determination area AR1), the process proceeds to STEP 105, where the image processing unit 1 enters the approach determination area AR1 and the own vehicle An approach collision determination process for determining whether or not there is a possibility of collision with the vehicle 10 is performed (STEP 105 to 106). Here, as shown in FIG. 6, regions AR2 and AR3 in which the absolute value of the X coordinate is larger than the above-described approach determination region AR1 (outside in the lateral direction of the approach determination region AR1) in the region AR0 are set as the approach determination regions. . The approach collision determination process is a process for determining whether or not an object in the entry determination areas AR2 and AR3 moves into the approach determination area AR1 and collides with the host vehicle 10 by moving. The entry determination areas AR2 and AR3 also have a predetermined height H.

  First, in STEP 105, an approach collision determination process based on the movement vector of the object is performed. This approach collision determination process based on the movement vector corresponds to the second determination process of the determination algorithm executed by the avoidance target determination unit of the present invention. The second requirement for determining that the target object is an avoidance target (the possibility of a collision is high) is a requirement that the moving direction of the target object is a direction toward the host vehicle. Specifically, the X coordinate (position in the vehicle width direction) of the intersection of the XY plane (the plane perpendicular to the front-rear direction of the host vehicle 10) of the host vehicle 10 and a straight line including the movement vector of the target object is When the vehicle 10 is within a predetermined range slightly wider than the vehicle width α of the host vehicle 10 (when the object is relatively toward the host vehicle 10), it is determined that the possibility of a collision is high. If the movement vector cannot be calculated in the above STEP 017, the determination result in STEP 105 is NO.

  If the determination result in STEP 105 is YES, there is a high possibility that the object will collide with the host vehicle 10 in the future. Therefore, in this case, the process proceeds to STEP 108, and the image processing unit 1 determines that the object is an avoidance target, and ends the avoidance determination process.

  If the determination result in STEP 105 is NO, the process proceeds to STEP 106, and an approach collision determination process based on the posture is performed. The approach collision determination process based on this posture corresponds to the first determination process of the determination algorithm executed by the avoidance target determination unit of the present invention. In addition, the first requirement for determining that the target object is an avoidance target (the possibility of a collision is high) is that the target object is a pedestrian and the posture of the pedestrian is the lane in which the host vehicle 10 is traveling. This is a requirement that the posture is tilted forward to the center side. Details of the approach collision determination process based on the posture will be described later. If the determination result in STEP 106 is YES, there is a high possibility that the object will collide with the host vehicle 10 in the future. Therefore, in this case, the process proceeds to STEP 107, and the image processing unit 1 determines that the object is an avoidance target, and ends the avoidance determination process. If the determination result in STEP 106 is NO, the object is unlikely to collide with the host vehicle 10, so that the process proceeds to STEP 108, and the image processing unit 1 determines that the object is not an avoidance target and avoids it. The determination process ends.

  The above is the details of the avoidance determination process.

  Next, the alarm output process in STEP020 of the flowchart shown in FIG. 3 will be described in detail. In this alarm output process, the reference image is displayed on the display device 7 and the image of the target object to be avoided in the reference image is highlighted. Furthermore, a voice guidance is provided from the speaker 6 to the driver that there is an object to be avoided. As a result, the driver's attention to the target object to be avoided is alerted. Note that the warning for the driver may be performed by only one of the speaker 6 and the display device 7.

  At this time, when the determination result in STEP 105 is YES in the avoidance determination process in STEP 018 described above and the object is determined to be an avoidance object (hereinafter referred to as CASE 1), the image processing unit 1 When the determination result is YES and it is determined that the target object is an avoidance target (hereinafter referred to as CASE 2), a different type of warning is issued to the driver. Specifically, for example, when highlighting an object to be avoided with a frame on the image of the display device 7, different colors are used for CASE 1 and CASE 2, a color is lighter than CASE 1, or CASE 2 is used. Then, the frame is made thinner than CASE1. Alternatively, when voice guidance is performed using the speaker 6, the patterns of voice guidance texts are different between CASE1 and CASE2, and the type (for example, frequency) of voice guidance voices is different. Alternatively, CASE 1 performs both voice guidance by the speaker 6 and highlighting by the display device 7, and CASE 2 performs only highlighting by the display device 7. As a result, a warning is issued more appropriately so that the driver can recognize that the determination result of the avoidance target is different in reliability, avoidance urgency, and the like.

  The above is the details of the alarm output process.

  Next, the approach collision determination process based on the posture in STEP 106 of the flowchart shown in FIG. 5 will be described in detail. The approach collision determination process based on the posture is performed when the object in the entry determination areas AR2 and AR3 is a pedestrian, based on the posture of the pedestrian (the posture related to the inclination of the trunk axis of the pedestrian). This is a process for determining whether or not the pedestrian is an avoidance target (whether or not there is a possibility of a collision with the host vehicle 10). In the following description, the example shown in FIG. 4A (a pedestrian is present on the left side in front of the host vehicle 10, another vehicle is present in front of the host vehicle 10, and the pedestrian is on the road (If you are going to cross)

  In STEP 106, first, the pedestrian extraction means 12 performs a process of extracting a pedestrian from the detected objects. At this time, when the aspect ratio ASPECT of the object and the ratio SA / SB of the area SA of the object and the area SB of the circumscribed rectangle of the object are within a predetermined range, the object is a pedestrian. It is determined and the object is extracted as a pedestrian. The predetermined range is a range in which the object is predetermined as a value corresponding to the upper body or whole body of the pedestrian. Thereby, as illustrated in FIG. 4B, the object T1 surrounded by the frame R1 is extracted as a pedestrian from the objects T1 to T6. When no pedestrian is extracted, the determination result in STEP 106 is NO. In the process of extracting pedestrians, other methods such as those used in the pedestrian determination process of STEP 103 may be used.

  Next, when a pedestrian is extracted, the posture determination means 13 performs processing for determining the posture of the extracted pedestrian T1. Below, it demonstrates with reference to Fig.7 (a) (b). FIGS. 7A and 7B each show a region (including a pedestrian T1) surrounded by the frame R1 illustrated in FIG. 4B.

  First, the posture determination means 13 calculates the trunk axis of the pedestrian T1. Specifically, first, the posture determination means 13 is a point sequence composed of the coordinates of the middle point pixel of the line indicated by each run-length data Li from the extracted run-length data L1 to Ln of the pedestrian T1. Data P_T1 is calculated. With reference to Fig.7 (a), the binarization area | region equivalent to the target object extracted as the pedestrian T1 is represented by n pieces of run length data L1-Ln as shown in the figure. FIG. 7B shows run-length data L1 to Ln, and the midpoints M1 to Mn are as shown. The point sequence data P_T1 includes coordinate data {(X1, Y1), (X2, Y2),. . . , (Xn, Yn)}. In FIG. 7B, the coordinate axes X and Y are set as shown. At this time, the X-axis is an axis parallel to the left-right direction of the host vehicle 10, and the Y-axis is an up-down direction axis.

  Next, the posture determination unit 13 calculates an approximate line S1 that approximates the calculated point sequence data P_T1. At this time, the least square method is used as an approximation method. Thereby, as shown in FIG.7 (b), approximate straight line S1 = aX + b is calculated. The approximate straight line S1 corresponds to the trunk axis of the pedestrian T1.

Next, the posture determination means 13 determines the degree of inclination of the trunk axis of the pedestrian T1 based on the calculated approximate straight line S1. Specifically, in the present embodiment, whether or not the trunk axis of the pedestrian T1 is tilted forward to the center side of the lane in which the host vehicle 10 travels (the lane in which the pedestrian T1 is traveling in the lane in which the host vehicle 10 travels). Whether or not it is a forward leaning posture toward the vehicle). First, as shown in FIG. 7B, the posture determination unit 13 calculates an angle θ = tan ⁻¹ a formed by the approximate straight line S1 and the vertical axis Y. Next, the posture determination means 13 determines whether or not the trunk axis of the pedestrian T1 is tilted forward toward the center of the lane in which the host vehicle 10 travels based on the angle θ. In the present embodiment, the angle θ is positive in the direction from the vertical axis Y to the center side of the lane in which the host vehicle 10 travels, with the vertical axis Y direction being 0 degree (see FIG. 7B). In the example, the clockwise rotation is set to be positive).

  At this time, when the angle θ is greater than or equal to the threshold θ1 and less than or equal to the threshold θ2 (0 <θ1 <θ2), it is determined that the vehicle 10 is leaning forward toward the lane on which the host vehicle 10 travels. The threshold values θ1 and θ2 are predetermined values that indicate the range of angles that the trunk axis generally takes with respect to the traveling direction when a pedestrian is moving. Based on the determination result of the posture of the pedestrian T1, the behavior of the pedestrian T1 such as crossing a road is quickly predicted in advance. In the example of FIG. 7B, θ1 ≦ θ ≦ θ2, and it is determined that the pedestrian T1 is in a forward leaning posture toward the lane in which the host vehicle 10 is traveling. Thereby, it is predicted that pedestrian T1 is going to cross the road.

  Next, the avoidance target determination unit 14 determines whether or not the pedestrian T1 is an avoidance target based on the determination result of the posture of the pedestrian T1 by the posture determination unit 13. At this time, for example, when the pedestrian T1 is in a forward leaning posture toward the lane in which the host vehicle 10 is traveling (YES in STEP 106), it is predicted that the pedestrian T1 is about to cross the road. Therefore, in STEP 107, it is determined that the object is an avoidance target. Thereby, for example, when the pedestrian T1 existing in the entry determination areas AR2 and AR3 suddenly jumps out on the roadway, the warning output determination and the warning output in STEPs 019 to 020 in FIG. 3 are quickly performed.

  By the above process, according to the present embodiment, from the images acquired via the infrared cameras 2R and 2L, the pedestrian to be avoided among the pedestrians around the host vehicle 10 is quickly determined, Information can be presented to the driver of the host vehicle 10.

  In the present embodiment, as described above, the image processing unit 1 functions as the object extraction unit 11, the pedestrian extraction unit 12, the posture determination unit 13, and the avoidance target determination unit 14 according to the present invention. Vehicle device control means 15, relative position detection stage 16, and moving direction feature quantity calculation means 17. More specifically, STEPs 004 to 008 in FIG. 3 correspond to the object extraction unit 11, STEPs 011 to 016 correspond to the relative position detection stage 16, STEP017 corresponds to the movement direction feature amount calculation unit 17, and STEP018 includes It corresponds to the avoidance target determination means 14, and STEPs 019 to 020 correspond to the vehicle equipment control means 15. Furthermore, the process of extracting a pedestrian from the objects in STEP 106 of FIG. 5 corresponds to the pedestrian extraction means 12, and the process of determining the pedestrian posture in STEP 106 corresponds to the attitude determination means 13.

  Next, a second embodiment of the present invention will be described. Note that the present embodiment differs from the first embodiment only in the operations of the posture determination means 13 and the avoidance target determination means 14 that are functions of the image processing unit 1, and the vehicle periphery monitoring device in the present embodiment is different. The functional block diagram is the same as FIG. In the following description, the same components as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment, and the description thereof is omitted.

  In the vehicle periphery monitoring apparatus of the present embodiment, the posture determination means 13 that is a function of the image processing unit 1 is symmetrical in the left-right direction of the host vehicle 10 on the pedestrian image extracted by the pedestrian extraction means 12. Is a feature quantity representing the posture of the pedestrian, and the degree of symmetry is determined. In addition, as a method for calculating the symmetry in the left-right direction of the host vehicle 10 on the pedestrian image, for example, the pedestrian head region is set based on the luminance distribution on the grayscale image, and the head region A method is used in which the right region and the left region are set with reference to, and the symmetry is calculated by the correlation calculation between the right region and the left region. The configuration other than that described above is the same as that of the first embodiment.

  Next, the overall operation (object detection / alarm operation) of the vehicle periphery monitoring device of the present embodiment will be described. Note that the object detection / alarm operation of the vehicle periphery monitoring process in the present embodiment is only the first embodiment and the process of determining the approach collision by determining the pedestrian's posture in the avoidance determination process (STEP 106 in FIG. 5). Are different. Since the flowchart of the avoidance determination process in the present embodiment is the same as that in FIG. 5, the following description will be given with reference to the flowchart shown in FIG. In the following description, as in the first embodiment, the example shown in FIG. 4A (a pedestrian is present on the left side in front of the host vehicle 10 and another vehicle is present in front of the host vehicle 10). And pedestrians are trying to cross the road).

  In STEP 106, first, similarly to the first embodiment, the pedestrian extraction means 12 performs a process of extracting a pedestrian from the detected objects. Thereby, as illustrated in FIG. 4B, the object T1 surrounded by the frame R1 is extracted as a pedestrian from the objects T1 to T6. When no pedestrian is extracted, the determination result in STEP 106 is NO.

  Next, when a pedestrian is extracted, the posture determination means 13 performs processing for determining the posture of the extracted pedestrian T1. Hereinafter, a description will be given with reference to FIG. FIG. 8 illustrates a region corresponding to a region (including pedestrian T1) surrounded by the frame R1 illustrated in FIG. 4B on the grayscale image.

  First, the posture determination means 13 calculates the symmetry in the left-right direction of the host vehicle 10 on the image of the pedestrian T1. At this time, first, the posture determination means 13 sets the extracted head region TH1 of the pedestrian T1 on the gray scale image. Specifically, the posture determination means 13 sets a projection area in the region corresponding to the extracted pedestrian T1 on the gray scale image, and sets the vertical luminance projection (the luminance for each pixel in the vertical direction). The integrated luminance distribution in the horizontal direction) is calculated, and the horizontal coordinate at which the integrated luminance shows the maximum peak is detected. Then, the posture determination means 13 sets an area estimated to correspond to the head position of the pedestrian T1 with the detected lateral coordinate as the center, as shown in FIG. 8, as the head area TH1.

  Next, as shown in FIG. 8, the posture determination means 13 has a right region TR1 and a left region TL1 on the left and right sides of the lower side of the head region TH1 in the region surrounded by the frame R1. Set. The right region TR1 and the left region TL1 are respectively the same when the front-rear direction of the host vehicle 10 and the front-rear direction of the pedestrian are the same (when the traveling direction of the host vehicle 10 and the movement direction of the pedestrian are substantially the same), It is set to include the pedestrian's right shoulder and left shoulder. In the example shown in FIG. 8, the moving direction of the pedestrian T1 is the right side (center side of the lane), and the right region TR1 is a portion close to the center side of the lane.

  Next, the posture determination means 13 calculates the absolute difference sum of the luminance values indicating the degree of correlation between the image obtained by horizontally inverting the right region TR1 and the left region TL1. This absolute difference sum represents the degree of symmetry in the left-right direction of the host vehicle 10 on the image of the pedestrian T1.

  Next, the posture determination means 13 determines the degree of symmetry in the left-right direction of the host vehicle 10 on the image of the pedestrian T1 based on the calculated absolute difference sum. Specifically, in this embodiment, it is determined whether or not the degree of symmetry is low (whether or not the posture of the pedestrian T1 is a forward leaning posture). At this time, when the calculated absolute difference sum is equal to or greater than a predetermined value (the degree of correlation is low), it is determined that the degree of symmetry is low. In the example of FIG. 8, it is determined that the calculated absolute difference sum is equal to or greater than a predetermined value and the degree of symmetry is low. Thereby, it turns out that the pedestrian T1 is the forward leaning attitude to the center side or the opposite side of the lane.

  Further, when it is determined that the degree of symmetry is low, the posture determination unit 13 determines that the pedestrian T1 is leaning forward toward the center of the lane based on the luminance distribution in the right region TR1 and the left region TL1. It is determined whether or not there is. For example, the posture determination means 13 calculates the average value of the luminance in the regions TR1 and TL1, and if the average value in the central region of the lane is smaller than the average value in the opposite region, the pedestrian T1 is in the lane It is determined that the posture is forward tilted toward the center side. In the example of FIG. 8, the region on the center side of the lane is the right region TR1, and the calculated average value of the right region TR1 is smaller than the average value of the left region TL1, and the pedestrian T1 travels on the host vehicle 10. It is determined as a forward leaning posture toward the center side of the lane in which the vehicle is located. Thereby, it is predicted that pedestrian T1 is going to cross the road.

  Next, the avoidance target determination unit 14 determines whether or not the pedestrian T1 is an avoidance target based on the determination result of the posture of the pedestrian T1 by the posture determination unit 13. At this time, for example, when the pedestrian T1 is leaning forward toward the center of the lane in which the host vehicle 10 is traveling (YES in STEP 106), the pedestrian T1 may be trying to cross the road. Since it is predicted, it is determined in STEP 107 that the object is an avoidance target. Thereby, for example, when the pedestrian T1 existing in the entry determination areas AR2 and AR3 suddenly jumps out on the roadway, the alarm output determination and the alarm output in STEP019 to STEP020 in FIG. 3 are quickly performed. The operations other than those described above are the same as in the first embodiment.

  Through the above processing, according to the present embodiment, as in the first embodiment, the walking to be avoided among the pedestrians around the host vehicle 10 from the images acquired via the infrared cameras 2R and 2L. The person can be quickly determined and information can be presented to the driver of the host vehicle 10.

  In the first and second embodiments, the image processing unit 1 performs the approach collision determination process (STEP 105) based on the movement vector of the object and the pedestrian's posture as the approach collision determination process in the avoidance determination process of STEP018. However, as another embodiment, for example, only the approach collision determination process (STEP 106) based on the posture of the pedestrian is performed as the approach collision determination process. You may do it.

  In the first and second embodiments, the image processing unit 1 issues a warning to the driver of the host vehicle 10 for the object determined to be an avoidance object in STEP 018. As a form, for example, when the vehicle 10 can operate any one of the steering device, the brake device, and the accelerator device of the vehicle by an actuator (and consequently the traveling behavior of the vehicle 10 can be operated), the object to be avoided in STEP018 The steering device, the brake device, and the accelerator device of the host vehicle 10 may be controlled so as to avoid contact with the object determined to be or to facilitate the avoidance.

  For example, the accelerator device is controlled so that the required pedaling force of the accelerator pedal by the driver is greater than when there is no target object to be avoided (normal case), thereby making acceleration difficult. Alternatively, the required rotational force of the steering handle toward the steering direction of the steering device required to avoid contact between the avoidance target and the vehicle 10 is made lower than the required rotational force of the steering handle toward the opposite side, The steering handle can be easily operated in the steering direction. Or the increase speed of the braking force of the vehicle 10 according to the depression amount of the brake pedal of a brake device is made higher than usual. By doing in this way, the driving | operation of the vehicle 10 for avoiding a contact with an avoidance object becomes easy.

  When such a steering device, accelerator device, and brake device of the host vehicle 10 are controlled, these devices correspond to devices controlled by the vehicle device control means in the present invention. Further, the control of the steering device as described above and the alarm by the display device 7 or the speaker 6 may be performed in parallel.

  In the first and second embodiments, an infrared camera is used as the imaging means. However, for example, a CCD camera that can detect only normal visible light may be used. However, by using an infrared camera, it is possible to simplify the extraction process of pedestrians, running vehicles, and the like, and it can be realized even when the calculation capability of the calculation device is relatively low.

The functional block diagram of the vehicle periphery monitoring apparatus by 1st Embodiment of this invention. Explanatory drawing of the attachment aspect to the vehicle of the vehicle periphery monitoring apparatus shown in FIG. The flowchart which shows the target object detection and alarm operation | movement in the vehicle periphery monitoring apparatus of FIG. FIG. 4 is an exemplary diagram of a processed image in the object detection / alarm operation of FIG. 3. The flowchart of the avoidance determination process in the target object detection / alarm operation | movement of FIG. Explanatory drawing which shows the area division ahead of the vehicle in the avoidance determination process of FIG. Explanatory drawing regarding the process which discriminate | determines the attitude | position of a pedestrian in the avoidance determination process of FIG. Explanatory drawing regarding the process which discriminate | determines the attitude | position of a pedestrian in the avoidance determination process of 2nd Embodiment of this invention.

Explanation of symbols

  2R, 2L ... Infrared camera (imaging means), 10 ... vehicle, 11 ... object extraction means, 12 ... pedestrian extraction means, 13 ... posture discrimination means, 14 ... avoidance object judgment means, 15 ... vehicle equipment control means, 16 ... Relative position detection stage, 17 ... Movement direction feature amount calculation means.

Claims (7)

In a vehicle periphery monitoring device that monitors the periphery of the vehicle from an image acquired through an imaging means mounted on the vehicle,
An object extracting means for extracting an object existing around the vehicle from an image acquired via the imaging means;
Pedestrian extraction means for extracting pedestrians from the objects extracted by the object extraction means;
Posture determination means for determining a posture related to the inclination of the trunk axis of the pedestrian extracted by the pedestrian extraction means;
The object extracted by the object extraction unit should avoid contact with the vehicle by executing a determination algorithm including at least a first determination process related to the posture of the pedestrian determined by the position determination unit. Avoidance target determination means for determining whether or not the target is an avoidance target;
A vehicle periphery monitoring device comprising: vehicle equipment control means for controlling equipment of the vehicle according to at least a judgment result of the avoidance target judgment means.   2. The vehicle periphery monitoring device according to claim 1, wherein the device controlled by the vehicle device control means includes a device capable of issuing an alarm to a driver of the vehicle.   The vehicle periphery monitoring device according to claim 1 or 2, wherein the device controlled by the vehicle device control means includes a device capable of operating a running behavior of the vehicle. Relative position detection means for sequentially detecting the relative position of the object extracted by the object extraction means with respect to the vehicle;
The moving direction of the object relative to the vehicle is represented based on a time series of the relative position of the object detected by the relative position detection means in a time interval longer than the execution cycle of the determination process of the posture determination means. A moving direction feature amount calculating means for calculating a moving direction feature amount;
The determination algorithm executed by the avoidance target determination unit includes, together with the first determination process, a second determination process related to the movement direction feature amount of the target object calculated by the movement direction feature amount calculation unit. The process is a process for determining whether or not the posture of the pedestrian satisfies a predetermined first requirement, and the second determination process includes whether the moving direction feature amount of the object satisfies a predetermined second requirement. Is a process for determining whether or not
The avoidance target determination means includes a case where the determination result of the second determination process regarding the moving direction feature quantity of the object satisfies the second requirement, and the determination result of the second determination process does not satisfy the second requirement, And when the determination result of the 1st determination process regarding the said pedestrian's attitude | position satisfy | fills the said 1st requirement, it determines with the said target object being an avoidance object, The Claims 1-3 characterized by the above-mentioned. The vehicle periphery monitoring device according to any one of the above.   When the determination result of the second determination process related to the moving direction feature quantity of the object satisfies the second requirement, the determination result of the second determination process does not satisfy the second requirement. The vehicle periphery monitoring according to claim 4, wherein the vehicle device is controlled in a different form when the determination result of the first determination process related to the posture of the pedestrian satisfies the first requirement. apparatus. The posture determination means uses the symmetry of the pedestrian in the left-right direction of the vehicle on the image as a feature amount representing the posture of the trunk axis of the pedestrian, and determines the degree of symmetry. The vehicle periphery monitoring device according to any one of claims 1 to 5, wherein the vehicle periphery monitoring device is a means for performing the above. The posture discriminating means is a means for discriminating the degree of the inclination by using the inclination of the trunk axis of the pedestrian as a feature amount representing an attitude related to the inclination of the trunk axis of the pedestrian. The vehicle periphery monitoring device according to any one of claims 1 to 5.