JP6861599B2 - Peripheral monitoring device - Google Patents

Description

The present invention relates to a peripheral monitoring device that monitors the periphery of a vehicle.

Conventionally, a technique for realizing safer driving has been known by imaging the surroundings of the vehicle with an imaging unit provided around the vehicle and displaying a bird's-eye view image obtained by converting the captured image into a viewpoint to the driver. ing. However, it is not possible to judge a three-dimensional object from the bird's-eye view image. Therefore, there is known a technique of obtaining a difference image from at least two bird's-eye view images generated at different times and recognizing a three-dimensional object existing around the vehicle based on the difference image (see Patent Document 1). ..

However, when the bird's-eye view image is displayed to the driver, the three-dimensional object is displayed as if it collapses on the road surface. Therefore, among the three-dimensional objects existing around the vehicle, the floating objects that are not in contact with the road surface have a problem that they appear to be farther than the actual position due to the characteristics of the bird's-eye view image.

As a measure to solve this problem, one is a bird's-eye view image in which a three-dimensional object is imaged and the viewpoint is converted to the road surface, and a three-dimensional object is imaged and the viewpoint is converted to a horizontal plane higher than the installation position of the imaging unit. There is a technique for recognizing the position of a floating object by using a bird's-eye view image (see Patent Document 2). Another technique is to recognize the position of a floating object by comparing a plurality of bird's-eye view images imaged by a plurality of imaging units having different heights of installation positions and whose viewpoint is changed (see Patent Document 3).

Japanese Unexamined Patent Publication No. 2006-253872 Japanese Unexamined Patent Publication No. 2009-93332 Japanese Unexamined Patent Publication No. 2009-188635

However, with the techniques disclosed in Patent Documents 2 and 3, the position where the suspended matter is projected on the road surface directly below the suspended matter cannot be specified on the bird's-eye view image. Therefore, if the vehicle is brought close to the floating object by relying on the bird's-eye view image, the vehicle may come into contact with the floating object.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a peripheral monitoring device capable of suppressing the vehicle from coming into contact with floating objects existing in the vicinity of the vehicle.

The peripheral monitoring device according to the present invention has an imaging unit that is attached to a vehicle traveling on a road surface and images the periphery of the vehicle including floating objects that are not in contact with the road surface, and the peripheral image capturing unit. The bird's-eye view image generation unit that generates the first bird's-eye view image and the second bird's-eye view image by converting the viewpoints of the two images that can be changed according to the movement of the vehicle, and the first bird's-eye view image. The verticality of the floating object with respect to the road surface is evaluated from a plurality of three-dimensional object regions divided at predetermined angles about the position of the imaging unit by the first radiation extending in the radial direction from the imaging unit. A first three-dimensional object region extraction unit that extracts the first three-dimensional object region containing the suspended matter and a second radiation that extends the second bird's-eye view image in the radial direction from the imaging unit are predetermined with the position of the imaging unit as the center. A second three-dimensional object region extraction unit that evaluates the verticality of the suspended matter with respect to the road surface and extracts a second three-dimensional object region containing the suspended matter from a plurality of three-dimensional object regions divided at intervals. On the superimposed bird's-eye view image in which the first bird's-eye view image and the second bird's-eye view image are superimposed, a first straight line extending along the first radiation from the center of gravity of the floating object included in the first three-dimensional object region, and the first straight line. 2 Projection coordinate points at which the floating object is projected onto the road surface directly below the floating object at the intersection of the center of gravity of the suspended object included in the three-dimensional object region and the second straight line extending along the second radiation. It is characterized by having a projection coordinate point specifying portion specified as.

According to the peripheral monitoring device according to the present invention configured in this way, it is possible to prevent the vehicle from coming into contact with floating objects existing around the vehicle.

It is a block diagram which shows the schematic structure of the peripheral monitoring apparatus which concerns on one Embodiment of this invention. It is a hardware block diagram which shows the hardware composition of the peripheral monitoring apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the schematic structure of the projection coordinate point acquisition part which concerns on one Embodiment of this invention. It is explanatory drawing of the process performed in the 1st 3D object area extraction part (the 1). It is explanatory drawing of the process performed in the 1st three-dimensional object area extraction part (the 2). It is explanatory drawing of the process performed in the 2nd three-dimensional object area extraction part (the 1). It is explanatory drawing of the process performed in the 2nd three-dimensional object area extraction part (the 2). It is explanatory drawing of the process performed in the projection coordinate point identification part. It is explanatory drawing of the process performed in the projection coordinate point output part (the 1). It is explanatory drawing of the process performed in the projection coordinate point output part (the 2). It is explanatory drawing of the process performed in the projection coordinate point output part (the 3). It is explanatory drawing of the process performed in the projection coordinate point output part (the 4). It is explanatory drawing of the process performed in the projection coordinate point output part (the 5). It is explanatory drawing of the process performed in the projection coordinate point output part (the 6). It is explanatory drawing of the process performed in the projection coordinate point output part (the 7). It is explanatory drawing of the process performed in the projection coordinate point output part (the 8).

Hereinafter, the peripheral monitoring device according to the embodiment of the present invention will be described with reference to the drawings.

<Outline configuration of peripheral monitoring device>
FIG. 1 shows a schematic configuration of a peripheral monitoring device according to an embodiment of the present invention. The peripheral monitoring device 100 is mounted on the vehicle 10 traveling on the road surface. The peripheral monitoring device 100 mainly includes an imaging unit 12, an image input unit 20, a bird's-eye view image generation unit 30, a three-dimensional object detection unit 40, a projection coordinate point acquisition unit 50, and a projection coordinate point output unit 60. I have.

The imaging unit 12 is attached to the left side of the vehicle 10. The imaging unit 12 images the left observation range including the nearest road surface of the vehicle 10 over a viewing range (angle of view) of 180 °.

The image input unit 20 is a digital image format source that can handle two images obtained by being imaged by the image pickup unit 12 at two times (t−Δt) and (t) having different left observation ranges by a computer. It is converted into an image 70 (t−Δt) and an original image 70 (t). The image input unit 20 inputs the original image 70 (t−Δt) and the original image 70 (t) to the bird's-eye view image generation unit 30.

The bird's-eye view image generation unit 30 converts the original image 70 (t−Δt) and the original image 70 (t) into viewpoints, and views the first bird's-eye view image 72 (t−Δt) and the second bird's-eye view image viewed from a predetermined viewpoint position. Generate 72 (t).

The details of the three-dimensional object detection unit 40, the projection coordinate point acquisition unit 50, and the projection coordinate point output unit 60 will be described later.

<Hardware configuration>
FIG. 2 is a hardware block diagram showing a hardware configuration of a peripheral monitoring device according to an embodiment of the present invention. Hereinafter, the hardware configuration will be described with reference to FIG.

The peripheral monitoring device 100 includes an ECU (Electronic Control Unit) 110 mounted on the vehicle 10, a left camera 12a, a vehicle state sensor 140, and a monitor 150 (display unit).

The left camera 12a constitutes an imaging unit 12 (FIG. 1).

The vehicle condition sensor 140 is composed of a steering angle sensor and a distance sensor. The vehicle state sensor 140 calculates the movement amount and the movement direction of the vehicle 10 by detecting the behavior of the vehicle 10.

The monitor 150 displays the output result of the projection coordinate point output unit 60 (FIG. 1).

The ECU 110 includes a CPU (Central Processing Unit) 112, a camera interface 114, a sensor input interface 116, an image processing module 118, a memory 120, and a display control unit 122.

The CPU 112 transmits / receives necessary data and executes a program.

The camera interface 114 is connected to the CPU 112 via the bus 124. The camera interface 114 controls the left camera 12a.

The sensor input interface 116 is connected to the CPU 112 via the bus 124. The sensor input interface 116 acquires the measurement result of the vehicle condition sensor 140.

The image processing module 118 is connected to the CPU 112 via the bus 124. The image processing module 118 executes image processing by a predetermined program built in the module 118.

The memory 120 is connected to the CPU 112 via the bus 124. The memory 120 stores intermediate results of image processing, necessary constants, programs, and the like.

The display control unit 122 constitutes the projection coordinate point output unit 60 (FIG. 1). The display control unit 122 is connected to the CPU 112 via the bus 124. The display control unit 122 controls the monitor 150.

The image input unit 20, the bird's-eye view image generation unit 30, the three-dimensional object detection unit 40, the projection coordinate point acquisition unit 50, and the projection coordinate point output unit 60 described with reference to FIG. 1 are controlled by software that realizes an operation described later. There is. This software is stored inside the memory 120 (FIG. 2) and is appropriately executed as needed.

<Approximate configuration of the projected coordinate point acquisition unit>
FIG. 3 shows a schematic configuration of a projected coordinate point acquisition unit according to an embodiment of the present invention. Hereinafter, a schematic configuration of the projected coordinate point acquisition unit will be described with reference to FIG.

The projection coordinate point acquisition unit 50 includes a first three-dimensional object area extraction unit 51, a second three-dimensional object area extraction unit 52, and a projection coordinate point identification unit 53.

The first three-dimensional object region extraction unit 51 includes a region division unit 51A and a verticality evaluation unit 51B.

The region dividing portion 51A uses the first radiation extending in the radial direction from the left camera 12a (FIG. 4) to the first bird's-eye view image 72 (t−Δt) shown in FIG. 4 at a predetermined angle about the position of the left camera 12a. Divide every other.

The verticality evaluation unit 51B evaluates the verticality of the bars 85 and 86 (FIG. 4) included in the specific three-dimensional object region with respect to the road surface 80 (FIG. 4) among the plurality of three-dimensional object regions divided by the region division unit 51A. To do. Of course, the evaluation of the straightness is not limited to the bars 85 and 86, and may be performed on the columns 81, 82, 83, 84 (FIG. 4).

The second three-dimensional object region extraction unit 52 includes a region division unit 52A and a verticality evaluation unit 52B.

The region dividing portion 52A uses the second bird's-eye view image 72 (t) shown in FIG. 6 as the second radiation extending in the radial direction from the left camera 12a (FIG. 6) at predetermined angles about the position of the left camera 12a. To divide.

The verticality evaluation unit 52B evaluates the verticality of the bars 85 and 86 (FIG. 6) included in the specific three-dimensional object region with respect to the road surface 80 (FIG. 6) among the plurality of three-dimensional object regions divided by the region division unit 52A. To do.

The details of the projected coordinate point specifying unit 53 will be described later.

<Explanation of each process performed by the peripheral monitoring device>
Hereinafter, the contents of each process performed by the peripheral monitoring device 100 will be described step by step.

<Explanation of processing performed by the three-dimensional object detection unit 40>
First, the process performed by the three-dimensional object detection unit 40 will be described with reference to FIGS. 4 and 6.

The first bird's-eye view image 72 (t−Δt) shown in FIG. 4 is a bird's-eye view in which the left observation range including the nearest road surface 80 of the vehicle 10 is imaged by the left camera 12a at the time (t−Δt) and the viewpoint is changed. It is an image. The vehicle 10 reflected in the lower part of the first bird's-eye view image 72 (t−Δt) is, for example, a vehicle created by CG (Computer Graphics).

The second bird's-eye view image 72 (t) shown in FIG. 6 is a bird's-eye view image obtained by capturing the left observation range including the nearest road surface 80 of the vehicle 10 at the time (t) by the left camera 12a and changing the viewpoint. The vehicle 10 reflected in the lower part of the second bird's-eye view image 72 (t) is, for example, a vehicle created by CG.

As shown in FIGS. 4 and 6, four columns 81, 82, 83, and 84 are erected on the road surface 80. A bar 85 as a linear body (floating matter) is erected between the tips of the columns 81 and 82. A bar 86 as a linear body (floating matter) is erected between the tips of the columns 83 and 84.

As shown in FIGS. 4 and 6, water pools P are connected to the foot portions 83a and 84a of the columns 83 and 84 by the water collected on the road surface 80. As shown in FIGS. 4 and 6, the light from the columns 83 and 84 is mirror-reflected on the surface of the puddle P formed on the road surface 80.

Therefore, the bird's-eye view image in which the columns 83 and 84 are imaged and the viewpoint is converted is integrated with the bird's-eye view image in which the columns 87 and 88 reflected on the surface of the puddle P are imaged and the viewpoint is converted. As a result, the two columns 83, 84 erected on the road surface 80 are reflected so as to be closer to the vehicle 10 than the actual positions, as shown in FIGS. 4 and 6.

The three-dimensional object detection unit 40 translates or rotates the first bird's-eye view image 72 (t−Δt) shown in FIG. 4 so as to correspond to the movement amount and the movement direction of the vehicle 10 during the short time interval Δt. , Generates a virtual second predicted bird's-eye view image 72'(t) at time (t). FIG. 12 shows an example of the second predicted bird's-eye view image 72'(t).

Specifically, the three-dimensional object detection unit 40 includes columns 81 to 84 and bars 85 and 86 depicted on the road surface 80 reflected in the second bird's-eye view image 72 (t) shown in FIG. 6, and the third object detection unit 40 shown in FIG. 2 Based on the columns 81 to 84 and bars 85 and 86 drawn on the road surface 80 reflected in the predicted bird's-eye view image 72'(t), the columns 81 to 84 and bars 85 and 86 respond to the movement of the vehicle 10. Calculate the amount of movement and the amount of rotation. Based on the amount of movement and the amount of rotation, the three-dimensional object detection unit 40 includes columns 81 to 84 and bars 85 and 86 reflected in the second bird's-eye view image 72 (t), and the second predicted bird's-eye view image 72'(t). Align with the columns 81 to 84 and the bars 85 and 86 reflected in.

After the alignment, the three-dimensional object detection unit 40 performs a subtraction process of subtracting the second bird's-eye view image 72 (t) shown in FIG. 6 from the second predicted bird's-eye view image 72'(t) shown in FIG. Ask for an image.

Even if the vehicle 10 moves, the foot portion 83a of the support column 83 and the foot portion 84a of the support column 84 are generated at substantially the same positions of the second predicted bird's-eye view image 72'(t) and the second bird's-eye view image 72 (t). .. Therefore, it is presumed that the portions removed in the difference image between the second predicted bird's-eye view image 72'(t) and the second bird's-eye view image 72 (t) are the foot portion 83a of the support column 83 and the foot portion 84a of the support column 84. can do. Therefore, the foot portion 83a of the support column 83 integrated with the support column 87 reflected on the surface of the water pool P on the bird's-eye view image, and the support column 84 integrated with the support column 88 reflected on the surface of the water pool P on the bird's-eye view image. The position of the foot portion 84a can be extracted.

In extracting the positions of the foot portion 83a of the support column 83 and the foot portion 84a of the support column 84, the three-dimensional object detection unit 40 uses the virtual first predicted bird's-eye view image 72'(t-) at the time (t−Δt). Δt) may be generated. FIG. 10 shows an example of the first predicted bird's-eye view image 72'(t−Δt).

Specifically, the three-dimensional object detection unit 40 includes columns 81 to 84 and bars 85 and 86 depicted on the road surface 80 reflected in the first bird's-eye view image 72 (t−Δt) shown in FIG. Based on the columns 81 to 84 and bars 85 and 86 drawn on the road surface 80 reflected in the first predicted bird's-eye view image 72'(t-Δt), the columns 81 to 84 and bars 85 and 86 are vehicles 10. The amount of movement and the amount of rotation are calculated according to the movement of. Based on the amount of movement and the amount of rotation, the three-dimensional object detection unit 40 includes columns 81 to 84 and bars 85 and 86 reflected in the first bird's-eye view image 72 (t-Δt), and the first predicted bird's-eye view image 72'(. Alignment with the columns 81 to 84 and bars 85 and 86 reflected in t−Δt) is performed.

After the alignment, the three-dimensional object detection unit 40 subtracts the first bird's-eye view image 72 (t−Δt) shown in FIG. 4 from the first predicted bird's-eye view image 72 ′ (t−Δt) shown in FIG. To obtain the difference image. The portion removed in the difference image is also the same as the portion removed in the difference image between the second predicted bird's-eye view image 72'(t) shown in FIG. 10 and the second bird's-eye view image 72 (t) shown in FIG. , It can be inferred that it is the foot portion 83a of the support column 83 or the foot portion 84a of the support column 84.

The brightness of the difference image obtained as a result of the subtraction process performed between the predicted bird's-eye view image and the actually obtained bird's-eye view image as described above becomes smaller than the threshold value in the region where the road surface 80 exists, and the support column It tends to be larger than the threshold value in the region where three-dimensional objects such as 81 to 84 and bars 85 and 86 exist. Utilizing this tendency, the three-dimensional object detection unit 40 detects a region where the brightness of the difference image is larger than the threshold value as a three-dimensional object existence region.

<Explanation of processing performed by the first three-dimensional object region extraction unit 51>
Next, the process performed by the first three-dimensional object region extraction unit 51 will be described with reference to FIGS. 4 and 5.

(Explanation of processing performed in the area division section 51A)
First, as shown in FIG. 4, the region dividing portion 51A extends the first bird's-eye view image 72 (t−Δt) from the left camera 12a in the radial direction of the seven first radiations RL10, RL11, RL12, RL13, RL14. , RL15, RL16, RL17 are divided at equal angles (= 30 °) around the position of the left camera 12a.

Due to the first radiation RL10 to RL17, the first bird's-eye view image 72 (t−Δt) becomes six three-dimensional object regions 72 (t−Δt) _1,72 (t−Δt) _2,72 (t−Δt) as shown in FIG. It is divided into t-Δt) _3,72 (t-Δt) _4,72 (t-Δt) _5,72 (t-Δt) _6.

The first radiations RL10 to RL17 are indicators of the verticality of the columns 81 to 84 and the bars 85 and 86 with respect to the road surface 80. Since the stanchion is erected vertically with respect to the road surface 80, the longitudinal direction of the stanchion and the extension direction of the first radiation coincide with each other.

(Explanation of processing performed by verticality evaluation unit 51B)
As shown in FIG. 5, the verticality evaluation unit 51B evaluates the verticality of the columns 81 to 84 and the bars 85 and 86 with respect to the road surface 80. The verticality evaluation unit 51B distinguishes the columns 81 and 82 from the bars 85 from the mass region consisting of the columns 81 and 82 and the bars 85. Similarly, the verticality evaluation unit 51B includes the lumps of the struts 83 and 87, the struts 84, and the struts 84 from the region of the lumps consisting of the struts 83, 84, the bars 86, and the struts 87, 88 reflected on the surface of the puddle P. Distinguish between the mass of stanchions 88 and the bar 86.

As a method for distinguishing columns and bars, for example, a block area is divided into a plurality of blocks, edge detection is performed for each block in the vertical direction or the horizontal direction, and the edge direction similarity obtained as a result of the edge detection is performed. There is a method using sex and so on. In the following, the verticality evaluation of the bars 85 and 86 with respect to the road surface 80 and the verticality evaluation of the columns 81 to 84 with respect to the road surface 80 will be described separately.

(Evaluation of verticality of bars 85 and 86 with respect to road surface 80)
As shown in FIG. 5, the verticality evaluation unit 51B determines whether or not the angle formed by the vector V1 in the longitudinal direction of the bar 85 and the vector in the radial direction of the left camera 12a exceeds a predetermined reference angle. To judge. The verticality evaluation unit 51B determines that the bar 85 is not erected vertically with respect to the road surface 80 when the angle formed by both vectors exceeds the reference angle. Based on this determination, the first three-dimensional object region extraction unit 51 (FIG. 3) extracts the three-dimensional object region 72 (t−Δt) _1,72 (t−Δt) _2 including the bar 85. The bar 85 is a target for obtaining the center of gravity G1 (FIG. 8) in the projected coordinate point specifying unit 53 (FIG. 3) described later.

As shown in FIG. 5, the verticality evaluation unit 51B determines whether or not the angle formed by the vector V2 in the longitudinal direction of the bar 86 and the vector in the radial direction of the left camera 12a exceeds a predetermined reference angle. To judge. The verticality evaluation unit 51B determines that the bar 86 is not erected vertically with respect to the road surface 80 when the angle formed by both vectors exceeds the reference angle. Based on this determination, the first three-dimensional object region extraction unit 51 (FIG. 3) extracts the three-dimensional object regions 72 (t−Δt) _5 and 72 (t−Δt) _6 including the bar 86. The bar 86 is a target when the center of gravity G21 (FIG. 8) is obtained in the projected coordinate point specifying unit 53 (FIG. 3) described later.

(Evaluation of verticality of columns 81 to 84 with respect to the road surface 80)
As shown in FIG. 5, the verticality evaluation unit 51B evaluates the verticality of the columns 81 to 84 with respect to the road surface 80 in the same manner as the bars 85 and 86. The columns 81 to 84 of this embodiment are erected vertically with respect to the road surface 80. Therefore, the direction of the vector V3 in the longitudinal direction of the support column 81 coincides with the radiation direction of the left camera 12a.

The direction of the vector V4 in the longitudinal direction of the support column 82 and the radiation direction of the left camera 12a also coincide with each other. The direction of the vector V5 in the longitudinal direction of the support column 87 reflected on the surfaces of the support column 83 and the puddle P and the radiation direction of the left camera 12a are also the same. The direction of the vector V6 in the longitudinal direction of the support column 88 reflected on the surfaces of the support column 84 and the puddle P and the radiation direction of the left camera 12a are also the same.

In FIG. 5, it is determined that the acute angle formed by the vector in the longitudinal direction of the column and the vector in the extending direction of the first radiation is within the reference angle. Therefore, the columns 81 to 84 are excluded from the target for which the center of gravity is obtained in the projected coordinate point specifying unit 53 (FIG. 3), which will be described later.

<Explanation of processing performed by the second three-dimensional object region extraction unit 52>
Next, the process performed by the second three-dimensional object region extraction unit 52 will be described with reference to FIGS. 6 and 7. The region division unit 51A and the verticality evaluation unit 51B of the first three-dimensional object region extraction unit 51 and the region division unit 52A and the verticality evaluation unit 52B of the second three-dimensional object region extraction unit 52 are the same in order. Therefore, the description may be omitted.

(Explanation of processing performed by the area dividing unit 52A)
First, as shown in FIG. 6, the region dividing unit 52A extends the second bird's-eye view image 72 (t) from the left camera 12a in the radial direction to seven second radiations RL20, RL21, RL22, RL23, RL24, RL25. , RL26, RL27 are divided at equal angles θ2 (= 30 °) around the position of the left camera 12a. As a result, the second bird's-eye view image 72 (t) has six three-dimensional object regions 72 (t) _1,72 (t) _2,72 (t) _3,72 (t) _4,72 (t) _5,72 ( t) Divided into _6.

(Explanation of processing performed by verticality evaluation unit 52B)
Next, the verticality evaluation unit 52B evaluates the verticality of the columns 81 to 84 and the bars 85 and 86 with respect to the road surface 80. In the following, the verticality evaluation of the bars 85 and 86 with respect to the road surface 80 and the verticality evaluation of the columns 81 to 84 with respect to the road surface 80 will be described separately.

(Vertical evaluation of bars 85 and 86 for road surface 80)
As shown in FIG. 7, the verticality evaluation unit 52B determines whether or not the angle formed by the vector V21 in the longitudinal direction of the bar 85 and the vector in the radial direction of the left camera 12a exceeds a predetermined reference angle. To judge. In FIG. 7, assuming that the angle formed by both vectors exceeds the reference angle, the second three-dimensional object region extraction unit 52 extracts the three-dimensional object regions 72 (t) _1 and 72 (t) _2 including the bar 85. The bar 85 is a target for obtaining the center of gravity G2 (FIG. 8) in the projected coordinate point specifying unit 53 (FIG. 3) described later.

As shown in FIG. 7, the verticality evaluation unit 52B determines whether or not the angle formed by the vector V22 in the longitudinal direction of the bar 86 and the vector in the radial direction of the left camera 12a exceeds a predetermined reference angle. To judge. In FIG. 7, assuming that the angle formed by both vectors exceeds the reference angle, the second three-dimensional object region extraction unit 52 (FIG. 3) sets the three-dimensional object regions 72 (t) _4 to 72 (t) _6 including the bar 86. Extract. The bar 86 is a target for obtaining the center of gravity G22 (FIG. 8) in the projected coordinate point specifying unit 53 (FIG. 3) described later.

(Evaluation of verticality of columns 81 to 84 with respect to the road surface 80)
As shown in FIG. 7, the verticality evaluation unit 52B evaluates the verticality of the columns 81 to 84 with respect to the road surface 80, similarly to the bars 85 and 86. As described above, the columns 81 to 84 are erected vertically with respect to the road surface 80. Therefore, the direction of the vector V23 in the longitudinal direction of the support column 81 coincides with the radiation direction of the left camera 12a.

The orientation of the vector V24 in the longitudinal direction of the support column 82 and the radiation direction of the left camera 12a also coincide with each other. The direction of the vector V25 in the longitudinal direction of the support column 87 reflected on the surfaces of the support column 83 and the puddle P and the radiation direction of the left camera 12a are also the same. The direction of the vector V26 in the longitudinal direction of the support column 88 reflected on the surfaces of the support column 84 and the puddle P and the radiation direction of the left camera 12a are also the same.

In FIG. 7, it is determined that the acute angle formed by the vector in the longitudinal direction of the column and the vector in the extending direction of the first radiation is within the reference angle. Therefore, the columns 81 to 84 are excluded from the target for which the center of gravity is obtained in the projected coordinate point specifying unit 53 (FIG. 3), which will be described later.

<Explanation of processing performed by the projected coordinate point identification unit 53>
Subsequently, the process performed by the projected coordinate point specifying unit 53 will be described with reference to FIG.

The superimposed bird's-eye view image SVI shown in FIG. 8 is obtained by moving and rotating the first bird's-eye view image 72 (t−Δt) shown in FIG. 4 and the second bird's-eye view image 72 (t) shown in FIG. It is a superposed bird's-eye view image superimposed after aligning the positions. The vehicle 10 shown by a chain line at the lower part of the superimposed bird's-eye view image SVI is reflected at the same position as the lower part of the first bird's-eye view image 72 (t−Δt) shown in FIG. The vehicle 10 shown by a solid line at the lower part of the superimposed bird's-eye view image SVI is reflected at the same position as the lower part of the second bird's-eye view image 72 (t) shown in FIG.

In FIG. 8, in order to make the figure easier to understand, the columns 81 to 84 and the bars 85 and 86 included in the first bird's-eye view image 72 (t−Δt) are shown by diagonal lines and are included in the second bird's-eye view image 72 (t). The columns 81 to 84 and the bars 85 and 86 are shaded. Further, in FIG. 8, in order to make the figure easier to understand, the columns 87 and 88 reflected on the surface of the puddle P included in the first bird's-eye view image 72 (t−Δt) and the second bird's-eye view image 72 (t). The type of hatching is changed with the columns 87 and 88 reflected on the surface of the puddle P contained in.

As shown in FIG. 8, the projected coordinate point specifying unit 53 sets the bar 85 on the road surface 80 directly below the bar 85 at the point where the first straight line L1 and the first straight line L21 intersect on the superimposed bird's-eye view image SVI. It is specified as the projected projected coordinate point SP1. The first straight line L1 is a straight line extending from the center of gravity G1 of the bar 85 indicated by the diagonal line in the radial direction of the left camera 12a installed in the vehicle 10 indicated by the chain line. The first straight line L21 is a straight line extending in the radial direction of the left camera 12a installed in the vehicle 10 shown by the solid line from the center of gravity G2 of the bar 85 shown by shading.

As shown in FIG. 8, the projected coordinate point specifying unit 53 sets the bar 86 on the road surface 80 directly below the bar 86 at the point where the second straight line L2 and the second straight line L22 intersect on the superimposed bird's-eye view image SVI. It is specified as the projected projected coordinate point SP2. The second straight line L2 is a straight line extending from the center of gravity G21 of the bar 86 indicated by the diagonal line in the radial direction of the left camera 12a installed in the vehicle 10 indicated by the chain line. The second straight line L22 is a straight line extending in the radial direction of the left camera 12a installed in the vehicle 10 shown by the solid line from the center of gravity G22 of the bar 86 shown by shading.

<Explanation of processing performed by the projected coordinate point output unit 60>
Subsequently, the processing performed by the projected coordinate point output unit 60 will be described with reference to FIGS. 9 to 16.

(Vertical judgment processing using the bird's-eye view image and the first predicted bird's-eye view image)
First, as shown in FIGS. 9 and 10, the projected coordinate point output unit 60 is generated by translating or rotating the second bird's-eye view image 72 (t) and the second bird's-eye view image 72 (t). 1 Using the predicted bird's-eye view image 72'(t−Δt), the verticality of a three-dimensional object included in the three-dimensional object region including the projected coordinate points with respect to the road surface 80 is determined.

In the following, the verticality determination process for the three-dimensional object region including the projected coordinate point SP1 and the verticality determination process for the three-dimensional object region including the projected coordinate point SP2 will be described separately.

(Verticality determination process for a three-dimensional object region including the projected coordinate point SP1)
First, the verticality determination process for the three-dimensional object region including the projected coordinate point SP1 will be described with reference to FIGS. 9 and 10.

As shown in FIG. 9, the projection coordinate point output unit 60 obtains a three-dimensional object region 72 (t) _1 including the projection coordinate point SP1 specified by the projection coordinate point identification unit 53 from the second bird's-eye view image 72 (t). Extract. As shown in FIG. 9, the projected coordinate point output unit 60 is a composite vector of the vector V31 in the longitudinal direction of the bar 85 and the vector V32 in the longitudinal direction of the support column 81 included in the three-dimensional object region 72 (t) _1. Calculate SV1.

As shown in FIG. 10, the projected coordinate point output unit 60 has the projected coordinate point SP1 corresponding to the projected coordinate point SP1 specified by the projected coordinate point specifying unit 53 from the first predicted bird's-eye view image 72'(t−Δt). The three-dimensional object region 72'(t-Δt) _2 including'is extracted. As shown in FIG. 10, the three-dimensional object region 72'(t-Δt) _2 includes only the vector V31'in the longitudinal direction of the bar 85. Therefore, the projected coordinate point output unit 60 does not perform the calculation of the composite vector.

(First judgment process)
Next, as shown in FIG. 9, the projected coordinate point output unit 60 determines whether or not the angle formed by the composite vector SV1 and the vector in the radial direction of the left camera 12a is within a predetermined reference angle. The first determination process for determination is performed.

(Second judgment process)
Subsequently, as shown in FIGS. 9 and 10, the projected coordinate point output unit 60 determines whether or not the angle formed by the composite vector SV1 and the vector V31'is within a predetermined reference angle. 2 Perform the judgment process.

The projection coordinate point output unit 60 extracts the projection coordinate point SP1 from the second bird's-eye view image 72 (t) as a target to be displayed on the monitor 150 only when the determination conditions of both the first determination process and the second determination process are satisfied. ..

(Verticality determination process for a three-dimensional object region including the projected coordinate point SP2)
Next, the verticality determination process for the three-dimensional object region including the projected coordinate point SP2 will be described with reference to FIGS. 9 and 10.

As shown in FIG. 9, the projection coordinate point output unit 60 obtains a three-dimensional object region 72 (t) _5 including the projection coordinate point SP2 specified by the projection coordinate point identification unit 53 from the second bird's-eye view image 72 (t). Extract. As shown in FIG. 9, the three-dimensional object region 72 (t) _5 includes only the vector V33 in the longitudinal direction of the bar 86. Therefore, the projected coordinate point output unit 60 does not perform the calculation of the composite vector.

As shown in FIG. 10, the projected coordinate point output unit 60 has the projected coordinate point SP2 corresponding to the projected coordinate point SP2 specified by the projected coordinate point specifying unit 53 from the first predicted bird's-eye view image 72'(t−Δt). The three-dimensional object region 72'(t-Δt) _6 including'is extracted. The projected coordinate point output unit 60 is included in the vector V33'in the longitudinal direction of the bar 86 included in the three-dimensional object region 72'(t-Δt) _6 and in the three-dimensional object region 72'(t-Δt) _6. , The composite vector SV2'with the longitudinal vector V34'of the mass of the stanchions 88 reflected on the surfaces of the stanchions 84 and the puddle P is calculated.

(Third judgment process)
Next, as shown in FIG. 9, the projected coordinate point output unit 60 determines whether or not the angle formed by the vector V33 and the vector in the radial direction of the left camera 12a is within a predetermined reference angle. The third determination process is performed.

(4th judgment process)
Subsequently, as shown in FIGS. 9 and 10, the projected coordinate point output unit 60 determines whether or not the angle formed by the vector V33 and the composite vector SV2'is within a predetermined reference angle. 4 Perform the judgment process.

The projection coordinate point output unit 60 extracts the projection coordinate point SP2 from the second bird's-eye view image 72 (t) as a target to be displayed on the monitor 150 only when the determination conditions of both the third determination process and the fourth determination process are satisfied. ..

(Vertical judgment processing using a bird's-eye view image and a predicted bird's-eye view image)
Next, as shown in FIGS. 11 and 12, the projected coordinate point output unit 60 translates or rotates the first bird's-eye view image 72 (t−Δt) and the first bird's-eye view image 72 (t−Δt). Using the second predicted bird's-eye view image 72'(t) generated in the above procedure, the verticality of the three-dimensional object included in the three-dimensional object region including the projected coordinate points with respect to the road surface 80 is determined. In the following, the verticality determination process for the three-dimensional object region including the projected coordinate point SP1 and the verticality determination process for the three-dimensional object region including the projected coordinate point SP2 will be described separately.

(Verticality determination process for a three-dimensional object region including the projected coordinate point SP1)
First, the verticality determination process for the three-dimensional object region including the projected coordinate point SP1 will be described with reference to FIGS. 11 and 12.

As shown in FIG. 11, the projected coordinate point output unit 60 includes a three-dimensional object region 72 (t−t) including the projected coordinate point SP1 specified by the projected coordinate point specifying unit 53 from the first bird's-eye view image 72 (t−Δt). Δt) _2 is extracted. As shown in FIG. 11, the three-dimensional object region 72 (t−Δt) _2 includes only the vector V35 in the longitudinal direction of the bar 85. Therefore, the projected coordinate point output unit 60 does not perform the calculation of the composite vector.

As shown in FIG. 12, the projected coordinate point output unit 60 obtains the projected coordinate point SP1'corresponding to the projected coordinate point SP1 specified by the projected coordinate point specifying unit 53 from the second predicted bird's-eye view image 72'(t). The three-dimensional object region 72'(t) _1 to be included is extracted. As shown in FIG. 12, the projected coordinate point output unit 60 includes the vector V35'in the longitudinal direction of the bar 85 included in the three-dimensional object region 72'(t) _1 and the three-dimensional object region 72'(t) _1. The composite vector SV3'with the vector V36'in the longitudinal direction of the support column 81 included in the above is calculated.

(Fifth judgment process)
Next, as shown in FIG. 11, the projected coordinate point output unit 60 determines whether or not the angle formed by the vector V35 and the vector in the radial direction of the left camera 12a is within a predetermined reference angle. The fifth determination process is performed.

(6th judgment process)
Subsequently, as shown in FIGS. 11 and 12, the projected coordinate point output unit 60 determines whether or not the angle formed by the vector V35 and the composite vector SV3'is within a predetermined reference angle. 6 Judgment processing is performed.

The projection coordinate point output unit 60 sets the projection coordinate point SP1 from the first bird's-eye view image 72 (t−Δt) as a target to be displayed on the monitor 150 only when the determination conditions of both the fifth determination process and the sixth determination process are satisfied. Extract.

(Verticality determination process for a three-dimensional object region including the projected coordinate point SP2)
Next, the verticality determination process for the three-dimensional object region including the projected coordinate point SP2 will be described with reference to FIGS. 11 and 12.

As shown in FIG. 11, the projected coordinate point output unit 60 obtains a three-dimensional object region 72 (t) _6 including the projected coordinate point SP2 specified by the projected coordinate point specifying unit 53 from the second bird's-eye view image 72 (t). Extract. As shown in FIG. 11, the projected coordinate point output unit 60 is included in the vector V37 in the longitudinal direction of the bar 86 included in the three-dimensional object region 72 (t) _6 and in the three-dimensional object region 72 (t) _4. , The composite vector SV4 of the support column 83 and the vector V38 in the longitudinal direction of the mass of the support column 87 reflected on the surface of the water pool P is calculated.

As shown in FIG. 12, the projected coordinate point output unit 60 has the projected coordinate point SP2 corresponding to the projected coordinate point SP2 specified by the projected coordinate point specifying unit 53 from the first predicted bird's-eye view image 72'(t−Δt). The three-dimensional object region 72'(t-Δt) _5 including'is extracted. The three-dimensional object region 72'(t-Δt) _5 includes only the longitudinal vector V37'of the bar 86. Therefore, the projected coordinate point output unit 60 does not perform the calculation of the composite vector.

(7th judgment process)
Next, as shown in FIG. 11, the projected coordinate point output unit 60 determines whether or not the angle formed by the composite vector SV4 and the vector in the radial direction of the left camera 12a is within a predetermined reference angle. The seventh determination process for determination is performed.

(8th judgment process)
Subsequently, as shown in FIGS. 11 and 12, the projected coordinate point output unit 60 determines whether or not the angle formed by the composite vector SV4 and the vector V37'is within a predetermined reference angle. 8 Judgment processing is performed.

The projection coordinate point output unit 60 sets the projection coordinate point SP2 from the first bird's-eye view image 72 (t−Δt) as a target to be displayed on the monitor 150 only when the determination conditions of both the seventh determination process and the eighth determination process are satisfied. Extract.

(Display control processing using a bird's-eye view image and a predicted bird's-eye view image)
Subsequently, the display control process performed by the projected coordinate point output unit 60 will be described with reference to FIGS. 13 to 16.

First, as shown in FIGS. 13 and 14, the projected coordinate point output unit 60 translates or rotates the first bird's-eye view image 72 (t−Δt) and the first bird's-eye view image 72 (t−Δt). The display control process on the monitor 150 is performed using the second predicted bird's-eye view image 72'(t) generated in the above.

The first bird's-eye view image 72 (t−Δt) shown in FIG. 13 is displayed on the monitor 150. The coordinate points SP1 and SP2 in the first bird's-eye view image 72 (t−Δt) are the road surfaces 80 directly below the bars 85 and 86, which are the targets to be displayed on the monitor 150 as a result of the first to seventh determination processes. It is a projected coordinate point on which bars 85 and 86 are projected. The coordinate point SP3 is a coordinate point indicating the position of the foot portion of the support column 81. The coordinate point SP4 is a coordinate point indicating the position of the foot portion of the support column 82.

The coordinate point SP5 is a coordinate point indicating the position of the center of gravity of the bar 85. The coordinate point SP6 is a coordinate point indicating the position of the foot portion of the support column 83. The coordinate point SP7 is a coordinate point indicating the position of the foot portion of the support column 84. The coordinate point SP8 is a coordinate point indicating the position of the center of gravity of the bar 86. The coordinate point SP9 is a coordinate point indicating the position of the tip of the support column 87 reflected on the surface of the puddle P. The coordinate point SP10 is a coordinate point indicating the position of the tip of the support column 88 reflected on the surface of the puddle P. For example, the coordinate point SP6 and the coordinate point SP7 are obtained as coordinate points at the positions of the portions removed in the difference image between the second predicted bird's-eye view image 72'(t) and the second bird's-eye view image 72 (t) described above. In FIG. 13, the coordinate points SP1 to the coordinate points SP10 are indicated by white stars.

The coordinate points SP1'and SP2' reflected in the second predicted bird's-eye view image 72'(t) shown in FIG. 14 are the coordinates corresponding to the coordinate points SP1 and SP2 reflected in the first bird's-eye view image 72 (t−Δt). It is a point. Similarly, the coordinate points SP3', SP4', SP5', SP6', SP7', SP8', SP9', and SP10'are also coordinate points SP3, SP4, SP5, SP6, SP7, SP8, SP9, SP10, respectively. It corresponds to. In FIG. 14, the coordinate points SP1'to the coordinate points SP10' are indicated by black stars.

Next, as shown in FIG. 15, the projected coordinate point output unit 60 performs a comparison determination between the distance between the coordinate points and the threshold value. FIG. 15 shows that the first bird's-eye view image 72 (t−Δt) and the second predicted bird's-eye view image 72'(t) are moved and rotated to align the positions of both bird's-eye view images and then superimposed on the superimposed bird's-eye view image. A state in which only the coordinate points SP1 to SP10 and the coordinate points SP1'to SP10' are projected is shown. As shown in FIG. 15, the projected coordinate point output unit 60 is predetermined with intervals D1, interval D2, interval D3, interval D4, interval D5, interval D6, interval D7, interval D8, interval D9, and interval D10, respectively. Compare with the given threshold.

The interval D1 is an interval between the coordinate points SP1 and SP1'in the traveling direction of the vehicle 10. The interval D2 is the interval between the coordinate points SP2 and SP2'in the traveling direction of the vehicle 10. The interval D3 is the interval between the coordinate points SP3 and SP3'in the traveling direction of the vehicle 10. The interval D4 is the interval between the coordinate points SP4 and SP4'in the traveling direction of the vehicle 10. The interval D5 is the interval between the coordinate points SP5 and SP5'in the traveling direction of the vehicle 10. The interval D6 is an interval between the coordinate points SP6 and SP6'in the traveling direction of the vehicle 10. The interval D7 is the interval between the coordinate points SP7 and SP7'in the traveling direction of the vehicle 10. The interval D8 is the interval between the coordinate points SP8 and SP8'in the traveling direction of the vehicle 10. The interval D9 is the interval between the coordinate points SP9 and SP9'in the traveling direction of the vehicle 10. The interval D10 is the interval between the coordinate points SP10 and SP10'in the traveling direction of the vehicle 10.

In FIG. 15, since the interval D1 is substantially zero, the coordinate point SP1 and the corresponding coordinate point SP1'are indicated by the same stars, and the interval D1 is not shown. Since the interval D2 is also substantially zero, the coordinate point SP2 and the corresponding coordinate point SP2'are indicated by the same star mark, and the interval D2 is not shown. Similarly, since the intervals D3, the intervals D4, the intervals D6, and the intervals D7 are also substantially zero, the illustration of these intervals is omitted. As shown in FIG. 15, the projection coordinate point output unit 60 determines that each of the intervals D1, D2, D3, D4, D6, and D7 is equal to or less than the threshold value. Based on this determination, as shown in FIG. 16, the coordinate points SP1', SP2', SP3', SP4', SP6', and SP7'are left on the superimposed bird's-eye view image.

The projection coordinate point output unit 60 determines that the threshold values are exceeded for each of the intervals D5, D8, D9, and D10. Based on this determination, the coordinate points SP5', SP8', SP9', and SP10'are deleted from the superimposed bird's-eye view image as shown in FIG.

As described above, in the peripheral monitoring device 100 of the present embodiment, the projected coordinate point specifying unit 53 points the point where the first straight line L1 and the first straight line L21 intersect on the superimposed bird's-eye view image SVI on the bar 85. It is specified as a projected coordinate point SP1 in which the bar 85 is projected onto the road surface 80 directly below. The projected coordinate point specifying unit 53 sets the point where the second straight line L2 and the second straight line L22 intersect on the superimposed bird's-eye view image SVI as the projected coordinate point SP2 in which the bar 86 is projected onto the road surface 80 directly below the bar 86. Identify. The specified projected coordinate points SP1 and SP2 are displayed on the monitor 150 on the superimposed bird's-eye view image in the projected coordinate point output unit 60. Therefore, the driver of the vehicle 10 can clearly grasp the positions of the bars 85 and 86 through the superimposed bird's-eye view image projected on the monitor 150. Therefore, it is possible to prevent the vehicle 10 from coming into contact with the bars 85 and 86 existing around the vehicle 10.

Further, according to the peripheral monitoring device 100 of the present embodiment, the projected coordinate point output unit 60 determines that the threshold values are exceeded for each of the intervals D5 and D8, and the coordinate points SP5'and SP8' from the superimposed bird's-eye view image. Erase. The coordinate point SP5'corresponds to the coordinate point SP5 indicating the position of the center of gravity of the bar 85. The coordinate point SP8'corresponds to the coordinate point SP8 indicating the position of the center of gravity of the bar 86. The coordinate points SP5 and SP8 indicating the positions of the centers of gravity of the bars 85 and 86 appear to be farther than the actual positions of the centers of gravity due to the characteristics of the bird's-eye view image. Therefore, by erasing the coordinate points different from the actual positions from the superimposed bird's-eye view image, the usability is improved.

In the above embodiment, an example in which the field of view (angle of view) of the imaging unit 12 is set to 180 ° is shown. However, it is not limited to this. The viewing range (angle of view) of the imaging unit 12 may be set to an angle other than 180 °. For example, the field of view (angle of view) of the imaging unit 12 is set to 120 °, and the three-dimensional object region 72 (t−Δt) _1,72 (t−Δt) _6 shown in FIG. 4 is out of the field of view of the imaging unit 12. It may be an invisible area.

In the above embodiment, the projected coordinate point output unit 60 shows an example in which the projected coordinate point obtained by projecting the bar onto the road surface 80 directly below the bar is displayed on the monitor 150. However, it is not limited to this. The projection coordinate point output unit 60 may output the projection coordinate point by voice.

In the above embodiment, an example is shown in which the image pickup unit 12 is configured by the left camera 12a attached to the left side of the vehicle 10. However, it is not limited to this. For example, the image pickup unit 12 may be configured by a front camera attached to the front of the vehicle 10, a rear camera attached to the rear of the vehicle 10, or a right camera attached to the right side of the vehicle 10.

In the above embodiment, the three-dimensional object detection unit 40 shows an example of detecting a three-dimensional object region by utilizing the difference in brightness between the bird's-eye view images. However, it is not limited to this. For example, the three-dimensional object detection unit 40 divides the bird's-eye view images into a plurality of small blocks and divides the bird's-eye view images into a plurality of small blocks and the similarity of the result of edge detection (edge strength, similarity in the edge direction). The three-dimensional object region may be detected by using the similarity of the grayscale histogram and the histogram of the edge detection result obtained from the above.

In the above embodiment, an example is shown in which the three-dimensional object region extraction unit is composed of the first three-dimensional object region extraction unit 51 and the second three-dimensional object region extraction unit 52. However, it is not limited to this. For example, the three-dimensional object region extraction unit may be composed of a single three-dimensional object region extraction unit.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the present invention is not limited to the configuration of the embodiments because the embodiments are merely examples of the present invention. It goes without saying that even if there is a design change or the like within a range that does not deviate from the gist, it is included in the present invention.

10 ... Vehicle 12 ... Imaging unit 12a ... Left camera 30 ... Bird's-eye view image generation unit 50 ... Projection coordinate point acquisition unit 51 ... First three-dimensional object area extraction unit 51A ... Region division 51B ... Vertical evaluation unit 52 ... Second three-dimensional object region extraction unit 52A ... Region division 52B ... Vertical evaluation unit 53 ... Projected coordinate point identification unit 72 (t- Δt), 72 (t) ... Bird's-eye view image 72'(t-Δt), 72'(t) ... Predicted bird's-eye view image 72 (t-Δt) _1,72 (t-Δt) _2,72 (t) -Δt) _3,72 (t-Δt) _4,72 (t-Δt) _5,72 (t-Δt) _6,72 (t) _1,72 (t) _2,72 (t) _3,72 (t) ) _4,72 (t) _5,72 (t) _6 ... Three-dimensional object area 80 ... Road surface 81, 82, 83, 84 ... Supports 85, 86 ... Bar (floating object)
87, 88 ... Support column 100 reflected on the surface of a puddle ... Peripheral monitoring device 122 ... Display control unit 150 ... Monitor (display unit)
G1, G2, G21, G22 ... Center of gravity L1, L21 ... 1st straight line L2, L22 ... 2nd straight line RL10, RL11, RL12, RL13, RL14, RL15, RL16, RL17 ... 1st radiation RL20, RL21, RL22, RL23, RL24, RL25, RL26, RL27 ... Second radiation SP1, SP1', SP2, SP2' ... Projected coordinate points SV1, SV2', SV3', SV4 ... Composite vector SVI ... Superimposed bird's-eye view image V1, V2, V3, V4, V5, V6, V21, V22, V23, V24, V25, V26, V31, V31', V32, V33, V33', V34'... Vector θ1 , Θ2 ・ ・ ・ Equal angle (predetermined angle)

Claims (5)

An imaging unit attached to a vehicle traveling on the road surface to image the surroundings of the vehicle including floating objects that are not in contact with the road surface.
A bird's-eye view image generation unit that generates a first bird's-eye view image and a second bird's-eye view image by converting the viewpoints of two images obtained by capturing the surroundings by the imaging unit and changing according to the movement of the vehicle. When,
The first bird's-eye view image is divided into a plurality of three-dimensional object regions centered on the position of the imaging unit by the first radiation extending in the radial direction from the imaging unit at predetermined angles, and the floating object is vertical to the road surface. A first three-dimensional object region extraction unit that evaluates the property and extracts the first three-dimensional object region containing the suspended matter,
The floating object is vertical to the road surface from a plurality of three-dimensional object regions divided at predetermined angles about the position of the imaging unit by the second radiation extending from the imaging unit in the radial direction. A second three-dimensional object region extraction unit that evaluates the property and extracts the second three-dimensional object region containing the suspended matter,
On the superimposed bird's-eye view image in which the first bird's-eye view image and the second bird's-eye view image are superimposed, a first straight line extending along the first radiation from the center of gravity of the floating object included in the first three-dimensional object region, and the first straight line. 2 Projection coordinate points at which the floating object is projected onto the road surface directly below the floating object at the intersection of the center of gravity of the floating object included in the three-dimensional object region and the second straight line extending along the second radiation. A peripheral monitoring device characterized by having a projection coordinate point specifying unit specified as. In the peripheral monitoring device according to claim 1,
The display unit mounted on the vehicle and
A peripheral monitoring device including a display control unit that displays the superimposed bird's-eye view image including the projected coordinate points specified by the projected coordinate point specifying unit on the display unit. In the peripheral monitoring device according to claim 2,
The display control unit
When a particular said projected coordinates point by the projection coordinate point specifying unit is completed, a coordinate point indicating the position of the centroid of the suspended matter contained in the first three-dimensional object area, the included in the second three-dimensional object area to erase a coordinate point indicating the center of gravity position of the suspended solids from the display unit surroundings monitoring apparatus according to claim. In the peripheral monitoring device according to claim 2 or 3.
The image pickup unit takes an image of a linear body that intersects a support column erected on the road surface at a position higher than the road surface as the floating object.
The display control unit
A vector in the longitudinal direction of the support column included in the first three-dimensional object region or the second three-dimensional object region, and a vector in the longitudinal direction of the linear body included in the first three-dimensional object region or the second three-dimensional object region. When the angle formed by the composite vector of the above and the radiation vector extending in the radiation direction from the imaging unit is within a predetermined reference angle, the superimposed bird's-eye view image including the projected coordinate points is displayed on the display unit. Peripheral monitoring device characterized by In the peripheral monitoring device according to any one of claims 1 to 4.
The image pickup unit takes an image of a linear body that intersects a support column erected on the road surface at a position higher than the road surface as the floating object.
The first three-dimensional object region extraction unit includes a vector in the longitudinal direction of the linear body included in a specific three-dimensional object region among a plurality of three-dimensional object regions obtained by dividing the first bird's-eye view image, and radiation of the imaging unit. The verticality of the linear body with respect to the road surface is evaluated by determining whether or not the angle formed by the direction vector exceeds a predetermined reference angle.
The second three-dimensional object region extraction unit includes a vector in the longitudinal direction of the linear body included in a specific three-dimensional object region among a plurality of three-dimensional object regions obtained by dividing the second bird's-eye view image, and radiation of the imaging unit. A peripheral monitoring device for evaluating the verticality of the linear body with respect to the road surface by determining whether or not the angle formed by the direction vector exceeds a predetermined reference angle.

Images