JP7485580B2 - Distance calculation device and distance calculation method - Google Patents

Description

The present invention relates to a distance calculation device and a distance calculation method.

Conventionally, there is technology for measuring distance based on images, etc. Patent Document 1 discloses a distance measuring device equipped with a distance measuring unit that selects and executes a first distance measurement that measures the distance to the target based on an image of the target captured by a stereo camera, and a second distance measurement that measures the distance to the target based on the reflected light after a light emitting unit irradiates the target with a light beam.

JP 2020-112434 A

Here, it is desirable to be able to calculate distance using an inexpensive configuration. For example, if distance could be calculated from an image taken with a monocular camera, costs could be reduced.

The object of the present invention is to provide a distance calculation device and a distance calculation method that can calculate distance with an inexpensive configuration.

The distance calculation device of the present invention is characterized by comprising an image acquisition unit that acquires a video image of the moving body in the traveling direction generated by a camera mounted on the moving body, a detection unit that detects a first structure and a second structure located at different positions along the traveling direction of the moving body from an image of the video, a distance coefficient calculation unit that acquires the moving distance of the moving body from a position corresponding to the first structure to a position corresponding to the second structure based on the speed data of the moving body and the detection result of the detection unit, and determines a coefficient indicating the relationship between the number of pixels in the vertical direction of the image and the distance from the detection result of the detection unit and the acquired moving distance, and a distance calculation unit that calculates the distance along the traveling direction of the moving body based on the coefficient and the number of pixels in the vertical direction of the image in the image.

The distance calculation device according to the present invention detects a first structure and a second structure from a video image, and obtains the distance traveled by a moving object from a position corresponding to the first structure to a position corresponding to the second structure. The distance coefficient calculation unit determines a coefficient indicating the relationship between the number of pixels in the vertical direction of the image and the distance from the detection result of the detection unit and the obtained distance traveled. The distance calculation unit calculates the distance along the traveling direction of the moving object based on the coefficient and the number of pixels in the vertical direction of the image in the image. The distance calculation device according to the present invention has the effect of being able to calculate the distance with an inexpensive configuration.

FIG. 1 is a block diagram of a distance calculation device according to an embodiment. FIG. 2 is a diagram showing an image of a structure. FIG. 3 is a diagram for explaining the position of a pixel that represents a structure. FIG. 4 is a diagram showing an image of a structure. FIG. 5 is a diagram showing the position of a vehicle. FIG. 6 is a diagram showing an image of a structure. FIG. 7 is a diagram showing the position of a vehicle. FIG. 8 is a diagram showing a number of frames. FIG. 9 is a diagram for explaining a method of calculating the average speed. FIG. 10 is a diagram showing an image of a structure. FIG. 11 is a diagram showing the distance between structures. FIG. 12 is a diagram for explaining a method of calculating the coefficient and the distance. FIG. 13 is a diagram for explaining a method of calculating the distance. FIG. 14 is a flowchart of the distance calculation device and distance calculation method according to the embodiment. FIG. 15 is a flowchart of the distance calculation device and distance calculation method according to the embodiment.

The distance calculation device and distance calculation method according to the embodiment of the present invention will be described in detail below with reference to the drawings. Note that the present invention is not limited to these embodiments. Furthermore, the components in the following embodiments include those that can be easily imagined by a person skilled in the art or those that are substantially the same.

[Embodiment]
An embodiment will be described with reference to Fig. 1 to Fig. 15. The embodiment relates to a distance calculation device and a distance calculation method. Fig. 1 is a block diagram of the distance calculation device according to the embodiment, Fig. 2 is a diagram showing an image in which a structure is captured, Fig. 3 is a diagram explaining the position of a pixel representing the structure, Fig. 4 is a diagram showing an image in which the structure is captured, Fig. 5 is a diagram showing the position of a vehicle, Fig. 6 is a diagram showing an image in which the structure is captured, Fig. 7 is a diagram showing the position of a vehicle, Fig. 8 is a diagram showing a plurality of frames, Fig. 9 is a diagram explaining a method for calculating an average speed, and Fig. 10 is a diagram showing an image in which the structure is captured.

Figure 11 shows the distance between structures, Figure 12 explains the coefficient and distance calculation method, Figure 13 explains the distance calculation method, and Figures 14 and 15 are flowcharts of the distance calculation device and distance calculation method according to the embodiment.

The distance calculation device 1 of this embodiment shown in FIG. 1 is mounted on a vehicle 100. The illustrated distance calculation device 1 is configured to assist the driver in driving operations based on the calculated distance. The vehicle 100 is, for example, an automobile, and may be a hybrid automobile, a plug-in hybrid automobile, or an electric automobile. The vehicle 100 may be a vehicle for transporting cargo or a vehicle for transporting personnel.

The vehicle 100 is equipped with a drive recorder 20. The drive recorder 20 is an example of an on-board device that is installed in the vehicle 100. The drive recorder 20 has a monocular camera 21. The camera 21 continuously captures images in the traveling direction of the vehicle 100. In other words, the camera 21 is a device that generates video data. The camera 21 captures images in front of the vehicle 100. The drive recorder 20 may also have a camera that captures images behind the vehicle 100.

The drive recorder 20 has a video recording unit 23 that records images captured by the camera 21. The video recording unit 23 is, for example, a non-volatile memory. The drive recorder 20 may use a memory card as the video recording unit 23. The drive recorder 20 records image data captured by the camera 21 and the capture time of the image data in the video recording unit 23.

The vehicle 100 is equipped with a speed sensor 22 that detects the traveling speed of the vehicle 100. The speed sensor 22 detects the traveling speed at a predetermined time interval. The drive recorder 20 has an interface for communicating with the speed sensor 22. The drive recorder 20 has a speed recording unit 24 that records the speed data acquired from the speed sensor 22 and the time when the speed data was acquired. The speed recording unit 24 is, for example, a non-volatile memory. The drive recorder 20 may use a memory card as the speed recording unit 24.

The distance calculation device 1 has a determination unit 2, a warning unit 3, a distance calculation unit 4, a detection unit 5, a distance factor recording unit 6, an image acquisition unit 7, and a distance factor calculation unit 8. The distance calculation device 1 has a distance calculation function 10. The distance calculation function 10 is realized by the distance calculation unit 4, the detection unit 5, the distance factor recording unit 6, the image acquisition unit 7, and the distance factor calculation unit 8.

The image acquisition unit 7 acquires video data from the video recording unit 23 of the drive recorder 20. This video is video captured by the camera 21 of the area in front of the vehicle 100. The video data includes images of each frame and the capture time. The image acquisition unit 7 may be, for example, a communication interface that communicates with the drive recorder 20. The video data acquired by the image acquisition unit 7 may be recorded in the memory of the distance calculation device 1, etc.

The detection unit 5 detects structures from the images of the video data. In image G1 shown in FIG. 2, structures detected by the detection unit 5 are captured. The dashboard 101 of the vehicle 100 is shown at the bottom of the image. The detection unit 5 detects structures installed along the road 200 on which the vehicle 100 is traveling. In image G1 shown in FIG. 2, the detection unit 5 detects structures A, B, C, and D. The structures A, B, C, and D are located at different positions along the traveling direction of the vehicle 100. In the following description, the traveling direction of the vehicle 100 is simply referred to as the "traveling direction X". The structures A, B, C, and D are lined up in this order from the rear to the front of the traveling direction X.

The structures A, B, C, and D detected by the detection unit 5 are structures that protrude upward from the ground, such as utility poles, street lights, traffic lights, and signs. These structures A, B, C, and D have, for example, supports that protrude upward from the ground. The detection unit 5 detects structures on both ends of the road 200, for example, but extracts structures along one edge. The detection unit 5 of this embodiment extracts structures A, B, C, and D along the left edge of the road 200 as seen by the driver.

As shown in FIG. 2, the detection unit 5 detects the range in which each of structures A, B, C, and D is imaged. Range RA is the range in which structure A is imaged in the image. In other words, range RA is a frame surrounding structure A. Similarly, ranges RB, RC, and RD are ranges in which structures B, C, and D are imaged, and are frames surrounding structures B, C, and D, respectively. The shapes of ranges RA, RB, RC, and RD are rectangular. In addition, the four corners of ranges RA, RB, RC, and RD are pixel positions in the image.

As shown in FIG. 3, the detection unit 5 determines positions Pa, Pb, Pc, and Pd of pixels representing structures A, B, C, and D. Pixel positions Pa, Pb, Pc, and Pd are positions in the vertical direction of the image. For example, pixel position Pa is the vertical position of the pixel representing structure A. Pixel position Pa is, for example, the position of the bottom end of range RA. In other words, pixel position Pa is the position of the pixel that captures the bottom end of structure A. Pixel positions Pb, Pc, and Pd are the vertical positions of the pixels representing structures B, C, and D. Pixel positions Pb, Pc, and Pd are, for example, the positions of the bottom ends of ranges RB, RC, and RD, respectively.

The distance calculation device 1 of this embodiment determines a coefficient Ki (i = 1, 2, 3, ...) based on pixel positions Pa, Pb, Pc, Pd and the distance traveled when the vehicle 100 travels along structures A, B, C, D. The coefficient Ki is a coefficient that indicates the relationship between the number of pixels in the vertical direction of the image and the actual distance on the road 200. The distance calculation device 1 calculates the actual distance based on the number of pixels in the vertical direction of the image and the coefficient Ki, as described below.

The distance factor calculation unit 8 obtains the travel distance of the vehicle 100 based on the data on the traveling speed of the vehicle 100 and the detection results of the detection unit 5. Here, as an example, a method for obtaining the distance from structure C to structure D will be described. The distance factor calculation unit 8 determines the time Tc when the vehicle 100 reaches a position corresponding to structure C, and the time Td when the vehicle 100 reaches a position corresponding to structure D. The times Tc and Td are determined by the detection unit 5 as described below. The distance factor calculation unit 8 obtains information on the times Tc and Td from the detection unit 5.

The detection unit 5 selects an image captured when the vehicle 100 reaches a position corresponding to the structure C from the multiple images G acquired from the drive recorder 20. FIG. 4 shows image G3 captured when the vehicle 100 reaches the position of the structure C. FIG. 5 shows the position of the vehicle 100 when image G3 in FIG. 4 was captured. The vehicle 100 has reached position Xc corresponding to the structure C. Position Xc is the position of the structure C along the traveling direction X of the vehicle 100. In other words, position Xc is a position on a virtual line Ic on the road 200. The virtual line Ic is a straight line drawn from the structure C along the lateral direction Y perpendicular to the traveling direction X.

The position of the vehicle 100 when image G3 was captured is considered to be slightly closer than position Xc. However, this difference in position has only a small effect on calculating the distance from structure C to structure D. Therefore, in this embodiment, the position of the vehicle 100 when image G3 was captured is considered to be position Xc.

In image G3 shown in FIG. 4, the range RC of structure C is located at the left end. More specifically, the left edge RC1 of range RC is located at the leftmost pixel in image G3. In other words, image G3 is an image when structure C is cut off at the horizontal end GL of the image. In other words, image G3 is an image captured when structure C begins to move from within the imaging range of camera 21 to outside the range. The detection unit 5 selects as image G3 an image in which the range RC of structure C is located at the left end.

The detection unit 5 selects, from the multiple images G acquired from the drive recorder 20, an image captured when the vehicle 100 reaches a position Xd corresponding to the structure D. FIG. 6 shows an image G4 captured when the vehicle 100 reaches the position Xd. FIG. 7 shows the position of the vehicle 100 when the image G4 in FIG. 6 is captured. The vehicle 100 has reached the position Xd corresponding to the structure D. The position Xd is the position of the structure D along the traveling direction X of the vehicle 100. In other words, the position Xd is a position on the virtual line Id on the road 200. The virtual line Id is a straight line drawn from the structure D along the lateral direction Y.

The position of the vehicle 100 when image G4 was captured is considered to be slightly closer than position Xd. However, this difference in position has only a small effect on calculating the distance from structure C to structure D. Therefore, in this embodiment, the position of the vehicle 100 when image G4 was captured is considered to be position Xd.

In image G4 shown in FIG. 6, the range RD of structure D is located at the left end. The left side RD1 of range RD is located at the leftmost pixel in image G4. In other words, image G4 is an image when structure D is cut off at the horizontal end GL of the image. In other words, image G4 is an image captured when structure D begins to move from within the imaging range of camera 21 to outside the range. The detection unit 5 selects as image G4 an image in which range RD of structure D is located at the left end.

The distance factor calculation unit 8 acquires information about the images G3 and G4 from the detection unit 5. The distance factor calculation unit 8 sets the time when the image G3 is captured as the time Tc, and sets the time when the image G4 is captured as the time Td. The distance factor calculation unit 8 calculates the travel distance of the vehicle 100 from the time Tc to the time Td, as described below.

As shown in FIG. 8, there are a plurality of frames between image G3 and image G4. In the following description, the i-th frame in the video is described as frame F _i . The frame number i of image G3 is c, and the frame number i of image G4 is d. The distance coefficient calculation unit 8 calculates the travel distance of the vehicle 100 between each frame, for example, based on the average speed VA between each frame. FIG. 9 shows the speed value V _j acquired from one frame F _n to the next frame F _n+1 . The speed value V ₀ is acquired at the time t _n when the frame F _n is captured. This speed value V ₀ is not used to calculate the average speed VA. The speed value V _m is acquired at the time t _n+1 when the frame F _n+1 is captured. Based on each speed value V _j from the speed value V ₁ to the speed value V _m , the average speed VA is calculated by the following [Equation 1]. Here, m is the number of velocity values _Vj acquired from the velocity value _V1 acquired first after time _tn to time _tn+1 . This number m includes the velocity value _Vj acquired at time _tn+1 . In the example of FIG. 9, m=7.

Distance factor calculation unit 8 calculates distance Lcd from position Xc to position Xd using the following [Equation 2]. Here, _VAi is the average speed VA from frame F _i to frame F _i+1 , and ΔT is the time interval from one frame to the next. In other words, distance Lcd is the integral value of average speed VA from frame F _c to frame F _d .

The distance Lcd calculated by the above [Equation 2] is the travel distance of the vehicle 100 from the position Xc corresponding to the structure C to the position Xd corresponding to the structure D. As described above, the distance Lcd is obtained based on the speed data of the vehicle 100 and the detection result of the detection unit 5.

The distance Lab from structure A to structure B and the distance Lbc from structure B to structure C are calculated using a procedure similar to that used to calculate the distance Lcd from position Xc to position Xd. For example, when calculating the distance Lab, the detection unit 5 detects an image in which structure A is cut off. Image G1 in FIG. 2 is an image in which structure A is cut off. In image G1, the range RA of structure A is located at the left end. The time when image G1 is captured is set to the time Ta when the vehicle 100 reaches position Xa corresponding to structure A.

The detection unit 5 also detects an image when structure B is cut off. Image G2 in FIG. 10 is an image when structure B is cut off. In image G2, the range RB of structure B is located at the left end. The time when image G2 is captured is set to time Tb when the vehicle 100 reaches position Xb corresponding to structure B. FIG. 11 shows the times when the vehicle 100 reaches structures A, B, C, and D, and the distances Lab, Lbc, and Lcd between structures A, B, C, and D. The distance factor calculation unit 8 calculates the distance Lab based on the speed data of the vehicle 100 from time Ta to time Tb. Similarly, the distance factor calculation unit 8 calculates the distance Lbc based on the speed data of the vehicle 100 from time Tb to time Tc.

The distance coefficient calculation unit 8 determines a coefficient Ki (i = 1, 2, 3, ...) based on the acquired distances Lab, Lbc, and Lcd. The coefficient Ki is a coefficient that indicates the relationship between the number of pixels in the vertical direction of the image and the distance along the traveling direction X of the vehicle 100.

The distance factor calculation unit 8 determines the coefficient Ki based on an image in which all of the structures A, B, C, and D are captured, for example. All of the structures A, B, C, and D are captured in the image G0 shown in FIG. 12. The distance factor calculation unit 8 determines the coefficient Ki for each pixel range Si (i = 1, 2, 3, ...) of the image. The pixel range S1 is the range from pixel position Pa to pixel position Pb in the vertical coordinate value Z of the image. Note that the origin of the coordinate value Z is the bottom edge of the image.

Pixel range S2 is the range from pixel position Pb to pixel position Pc at coordinate value Z. Pixel range S3 is the range from pixel position Pc to pixel position Pd at coordinate value Z.

The distance factor calculation unit 8 calculates a factor K1 for the pixel range S1 by the following formula (1): where N1 is the number of pixels in the pixel range S1 in the vertical direction of the image.
K1=Lab/N1 (1)

Distance factor calculation unit 8 calculates coefficient K2 for pixel range S2 using the following formula (2), and calculates coefficient K3 for pixel range S3 using the following formula (3). Note that N2 is the number of pixels in the vertical direction of the image in pixel range S2, and N3 is the number of pixels in the vertical direction of the image in pixel range S3. Distance factor calculation unit 8 stores the calculated coefficients K1, K2, and K3 in distance factor recording unit 6.
K2=Lbc/N2 (2)
K3=Lcd/N3 (3)

The distance calculation unit 4 calculates the distance along the traveling direction X of the vehicle 100 based on the coefficients K1, K2, and K3 and the number of pixels in the vertical direction of the image. For example, the distance calculation unit 4 calculates the distance L1 along the traveling direction X from point 201 to point 202 on the road 200 as follows. The position of point 201 is position Xa corresponding to structure A. Point 202 is a point located between structure C and structure D in the traveling direction X. The distance calculation unit 4 acquires the number of pixels ΔZ in the vertical direction of the image from pixel position Pc to the pixel corresponding to point 202. The distance calculation unit 4 acquires the number of pixels ΔZ from the detection unit 5, for example. The distance calculation unit 4 calculates the distance L1 by the following formula (4).
L1 = Lab + Lbc + K3 × ΔZ (4)

When point 202 is a point located between structure B and structure C, distance calculation unit 4 calculates distance L1 as follows. Distance calculation unit 4 acquires the number of pixels ΔZ in the vertical direction of the image from pixel position Pb to the pixel corresponding to point 202. Distance calculation unit 4 calculates distance L1 by the following formula (5).
L1 = Lab + K2 × ΔZ (5)

When point 202 is a point located between structure A and structure B, distance calculation unit 4 calculates distance L1 as follows. Distance calculation unit 4 acquires the number of pixels ΔZ in the vertical direction of the image from pixel position Pa to the pixel corresponding to point 202. Distance calculation unit 4 calculates distance L1 by the following formula (6).
L1=K1×ΔZ (6)

The distance calculation unit 4 can calculate the distance from the vehicle 100 to a point ahead. Image Gx in FIG. 13 is an image of the leading vehicle 110. Image Gx is an image when structure A is cut off at the lateral edge of the image. In other words, image Gx is an image when the vehicle 100 reaches position Xa corresponding to structure A. Therefore, the distance from pixel position Pa to a point on the road 200 corresponds to the distance from the vehicle 100 to the point ahead.

13, pixel position Px is the position of a pixel representative of the preceding vehicle 110. Detection of the preceding vehicle 110 and detection of pixel position Px are performed by the detection unit 5. The preceding vehicle 110 is located between structure C and structure D. In this case, the distance calculation unit 4 acquires the number of pixels ΔZ in the vertical direction of the image from pixel position Pc to pixel position Px from the detection unit 5. The distance calculation unit 4 calculates the distance Lx from the vehicle 100 to the preceding vehicle 110 by the following formula (7).
Lx = Lab + Lbc + K3 × ΔZ (7)

As described above, the distance calculation device 1 according to this embodiment can calculate the distance to a forward position based on the video generated by the monocular camera 21. Therefore, it is possible to calculate the distance without using expensive distance detection devices such as a stereo camera, LiDAR, or ToF sensor. Furthermore, the distance calculation device 1 can calculate the distance between any two points ahead of the vehicle 100. Note that the above coefficient Ki is applicable to images in which structures A, B, C, and D are not captured. In other words, the coefficients K1, K2, and K3 for the pixel ranges S1, S2, and S3 are applicable to any image captured by the camera 21.

The operation of the distance calculation device 1 according to this embodiment will be described with reference to Figures 14 and 15. Figure 14 is a flowchart of the phase in which the coefficient Ki is calculated. The distance calculation device 1, for example, periodically executes the operation shown in the flowchart of Figure 14. The flowchart of Figure 14 may be executed, for example, once a day, or once every few days. In step S10, the distance coefficient calculation unit 8 acquires speed data. The acquired speed data is, for example, data on consecutive speed values and acquisition times acquired by the speed sensor 22 during a certain period of time. After step S10 is executed, the process proceeds to step S20.

In step S20, the distance factor calculation unit 8 calculates the average speed VA between each frame. For example, the distance factor calculation unit 8 obtains information on the image capture time of each frame from the detection unit 5, and calculates the average speed VA based on this information. After step S20 is executed, the process proceeds to step S80.

In step S30, the image acquisition unit 7 acquires video data. The acquired video data is video data captured by the camera 21, and includes multiple images captured continuously. Step S30 corresponds to a step of acquiring a video captured by the camera 21 in the traveling direction X. After step S30 is executed, the process proceeds to steps S40, S50, and S70.

In step S40, the image acquisition unit 7 calculates the time ΔT between each frame. The image acquisition unit 7 calculates the time ΔT between frames based on the time when one frame was captured and the time when the immediately preceding frame was captured. The image acquisition unit 7 records the calculated time ΔT, for example, by linking it to the frame number. Note that the time ΔT between frames may be calculated from the frame rate. Once step S40 has been executed, the process proceeds to step S80.

In step S50, the detection unit 5 detects objects from each image data. Objects to be detected include, for example, structures such as the exemplified structures A, B, C, and D. Objects to be detected include other vehicles including the preceding vehicle 110, pedestrians, traffic lights, stop lines, crosswalks, and the like. These objects are, for example, objects related to driving assistance for the driver. The detection unit 5 records, for example, the type of detected object and the range of the detected object by linking them to the frame number of the video.

Step S50 corresponds to a step of detecting a first structure and a second structure located at different positions along the traveling direction X of the vehicle 100 from the image. The first structure and the second structure are structures that serve as landmarks when calculating the distance. For example, in image G1 of FIG. 2, the first structure and the second structure are any two structures among structures A, B, C, and D. As an example, when obtaining the travel distance from structure A to structure B, the detection unit 5 detects structure A as the first structure and structure B as the second structure. After step S50 is executed, the process proceeds to step S60.

In step S60, the detection unit 5 calculates the number of frames between the objects. The detection unit 5 counts the number of frames from when one structure is cut off by the edge of the image until the next structure is cut off by the edge of the image. The detection unit 5, for example, records the calculated number of frames by linking it to the structure. Note that the number of frames between the objects may be calculated from the times Ta, Tb, Tc, Td and the frame rate. Once step S60 has been executed, the process proceeds to step S80.

In step S70, the detection unit 5 detects pixel positions in the vertical direction of the image. The detected pixel positions are, for example, positions Pa, Pb, Pc, and Pd of pixels that represent the structure. After step S70 is executed, the process proceeds to step S90.

In step S80, the distance factor calculation unit 8 calculates the distance between the objects. The distance factor calculation unit 8 calculates the travel distance of the vehicle 100 between frames based on the average speed VA calculated in step S20 and the time between frames ΔT calculated in step S40. The distance factor calculation unit 8 acquires the frame number when the structures A, B, C, and D are visible at the edge of the image from the detection unit 5. The distance factor calculation unit 8 acquires the distances Lab, Lbc, and Lcd based on the travel distance between frames and the acquired frame number. Once step S80 has been executed, the process proceeds to step S90.

In step S90, the distance factor calculation unit 8 calculates the coefficient Ki. The calculated coefficient Ki is a proportional coefficient of the distance between objects relative to the pixel position in the vertical direction of the image. For example, the distance factor calculation unit 8 determines the coefficient Ki (i = 1, 2, 3, ...) based on an image in which all of the structures A, B, C, and D are captured. The distance factor calculation unit 8 stores the calculated coefficient Ki in the distance factor recording unit 6. When step S90 is executed, the flowchart ends.

Figure 15 is a flowchart of the phase in which distance is calculated based on the coefficient Ki. The flowchart in Figure 15 is executed repeatedly, for example, while the vehicle 100 is traveling. In step S110, the distance calculation unit 4 acquires the coefficient Ki (i = 1, 2, 3 ...) from the distance coefficient recording unit 6. After step S110 is executed, the process proceeds to step S150.

In step S120, the image acquisition unit 7 acquires video data. Step S120 corresponds to a step of acquiring a video image of the traveling direction X captured by the camera 21. After step S120 is executed, the process proceeds to steps S130 and S160.

In step S130, the detection unit 5 detects objects from each image data. Objects to be detected include, for example, structures such as the exemplified structures A, B, C, and D. Objects to be detected include other vehicles including the preceding vehicle 110, pedestrians, traffic lights, stop lines, crosswalks, and the like. The detection unit 5 records, for example, the type of object detected and the range of the object detected by linking them to the frame number of the video. After step S130 is executed, the process proceeds to step S140.

In step S140, the detection unit 5 detects the pixel position in the vertical direction of the image of a moving object present ahead based on a command from the distance calculation unit 4. The moving object ahead is, for example, a leading vehicle. The distance calculation unit 4 obtains the pixel position of the moving object from the detection unit 5. After step S140 is executed, the process proceeds to step S150.

In step S150, the distance calculation unit 4 calculates the distance between the vehicle and the moving object ahead. If the moving object ahead is a vehicle ahead, the distance Lx between the vehicle and the moving object ahead is calculated. In this case, the distance calculation unit 4 calculates the distance Lx from the vehicle 100 to the leading vehicle 110 based on the coefficient Ki acquired in step S110 and the pixel position acquired in step S140.

Step S150 corresponds to a step of calculating the distance along the traveling direction X of the vehicle 100 based on the coefficient Ki and the number of pixels in the vertical direction of the image in the image. After step S150 is executed, the process proceeds to step S180.

In step S160, the detection unit 5 detects the behavior of the object. More specifically, the detection unit 5 has acquired the function of detecting the behavior of the object from video data by machine learning such as deep learning. The detection unit 5 detects the behavior of the object based on multiple image data in which the object is detected. For example, the detection unit 5 can detect a preceding vehicle 110 that is zigzagging or a bicycle in front that is zigzagging. The detection unit 5 records, for example, the frame number in which the zigzagging object was detected, the type of object, and the position of the object. After step S160 is executed, the process proceeds to step S170.

In step S170, the determination unit 2 determines whether the behavior of the detected moving object ahead is abnormal. The determination unit 2 makes the determination in step S170 based on the detection result by the detection unit 5. For example, the determination unit 2 determines that the behavior is abnormal when a preceding vehicle 110 that is zigzagging is detected, or when a bicycle that is traveling unsteadily is detected. If the result of the determination in step S170 is that the behavior of the moving object is abnormal, the process proceeds to step S180, and if the result is that the behavior is abnormal, the process proceeds to step S210.

In step S180, the determination unit 2 determines whether the distance to the moving object ahead is equal to or less than a lower limit. If the determination is affirmative that the distance is equal to or less than the lower limit, the process proceeds to step S190, and if the determination is negative, the process proceeds to step S210.

In step S190, the determination unit 2 determines that a moving object that is dangerous to the vehicle is present. After step S190 is executed, the process proceeds to step S200.

In step S200, the determination unit 2 commands the warning unit 3 to execute a warning operation. The warning unit 3 issues a warning to the driver by sound, light, video, etc. When step S200 is executed, the flowchart ends.

In step S210, the determination unit 2 determines that the detected object is safe for the vehicle 100. When step S210 is executed, this flowchart ends.

As described above, the distance calculation device 1 according to this embodiment includes the image acquisition unit 7, the detection unit 5, the distance factor calculation unit 8, and the distance calculation unit 4. The image acquisition unit 7 acquires a video generated by the camera 21 mounted on the vehicle 100. The acquired video includes a plurality of images captured continuously in the traveling direction X of the vehicle 100. The detection unit 5 detects a first structure and a second structure located at different positions along the traveling direction X of the vehicle 100 from the images of the video. The distance factor calculation unit 8 acquires the travel distance of the vehicle 100 from a position corresponding to the first structure to a position corresponding to the second structure based on the speed data of the vehicle 100 and the detection result of the detection unit 5.

The distance coefficient calculation unit 8 determines a coefficient Ki that indicates the relationship between the number of pixels in the vertical direction of the image and distance from the detection result of the detection unit 5 and the acquired travel distance. The distance calculation unit 4 calculates the distance L along the traveling direction X of the vehicle 100 based on the coefficient Ki and the number of pixels in the vertical direction of the image in the image. The distance calculation device 1 of this embodiment can calculate the distance along the traveling direction X based on a video captured by the monocular camera 21. Therefore, the distance calculation device 1 of this embodiment can calculate the distance with an inexpensive configuration.

The detection unit 5 of this embodiment detects a first position from a reference image. The reference image is an image in which both the first structure and the second structure are captured, for example, image G0 shown in FIG. 12. The first position is the vertical position of the pixel representing the first structure. When the detection unit 5 detects structure A as the first structure, it detects pixel position Pa as the first position. Furthermore, the detection unit 5 detects a second position from the reference image, which is the vertical position of the pixel representing the second structure. When the detection unit 5 detects structure B as the second structure, it detects pixel position Pb as the second position.

The distance factor calculation unit 8 determines the coefficient Ki in the pixel range from the first position to the second position based on the number of pixels in the vertical direction of the image from the first position to the second position and the acquired distance. If structure A is the first structure and structure B is the second structure, the distance factor calculation unit 8 determines the coefficient K1 in pixel range S1 based on the acquired distance Lab.

When three or more structures are captured in image G0, which is the reference image, the detection unit 5 and distance factor calculation unit 8 repeat the above process. For example, the detection unit 5 detects structure B as the first structure and structure C as the second structure. In this case, the detection unit 5 detects pixel position Pb as the first position and pixel position Pc as the second position. The distance factor calculation unit 8 determines the coefficient K2 in pixel range S2 from pixel position Pb to pixel position Pc based on the number of pixels in the vertical direction of the image from pixel position Pb to pixel position Pc and the acquired distance Lbc.

Furthermore, the detection unit 5 detects structure C as the first structure and structure D as the second structure. In this case, the detection unit 5 detects pixel position Pc as the first position and pixel position Pd as the second position. The distance coefficient calculation unit 8 determines a coefficient K3 in pixel range S3 from pixel position Pc to pixel position Pd based on the number of pixels in the vertical direction of the image from pixel position Pc to pixel position Pd and the acquired distance Lcd.

The first structure and the second structure do not have to be adjacent structures. For example, the detection unit 5 may detect structure A as the first structure and structure C as the second structure. In this case, the computational load in determining the coefficient Ki and calculating the distance can be reduced. On the other hand, when adjacent structures are detected as the first structure and the second structure, the image can be divided into small sections to calculate the distance, improving the accuracy of the distance calculation.

The image acquisition unit 7 of this embodiment acquires an image of the foreground of the vehicle 100 from the camera 21 mounted on the vehicle 100. The detection unit 5 detects structures A, B, C, and D arranged along the road 200 on which the vehicle 100 travels as a first structure and a second structure. It is considered that structures arranged along the road 200 are often arranged in an approximately straight line along the traveling direction X of the vehicle 100. This improves the accuracy of distance calculation.

In this embodiment, the distance coefficient calculation unit 8 obtains the travel distance of the vehicle 100 based on the time when the first structure is cut off at the edge GL of the image in the horizontal direction, and the time when the second structure is cut off at the edge GL of the image in the horizontal direction. This makes it less likely that calculation errors will occur in the travel distance.

[Modification of the embodiment]
The object for which the distance from the vehicle 100 is to be detected is not limited to the preceding vehicle 110. The object for which the distance is to be detected may be a vehicle such as a bicycle or a pedestrian, or may be a road marking such as a stop line or a pedestrian crossing displayed on the road surface of the road 200. The object for which the distance is to be detected may be a traffic light or a road sign on the road 200.

The detection unit 5 may be configured to extract a structure along the right edge of the road 200. In this case, the detection unit 5 may determine that the vehicle 100 has reached a position corresponding to the structure when the structure is cut off at the right edge of the image.

The distance calculation device 1 of this embodiment may be applied to vehicles other than automobiles, such as railway vehicles. The distance calculation device 1 may be applied to moving bodies other than vehicles.

The contents disclosed in the above embodiments and variations can be implemented in any suitable combination.

REFERENCE SIGNS LIST 1 Distance calculation device 2 Determination unit 3 Warning unit 4 Distance calculation unit 5 Detection unit 6 Distance factor recording unit 7 Image acquisition unit 8 Distance factor calculation unit 10 Distance calculation function 20 Drive recorder 21 Camera 22 Speed sensor 23 Video recording unit 24 Speed recording unit 100: Vehicle, 110: Leading vehicle 200 Road A, B, C, D Structure F _i frame G1, G2, G3, G4, Gx Image GL Horizontal end of image Ki Coefficient Lab, Lbc, Lcd, Lx Distance RA, RB, RC, RD Range Pa, Pb, Pc, Pd Position of pixel representing structure S1, S2, S3 Pixel range Ta, Tb, Tc, Td Time VA Average speed V _j Speed value Xa: Position corresponding to structure A, Xb: position corresponding to structure B;
Xc: position corresponding to structure C, Xd: position corresponding to structure D, X: direction of travel, Y: horizontal direction, Z: coordinate value in the vertical direction of the image, ΔZ: number of pixels in the vertical direction of the image, ΔT: time between frames

Claims (5)

an image acquisition unit that acquires a video image of a moving object in a traveling direction generated by a camera mounted on the moving object;
a detection unit that detects a first structure and a second structure that are located at different positions along a traveling direction of the moving object from an image of the moving image;
a distance coefficient calculation unit that acquires a moving distance of the moving object from a position corresponding to the first structure to a position corresponding to the second structure based on speed data of the moving object and a detection result of the detection unit, and determines a coefficient indicating a relationship between the number of pixels in the vertical direction of the image and the distance based on the detection result of the detection unit and the acquired moving distance;
a distance calculation unit that calculates a distance along a traveling direction of the moving object based on the coefficient and the number of pixels in a vertical direction of the image in the image;
A distance calculation device comprising: the detection unit detects a first position, which is a position in a vertical direction of an image of a pixel representative of the first structure, from a reference image in which both the first structure and the second structure are captured;
the detection unit detects, from the reference image, a second position that is a position in a vertical direction of an image of a pixel that represents the second structure;
The distance calculation device according to claim 1 , wherein the distance coefficient calculation unit determines the coefficient in the range from the first position to the second position based on the number of pixels in a vertical direction of an image from the first position to the second position and the acquired movement distance. The image acquisition unit acquires an image of a scene in front of the vehicle from the camera mounted on the vehicle,
The distance calculation device according to claim 1 , wherein the detection unit detects structures arranged along a road on which the vehicle travels as the first structure and the second structure. The distance calculation device according to claim 1 , wherein the distance coefficient calculation unit obtains the distance traveled by the moving body based on a time when the first structure is visible at a horizontal edge of the image and a time when the second structure is visible at a horizontal edge of the image. acquiring a video captured by a camera mounted on the moving object in the traveling direction of the moving object;
detecting, from images of the video, a first structure and a second structure that are located at different positions along a traveling direction of the moving object;
acquiring a moving distance of the moving object from the first structure to the second structure based on speed data of the moving object and a detection result of the detecting step;
determining a coefficient indicating a relationship between the number of pixels in the vertical direction of the image and the distance from the detection result of the detecting step and the acquired movement distance;
calculating a distance along a moving direction of the moving object based on the coefficient and the number of pixels in a vertical direction of the image in the image;
A distance calculation method comprising:

Images