CN111024040B

CN111024040B - Distance estimation method and device

Info

Publication number: CN111024040B
Application number: CN201910953997.5A
Authority: CN
Inventors: 李宰雨; 具滋厚; 李元周; 玄伦硕
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2018-10-10
Filing date: 2019-10-09
Publication date: 2023-05-30
Anticipated expiration: 2039-10-09
Also published as: CN111024040A; EP3637313A1; KR20200040374A; US11138750B2; US20200118283A1

Abstract

A distance estimation method and apparatus are provided. The method comprises the following steps: acquiring an actual target length corresponding to a target point of an object; calculating an image target length corresponding to the actual target length from the input image; and estimating a distance from the distance estimation device to the object based on the actual target length, the image target length, and a focal length of the distance estimation device.

Description

Distance estimation method and device

The present application claims priority to korean patent application No. 10-2018-010101010158, which was filed on 10 th month 10 of 2018, the disclosure of which is incorporated herein by reference in its entirety.

Technical Field

Methods and apparatus consistent with the present disclosure relate to distance estimation techniques.

Background

Active Cruise Control (ACC) technology is required in autonomous driving, such as Advanced Driving Assistance Systems (ADAS). The ACC technology is a technology for sensing the speed of a preceding vehicle in a lane in which the vehicle is currently traveling and adjusting the speed of the vehicle so that the vehicle can travel without colliding with the preceding vehicle by keeping a certain distance from the preceding vehicle.

Some vehicles currently on the market include the following functions: when a desired target speed is input to the vehicle, the vehicle is driven at the target speed when there is no preceding vehicle, and the preceding vehicle is maintained at a specific distance from the preceding vehicle by correspondingly reducing its speed when it is present. In order to realize this technology, a technology for stably measuring a distance from another vehicle is required.

Disclosure of Invention

One aspect is to provide a distance estimation method and apparatus.

According to an aspect of the embodiments, there is provided a distance estimation method, including: acquiring an actual target length corresponding to a target point of an object; calculating an image target length corresponding to the actual target length from the input image; and estimating a distance from the distance estimation device to the object based on the actual target length, the image target length, and a focal length of the distance estimation device.

According to another aspect of the embodiments, there is provided a distance estimating apparatus including: an image sensor configured to acquire an input image; and a processor configured to: acquiring an actual target length corresponding to a target point of an object; calculating an image target length corresponding to the actual target length from the input image; and estimating a distance from the distance estimation device to the object based on the actual target length, the image target length, and the focal length of the distance estimation device.

According to another aspect of the embodiments, there is provided a distance estimating apparatus including: an image sensor comprising a lens and an image plane, wherein the image sensor is configured to capture an input image of a target object; and a processor configured to: generating a projection image from the input image; estimating an actual lane width based on a lane distance between lane lines in the projection image; calculating an image lane width and an image target object width in pixels from an input image; calculating a first ratio between the image lane width and the image target object width; calculating an actual target object width based on the actual lane width and the first ratio; and estimating a distance from the image sensor to the target object based on the actual target object width, the image target object width, and a focal length from a focal point of the lens to the image plane.

Drawings

The embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings in which:

fig. 1 is a diagram showing a front object and a distance estimating apparatus according to an embodiment;

FIG. 2 is a flow chart illustrating a distance estimation method according to an embodiment;

fig. 3 is a diagram showing in detail a distance estimation process according to an embodiment;

fig. 4 is a diagram showing an estimation of an actual lane width according to an embodiment;

fig. 5 and 6 are diagrams showing estimation of a vehicle width according to an embodiment;

FIG. 7 is a top view illustrating distance estimation according to an embodiment;

fig. 8 is a block diagram showing a configuration of a distance estimation device according to an embodiment.

Detailed Description

Fig. 1 is a diagram illustrating a front object and a distance estimation apparatus according to an embodiment.

The distance estimation device 110 according to an embodiment may estimate the distance 119 from the front object 190. The distance 119 from the front object 190 may represent a straight line distance 119 from the distance estimation device 110 to the front object 190. The distance estimation device 110 may acquire an input image through an image sensor and estimate a distance 119 from the front object 190 based on the input image.

When the vanishing point changes due to camera motion (e.g., pitching motion), there may be an error in the distance estimation using the vanishing point. Such vanishing point change may occur when the vehicle passes over a deceleration strip, over a rough road, or over an inclined road. The distance estimating apparatus 110 according to the embodiment can stably estimate the distance by using the geometric relationship between the object width and the lane width, regardless of the vanishing point change, even if the ground is curved.

For example, the geometric relationship between the width of the object appearing in the input image and the width of the lane may be the same as or similar to the relationship between the width of the object and the width of the lane in the real world. Accordingly, the distance estimation device 110 may estimate the vehicle width in the real world based on the relationship between the width of the object appearing in the input image and the lane width. The distance estimation device 110 may estimate the distance 119 from the front object 190 based on the vehicle width in the real world. Hereinafter, with reference to fig. 2 to 8, a technique of stably estimating the distance 119 from the front object 190 by only a single input image will be described.

For reference herein, a roadway 101 may include one or more lanes 102. The lane line 103 may represent a boundary line between one lane 102 and another lane 102. However, the inventive concept is not limited to the case where the lane line 103 must be in the road 101 or the lane 102. The driving lane 102 may represent the lane 102 in which the vehicle is currently driving. As one type of lane line 103, a center line (not shown) may represent a boundary line indicating that the vehicle is prohibited from entering.

Fig. 2 is a flowchart illustrating a distance estimation method according to an embodiment.

First, in operation 210, the distance estimation apparatus may acquire an actual target length corresponding to a target point of an object. The target point may be a specific point specified in the object, and may represent two points specified as references for distance estimation. For example, the actual target length may be a length corresponding to the physical width of the object, and the target point may be two points designated for setting the width of the object, and may be a leftmost point of the object along the horizontal axis with respect to the vertical axis and a rightmost point parallel to the leftmost point. In the case of a three-dimensional (3D) bounding box having the real dimensions covering objects in the world, each vertex of the front or back of the bounding box may correspond to a target point. However, the inventive concept is not limited thereto, and as a distance between feature points in the visual appearance of the object, for example, in the case where the object is a vehicle, a point (for example, a center point) corresponding to two rear vehicle lamps may be set as the target point, or a point (for example, a center point) corresponding to two front vehicle lamps may be set as the target point.

Additionally or alternatively, the target point may be set for height instead of width. For example, when the object is a vehicle, a point corresponding to the bottom of the vehicle and a point corresponding to the top of the vehicle may be set as the target points. In this case, the actual target length may represent a height of the vehicle (e.g., a vehicle height).

The acquisition of the actual target length will be described below with reference to fig. 5 and 6.

In operation 220, the distance estimation apparatus may calculate an image target length corresponding to the actual target length from the input image. The image target length may represent a length corresponding to a target point in the input image. The distance estimation device may detect a bounding box including the object from the input image and determine the image target length based on the detected bounding box. For example, the image target length may represent a length between vertices of a bounding box comprising the object. When the bounding box is a two-dimensional (2D) bounding box, the image object length may be the length of the bottom line (base line) of the bounding box. When the bounding box is a 3D bounding box, the image target length may be the length of the bottom line that constitutes the back of the bounding box. For example, for a central axis of the road, the back side of the bounding box may represent the surface facing the distance estimation device. In other words, among the plurality of faces of the bounding box, a face located closer to the distance estimation device with respect to the center axis of the road may be referred to as a back face, and a face located farther from the distance estimation device may be referred to as a front face. The central axis of the road may be an axis defined along the central point of the lane where the distance estimation device is located, and may appear as a straight line in a straight road and a curve in a curved road.

For example, the input image may be a color image. The color image may include a plurality of color channel images. For example, the color channel images may include a red channel image, a green channel image, and a blue channel image. The pixels of each color channel image may represent the intensity of light corresponding to the wavelength of the color in the light received by the sensor. However, the input image is not limited thereto.

Subsequently, in operation 230, the distance estimating device may estimate a distance from the distance estimating device to the object based on the actual target length of the object, the image target length, and the focal length of the distance estimating device. For example, the distance estimation device may estimate the distance to the object from the focal length based on the relationship between the actual target length and the image target length. This is because the proportional relationship between the actual target length and the image target length is the same as the proportional relationship between the distance from the object and the focal length. Such distance estimation will be described below with reference to fig. 7.

Fig. 3 is a diagram showing in detail the distance estimation process according to the embodiment.

In operation 301, the distance estimation device may acquire a camera image. For example, the distance estimation device may receive camera images through one or more image sensors. The image sensor may be mounted to capture an image at the front side of the distance estimation device. The image sensor may capture a scene corresponding to a viewing angle of the front side. However, the number of image sensors and the mounting positions thereof are not limited thereto.

In operation 311, the distance estimation device may perform a projection image operation. For example, the distance estimation device may generate a projection image by projecting an input image on the ground. The distance estimation device may generate the projection image by homography. For example, the projected image may correspond to a bird's eye view.

In operation 312, the distance estimation device may detect a lane width around the distance estimation device. The distance estimating apparatus according to the embodiment may estimate a lane line of a lane in which the object is located in the input image. The distance estimation device may determine the image lane width based on the estimated lane lines. Here, the image lane width may be a length corresponding to the width of a lane detected from the input image, and may be, for example, a pixel distance between one lane line and another lane line, wherein the pixel distance defines a lane identified in the input image. The determination of the image lane width will be described below with reference to fig. 5.

For example, the distance estimation apparatus may detect a lane line by interpolating a portion corresponding to the same line in the input image. For example, the lane lines may be dotted lines, wherein portions corresponding to the lines are disposed at certain intervals. Even when the lane line is a broken line, the distance estimation apparatus can detect the lane line by converting the broken line into a solid line through the above interpolation.

In operation 313, the distance estimation apparatus may detect a lane line around the object. For example, the distance estimation apparatus may estimate the actual lane width based on a projection image obtained by projecting an input image on the ground. For example, the actual lane width may be estimated in meters. The distance estimation device may identify lane lines in the projection image and estimate a horizontal interval between the lane lines. The horizontal spacing between lane lines may correspond to an actual lane width. Lane line recognition and estimation of an actual lane width using the projection image will be described below with reference to fig. 4.

However, the acquisition of the actual lane width is not limited thereto. In response to a case where High Definition (HD) map data can be accessed, the distance estimation apparatus may acquire an actual lane width of a lane in which the object is located from map data corresponding to the current road. For example, the distance estimation apparatus may acquire lane information corresponding to geographic coordinates where the object is located from the HD map data, and extract an actual lane width from the lane information.

In operation 314, the distance estimation device may detect an actual width of the target object. For example, the distance estimation device may calculate the actual target length based on the image target length and the lane line appearing in the input image. The distance estimation apparatus may calculate a ratio between an image target length and an image lane width corresponding to a lane in which the object is located from the input image. The distance estimation device may estimate the actual target length from the actual lane width based on the ratio. The estimation of the actual target length will be described below with reference to fig. 5.

In operation 321, the distance estimation device may detect a bounding box of the target object. For example, the distance estimation device may detect a bounding box covering the object from the input image. The distance estimation device may detect the bounding box by using one of various algorithms. For example, the distance estimation device may use a neural network to detect a bounding box that includes an area corresponding to an object in the input image. The neural network may be trained to output from the image a bounding box region corresponding to an object (e.g., a vehicle) to be detected. The bounding box may represent a 2D box or a 3D box that includes the object. The bounding box may have a particular shape (e.g., rectangular or cuboid) and may represent a box that includes space occupied by objects in 2D space or 3D space.

For example, each edge of the 2D bounding box may contact a portion of the object, and the 2D bounding box may be a smallest bounding box defined to minimize the size of the 2D bounding box. The top edge of the 2D bounding box may contact the top portion of the object appearing in the input image and its bottom edge may contact the bottom portion of the object. Each face of the 3D bounding box may contact a portion of the object, and the 3D bounding box may be a smallest bounding box defined to minimize the size of the 3D bounding box. When the object is a vehicle, the front of the vehicle may contact the front of the 3D bounding box and the rear of the vehicle may contact the back of the 3D bounding box. An upper portion of the vehicle may contact a top surface of the 3D bounding box and a lower portion of the vehicle may contact a bottom surface of the 3D bounding box. The sides of the vehicle may contact the sides of the 3D bounding box.

In operation 322, the distance estimation device may track the bounding box. According to one embodiment, the distance estimation device may track objects in the input image acquired in each frame. The distance estimation device may determine a location in the current frame for the bounding box detected in the previous frame. The distance estimation device may store the result of tracking the bounding box in the object information database 340. For example, in response to having estimated an actual target length (e.g., an actual vehicle width) for a bounding box corresponding to an object in a previous frame, the distance estimation device may load the actual target length of the previous frame into the current frame from the object information database 340. The distance estimation device may track the object assigned the previous target length acquired in the previous frame. The distance estimation device may determine a current target length of a current frame for the tracked object as a previous target length acquired in a previous frame. Therefore, even when it is difficult to estimate the actual target length in the current frame, the distance estimation apparatus can stably acquire the actual target length corresponding to the object. Here, the frame may represent a time point, and when the current frame is assumed to be a t-th time point, the previous frame may represent any one of the 1 st time point to the (t-1) th time point. Here, "t" may represent an integer greater than or equal to 2.

The distance estimation device may estimate a previous target length of the object in a previous frame based on the additional sensor. The additional sensor may be another sensor used with the image sensor and may include, for example, a radar sensor or a LiDAR sensor. The distance estimation device may assign the estimated previous target length to the object. Thus, when the target length estimation using the additional sensor cannot be performed in the current frame, the distance estimation apparatus may use the actual target length previously acquired in the previous frame.

Object information database 340 may represent a database that includes information associated with objects. For example, the object information database 340 may include an Identification (ID) of the object, a size of the bounding box, a number of bounding boxes, and an actual target length estimated in a previous frame for the object.

The object information database 340 may include object size information mapped for each object model (e.g., vehicle model). The object size information may be information about the size of an object corresponding to a specific model, and may include, for example, the width, height, and length of the object. The distance estimation apparatus according to the embodiment may determine object size information corresponding to an object based on visual appearance of the object appearing in the input image. The distance estimation device may obtain the actual target length from the object size information.

In operation 331, the distance estimating apparatus may extract the distance. For example, the distance estimation device may calculate the distance from the focal length based on a ratio between the actual target length and the image target length. The distance calculation will be described in detail below with reference to fig. 7.

Fig. 4 is a diagram showing an estimation of an actual lane width according to an embodiment.

The distance estimation apparatus 491 according to the embodiment may estimate the actual lane width 421 by using the input image 410 captured at the front side thereof. Although the input image 410 shown in fig. 4 includes one vehicle as the front object 492 for convenience of description, the number and shape of the front objects 492 are not limited thereto. The distance estimation device 491 may be mounted on a vehicle and the input image 410 may include a portion of the vehicle (e.g., a hood). However, the inventive concept is not limited thereto, and a portion of the vehicle may appear in the input image 410 according to the installation position and the viewing angle of the image sensor.

The distance estimation device 491 may generate a projection image 420 by projecting the input image 410. For example, the distance estimation device 491 may generate a projection image 420 corresponding to the bird's eye view. The distance estimation device 491 may convert the input image 410 into the projection image 420 by a matrix operation (e.g., homography matrix calculation) that converts the coordinates of each pixel of the input image 410 into coordinates on the projection image 420. The coordinates of the region closer to the vehicle in the projected image 420 may be more accurate because the influence of the movement of the vehicle or the height of the ground is smaller relative to the region immediately in front of the vehicle.

The distance estimation device 491 may identify a lane line in the projection image 420. The distance estimation device 491 may calculate physical coordinates (e.g., 2D coordinates) of a position corresponding to the lane line. The distance estimation device 491 may calculate the horizontal interval between two parallel lane lines by using the physical coordinates of the lane lines. The distance estimation device 491 may estimate the actual lane width 421 based on the interval between the identified lane lines. The projected image 420 generated by the projected image operation corresponding to the bird's eye view may represent accurate coordinates with respect to the area closer to the sensor. Therefore, in order to more accurately estimate the lane width, the distance estimation device 491 may use information of an area closer to the distance estimation device 491 in the projection image 420.

For example, the distance estimation device 491 may identify a lane line corresponding to a lane in which the object 492 is located in the projection image 420. The distance estimation device 491 may estimate the actual lane width 421 based on a horizontal distance between boundary lines within a threshold distance from the distance estimation device 491 among the identified lane lines. Here, the actual lane width 421 may represent an actual width of a lane in the physical world.

The distance estimation device 491 according to the embodiment may assume that the lane width at the position of the object 492 is the same as or similar to the lane width around the distance estimation device 491. Thus, as described above, the distance estimation device 491 may estimate the actual lane width 421 of the lane around the object 492 based on the projection image 420 around the distance estimation device 491.

As shown in fig. 4, a lane line of another lane on the projected image 420 may be identified at a location spaced apart from the distance estimating device 491 due to the perspective of the image sensor. Thus, the distance estimation device 491 may also estimate the actual lane width 421 of the lane of the object located in a lane different from the distance estimation device 491 itself.

In response to a situation in which the point in the current frame at which the lane line corresponding to the lane in which the object 492 is located is identified as being closer than the point identified in the previous frame, the distance estimation device 491 may update the actual lane width 421 based on the point in the current frame at which the lane line is identified. Therefore, since the lane width can be calculated more accurately by using the information of the area closer to the distance estimation device 491 in the projection image 420, the distance estimation device 491 can update the actual lane width 421 more accurately.

According to another embodiment, the distance estimation device 491 may estimate the actual lane width 421 only when the width between the lane width at the location of the object 492 and the lane width around the device does not change much. For example, in response to a situation in which the change in the width of the lane in which the object 492 is located exceeds a threshold change, the distance estimation device 491 may exclude an estimation of the actual lane width 421. When excluding the estimation of the actual lane width 421, the distance estimation device 491 may acquire the estimation of the actual lane width 421 or the actual target length of the excluded object by another method.

Fig. 5 and 6 are diagrams showing estimation of a vehicle width according to an embodiment.

Fig. 5 is a diagram showing a process of estimating an actual target length in a state where a vehicle mounted with a distance estimating apparatus travels in a general road.

The distance estimation device may detect a front object 592 on the front side of the distance estimation device from the input image 510. For example, the distance estimation device may detect a bounding box 582 that includes the front object 592. Fig. 5 shows an input image 510 in which a bounding box for each of three front objects is detected in the input image 510. Fig. 5 shows an example of detecting bounding box 582 in 2D form. The magnified image 511 is a magnified partial image that includes a bounding box 582.

The distance estimation device may determine the image target length and the interval between the lane lines 503 from the input image 510.

For example, the distance estimation device may determine the image target length from the bounding box 582 detected from the input image 510. The distance estimation device may determine a pixel distance corresponding to a width of the object as the image target length. The pixel distance corresponding to the width of the object may correspond to the width of bounding box 582. The distance estimation device may determine the length of the bottom line or upper edge in the bounding box 582 as a pixel distance corresponding to the width of the object.

The distance estimation device may determine the image lane width based on the bounding box 582 detected from the input image 510. The distance estimation device may determine the image lane width based on the intersection between the lane line 503 and the extension of the bottom line of the bounding box 582. When the front object 592 is a vehicle, it is assumed that a vehicle that remains in a lane of travel generally travels parallel to the lane line 503 along the center axis of the lane. In this case, the longitudinal axis of the front vehicle may be parallel to the lane line 503, and the transverse axis of the front vehicle may be perpendicular to the lane line 503. Accordingly, since the bottom line of the bounding box 582 is parallel to the lateral axis of the preceding vehicle, the length between the intersection between the lane line 503 and the extension of the bottom line may correspond to the image lane width. The distance estimation device may determine the pixel distance between the intersection between the lane line 503 and the extension line of the bottom line as the image lane width.

Subsequently, the distance estimation device may estimate the actual target length based on the image lane width and the image target length calculated as described above. For example, the distance estimation device may calculate a ratio between the image target length and the image lane width corresponding to the lane in which the object 592 is located from the input image 510. The distance estimation device may calculate a ratio between a pixel distance corresponding to the interval between lane lines 503 in the input image 510 and a pixel distance corresponding to the width of the object 592. The distance estimation apparatus may estimate the actual target length from the actual lane width based on the above ratio according to equations 1 and 2 below.

Equation 1

Ratio _img ＝(W _img ，V/W _img，L )

Equation 2

W _real，V ＝Ratio _img ·Wr _eal，L

In equation 1, W _img,V Image target length in pixels (e.g., width of vehicle appearing in image) W can be represented _img,L The image lane width in pixels may be represented. Ratio _img Can represent the target length W of the image _img,V Width W of image lane _img,L The ratio between. In equation 2, W _real,L The actual lane width may be represented. For example, the distance estimation apparatus may acquire the actual lane width W by the operation described above with reference to fig. 4 _real,L 。W _real,V An actual target length (e.g., an actual width of the vehicle) may be represented.

For example, when the image target length W _img,V Is a distance corresponding to 15 pixels, and an image lane width W _img,L When the distance is 20 pixels, the Ratio is _img May be 0.75. When the actual lane width W is calculated by homography in FIG. 4 _real,L At 4 meters, the actual target length W _real,V May be 3 meters.

The actual target length W has been estimated for the object 592 at a close distance to the distance estimation device _real,V The distance estimation device may maintain the previously estimated actual target length W as the object 592 moves away from the distance estimation device _real,V While continuing to track object 592. This is because the value estimated at a close distance may be more accurate than the value estimated at a far distance. By maintaining the previously estimated actual target length W _real,V The distance estimating device can stably estimate the distance to the object 592 even when the object 592 is distant from the distance estimating device and thus the lane line 503 around the object 592 is not recognized or the lane width around the object 592 is not constant. Further, in response to the distance from object 592 being less than the pre-estimated actual target length W _real,V The distance estimation device may update the actual target length W of the object 592 _real,V . This is because, when the object 592 is located in a closer position, the actual target length W _real,V Can be estimated more accurately.

Although fig. 5 has been described with reference to a straight road, the inventive concept is not limited thereto. Even when traveling on a curved road, the distance estimation apparatus may detect the 2D bounding box and estimate the actual target length as described with reference to fig. 5.

Fig. 6 is a diagram showing a process of estimating an actual target length in a state where a vehicle mounted with a distance estimating apparatus is traveling on a curved road.

The overhead view image 620 shows a state in which the vehicle mounted with the distance estimation device 691 travels on the curved road 601. Although only one lane of the curved road 601 is shown in the top view image 620, this is merely for convenience of description and the inventive concept is not limited thereto. The input image 610 may include a front object 692 that is distorted when the front object 692 travels in a curved portion of the curved road 601. Accordingly, the distance estimation device 691 may more accurately estimate the actual target length of the object by considering the pose (post) of the object 692.

The distance estimation device 691 according to an embodiment may estimate the pose of the object 692 from the input image 610. The distance estimation device 691 may determine the image target length 671 based on the estimated pose. For example, the distance estimation device 691 may detect a 3D bounding box 682 corresponding to the object 692 from the input image 610. The distance estimation device 691 may determine the image target length 671 based on the 3D bounding box 682. The distance estimation device 691 may determine a width of the 3D bounding box 682 (e.g., a length of an upper or bottom line on the back of the box) as the image target length 671.

The distance estimation device 691 may determine the image lane width 672 based on the pose of the object 692 in the input image 610. For example, the distance estimation device 691 may determine the image lane width 672 based on the intersection between the lane line and the extension line of the bottom line that constitutes the back surface of the 3D bounding box 682.

The distance estimating device 691 may estimate the actual target length according to equations 1 and 2 based on the image target length 671 and the image lane width 672 determined according to the description with reference to fig. 6.

Although fig. 6 has been described with reference to the curved road 601, the inventive concept is not limited thereto. Even when traveling on the straight road 601, the distance estimation device 691 can detect the 3D bounding box 682 and estimate the actual target length as described with reference to fig. 6.

Further, the distance estimation apparatus may identify linearity of the travel road and determine the distance estimation method according to the identified linearity. For example, in response to a case where the current traveling road is identified as a straight road (e.g., a road having a curvature smaller than a threshold curvature), the distance estimation apparatus may detect a 2D bounding box from the input image and estimate the actual target length as described above with reference to fig. 5. As another example, in response to a case where the current travel road is identified as a curved road (e.g., a road having a curvature equal to or exceeding a threshold curvature), the distance estimation apparatus may detect the pose of the 3D bounding box or object from the input image and estimate the actual target length as described with reference to fig. 6. However, this is merely an example, and the operation of the distance estimation apparatus is not limited thereto.

Fig. 7 is a top view illustrating distance estimation according to an embodiment.

Fig. 7 is a top view showing an object 790, a focus 711, and an image plane 710 of an image sensor of a distance estimation device. The image plane 710 may be a plane corresponding to a point where light passing through the lens is received in the image sensor, and each point of the image plane 710 may correspond to each pixel of the input image. Light reflected and received from an object 790 external to the distance estimation device may pass through the focal point 711 of the lens and reach the image plane 710. The positional relationship between the focal length f and the distance D may be related to the image target length W appearing on the image plane 710 _img,V And an actual target length W _real,V The positional relationship between them is the same.

Equation 3

Thus, the distance estimation device may be based on the actual target length W according to equation 3 _real,V And image target length W _img,V The ratio between, distance D is calculated from the focal length f. The distance D may represent the distance from the focal point 711 of the lens to the object 790 in the image sensor.

The distance estimation device 800 according to an embodiment may include a sensor 810, a processor 820, and a memory 830.

The sensor 810 may be an image sensor 810 for acquiring an input image of a front side thereof. The image sensor 810 may generate a color image, and the color image may include a plurality of color channel images. However, the inventive concept is not limited thereto, and the image sensor 810 may generate a monochrome image (e.g., a black-and-white image).

Processor 820 may obtain an actual target length corresponding to a target point of an object. Processor 820 may calculate an image target length corresponding to the actual target length from the input image. Processor 820 may estimate the distance from distance estimation device 800 to the object based on the actual target length of the object, the image target length, and the focal length of distance estimation device 800. However, the operation of the processor 820 is not limited thereto, and the processor 820 may perform the operations described with reference to fig. 1 to 7.

The memory 830 may temporarily or permanently store data for performing the distance estimation method. For example, the memory 830 may store an actual target length estimated in a previous frame, an actual lane width, a distance from an object in the previous frame, and the like.

The above-described devices may be implemented as hardware components, software components, and/or a combination of hardware and software components. For example, the devices and components described in the embodiments may be implemented using one or more general purpose or special purpose computers, such as a processor, controller, arithmetic Logic Unit (ALU), digital signal processor, microcomputer, field programmable array (FPGA), programmable Logic Unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processor may execute an Operating System (OS) and one or more software applications executing on the OS. Further, in response to execution of the software, the processor may access, store, manipulate, process, and generate data. For ease of understanding, a processor may be described as being used alone, however, one of ordinary skill in the art will appreciate that a processor may include multiple processing elements and/or multiple types of processing elements. For example, a processor may include multiple processors or one processor and one controller. In addition, other processing configurations (such as parallel processors) are possible.

The software may include a computer program, code, instructions, or a combination of one or more of them, and may configure the processor to operate as desired or may command the processor individually or collectively. For purposes of explanation by the processor or for providing commands or data to the processor, the software and/or data may be permanently or temporarily contained in any type of machine, component, physical device, virtual equipment, computer storage medium or device or transmitted signal wave. The software may be distributed over networked computer systems so that it is stored and executed in a distributed fashion. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands executable by various computer devices, wherein the program commands may be recorded on a computer-readable recording medium. The computer readable recording medium may include program commands, data files, and data structures alone or may include a combination thereof. The program command recorded on the computer-readable recording medium may be a program command specially designed and configured for the embodiment, or may be a program command known and available to a computer programmer in the art. Examples of the computer-readable recording medium may include magnetic recording media (such as hard disks, floppy disks, and magnetic tapes); optical recording media such as compact disk read only memory (CD-ROM) and Digital Versatile Disks (DVD); magneto-optical recording media (such as a soft optical disc); and hardware devices that are specially configured to store and perform program commands (such as ROM, RAM, and flash memory). Examples of program commands may include machine language code, which may be generated by a compiler, and high-level language code, which may be executed by a computer using an interpreter. The hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments and vice versa.

Although the embodiments have been described with reference to the accompanying drawings, various changes and modifications may be made by one of ordinary skill in the art without departing from the spirit and scope of the inventive concept. For example, the described techniques may be performed in a different order than described, and/or components described (such as systems, structures, devices, and circuits) may be combined or combined in a different manner than described, or may be replaced or substituted with other components or equivalents thereof.

Accordingly, other implementations, other examples, and equivalents of the claims appended hereto are within the scope of the claims.

While the present inventive concept has been particularly shown and described with reference to embodiments thereof, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the appended claims.

Claims

1. A distance estimation method, comprising:

acquiring an actual target length corresponding to a target point of an object;

calculating an image target length corresponding to the actual target length from the input image; and

estimating a distance from the distance estimating device to the object based on the actual target length, the image target length and a focal length of the distance estimating device,

the step of obtaining the actual target length comprises the following steps: an actual target length is calculated based on the image target length and the lane lines appearing in the input image,

wherein the step of calculating the actual target length comprises:

calculating a ratio between an image target length and an image lane width corresponding to a lane in which the object is located from the input image; and

an actual target length is estimated from the actual lane width based on the ratio,

wherein the step of calculating the ratio comprises:

estimating a lane line of a lane in which the object is located in the input image; and

an image lane width is determined based on the estimated lane lines,

wherein the step of calculating the target length of the image comprises:

detecting a bounding box corresponding to the object from the input image; and

the image target length is determined from the bounding box,

wherein the image lane width is determined based on an intersection between the lane line and an extension of the bottom line of the bounding box.

2. The distance estimation method according to claim 1, wherein the step of estimating the lane line includes: the lane line is detected by interpolating a portion corresponding to the same line in the input image.

3. The distance estimation method of claim 1, wherein the step of calculating the image target length comprises:

estimating a pose of the object from the input image; and

the image target length is determined based on the pose,

wherein the image lane width is determined based on the pose of the object in the input image.

4. The distance estimation method of claim 1, wherein the step of calculating the image target length comprises:

detecting a three-dimensional bounding box corresponding to the object from the input image; and

the image target length is determined based on the three-dimensional bounding box,

wherein the image lane width is determined based on an intersection between the lane line and an extension line of a bottom line constituting the back surface of the three-dimensional bounding box.

5. The distance estimation method of claim 1, wherein the step of estimating the actual target length comprises: the actual lane width is estimated based on a projection image obtained by projecting an input image on the ground.

6. The distance estimation method according to claim 5, wherein the step of estimating the actual lane width includes:

identifying a lane line corresponding to a lane in which the object is located in the projected image; and

an actual lane width is estimated based on a horizontal interval between boundary lines within a threshold distance from the distance estimation device among the identified lane lines.

7. The distance estimation method of claim 1, wherein the step of estimating the actual target length comprises: the actual lane width of the lane in which the object is located is acquired from the map data corresponding to the current road.

8. The distance estimation method according to claim 1, wherein the step of calculating the ratio includes: a pixel ratio between a first pixel distance corresponding to an interval between lane lines in an input image and a second pixel distance corresponding to a width of an object is calculated.

9. The distance estimation method of claim 1, wherein the step of estimating the actual target length comprises: in response to identifying in the current frame that a first point of a lane line corresponding to a lane in which the object is located is closer than a second point identified in the previous frame, the actual lane width is updated based on the first point.

10. The distance estimation method according to claim 1, wherein the step of calculating the ratio includes: in response to a change in the width of the lane in which the object is located exceeding a threshold change, the estimated actual target length of the object is excluded.

11. The distance estimation method according to claim 1, wherein the step of acquiring the actual target length includes:

tracking an object allocated with a previous target length acquired in a previous frame; and

the current target length in the current frame of the tracked object is determined as the previous target length acquired in the previous frame.

12. The distance estimation method of claim 11, wherein the step of tracking the object comprises:

estimating a previous target length of the object in a previous frame based on the additional sensor; and

the estimated previous target length is assigned to the object.

13. The distance estimation method according to claim 1, wherein the step of acquiring the actual target length includes:

determining object size information corresponding to the object based on a visual appearance of the object represented in the input image; and

the actual target length is obtained from the object size information.

14. The distance estimation method according to claim 1, wherein the step of estimating the distance comprises: the distance is calculated from the focal length based on a ratio between the actual target length and the image target length.

15. A non-transitory computer-readable recording medium having recorded thereon at least one computer program comprising at least one instruction for performing the distance estimation method of claim 1.

16. A distance estimation device comprising:

an image sensor configured to acquire an input image; and

a processor configured to:

acquiring an actual target length corresponding to a target point of an object;

estimating a distance from the distance estimating device to the object based on the actual target length, the image target length and the focal length of the distance estimating device,

the process of obtaining the actual target length comprises the following steps: an actual target length is calculated based on the image target length and the lane lines appearing in the input image,

wherein the process of calculating the actual target length includes:

wherein the process of calculating the ratio comprises:

an image lane width is determined based on the estimated lane lines,

wherein the process of calculating the target length of the image comprises:

detecting a bounding box corresponding to the object from the input image; and

the image target length is determined from the bounding box,

17. A distance estimation device comprising:

an image sensor comprising a lens and an image plane, wherein the image sensor is configured to capture an input image of a target object; and

a processor configured to:

generating a projection image from the input image;

estimating an actual lane width based on a lane distance between lane lines in the projection image;

calculating an image lane width and an image target object width in pixels from an input image;

calculating a first ratio between the image lane width and the image target object width;

calculating an actual target object width based on the actual lane width and the first ratio; and

based on the actual target object width, the image target object width, and the focal length from the focal point of the lens to the image plane, the distance from the image sensor to the target object is estimated,

wherein the process of calculating the first ratio includes:

estimating a lane line of a lane in which a target object is located in an input image; and

an image lane width is determined based on the estimated lane lines,

wherein the process of calculating the width of the image target object includes:

detecting a bounding box corresponding to the target object from the input image; and

the image target object width is determined from the bounding box,

18. The distance estimation device according to claim 17, wherein the projection image is generated by homography matrix calculation.