JP2024065130A - Information processing device, information processing method, and program - Google Patents

Abstract

[Problem] To achieve highly accurate distance measurement using a single eye. [Solution] An information processing device includes a storage unit and a processing unit. The processing unit acquires an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction, detects at least a portion of a plurality of straight lines extending upward from a lower end in the second direction of the image with an inclination within a predetermined angle range, acquires virtual straight lines from at least a portion of the detected multiple straight lines, acquires actual distances between the virtual straight lines, and measures the distance to the object from the position of the object relative to the virtual straight lines in the image and the width between the virtual straight lines at the position of the object. [Selected Figure] Figure 1

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

Distance measurement is a necessary technology in a wide range of fields, including in-vehicle cameras and robot control, and various research and development efforts are underway. This distance measurement technology is often implemented using images captured by stereo cameras. This is because highly accurate measurements can be achieved by measuring distances using two or more cameras.

On the other hand, when using two or more cameras, there is a problem with the placement of the capture side, since the cameras are placed at positions separated by a certain distance. The use of multiple cameras requires calibration, which must be changed depending on conditions such as vibration and temperature, and the changes in calibration must be applied appropriately. In addition, since data obtained from multiple cameras must be processed at the same time, a reduction in processing is also required in processors that execute information processing. Another problem is that the cost of the cameras themselves increases by that much when multiple cameras are installed in a distance measuring device.

JP 2007-240316 A

Therefore, this disclosure provides an information processing device that can achieve highly accurate distance measurement using a single eye.

According to one embodiment, the information processing device includes a storage unit and a processing unit. The processing unit acquires an image having pixels in an array in a first direction and a second direction intersecting the first direction, detects at least a portion of a plurality of straight lines extending upward from a lower end in the second direction of the image with an inclination within a predetermined angle range, acquires virtual straight lines from at least a portion of the detected plurality of straight lines, acquires actual distances between the virtual straight lines, and measures the distance to the object from the position of the object relative to the virtual straight lines in the image and the width between the virtual straight lines at the position of the object.

The processing unit may obtain a ratio between the width of the virtual straight line in the first direction at the bottom end of the second direction in the image and the width of the actual straight lines based on the point of intersection of the virtual straight line and the bottom end.

The processing unit may acquire the coordinate value of the bottom end of the object in the second direction, and calculate the distance to the object based on the ratio between the width in the first direction between the virtual straight lines at the coordinate value of the bottom end of the object in the second direction and the width in the first direction of the virtual straight line at the bottom end of the image.

The processing unit may calculate the distance to the target by assuming that the width of the virtual straight line is a constant value.

The processing unit may detect multiple vanishing points from at least a portion of the multiple straight lines, and if the multiple vanishing points are not within a predetermined range, detect that there is a point where the slope changes between the straight lines and the target.

When the multiple vanishing points are not within a predetermined range, the processing unit may calculate the degree of change in the tilt from the deviation in the positions of the multiple vanishing points.

The processing unit may further measure the distance to the object based on the change in tilt.

The processing unit may correct the width between the virtual straight lines at the position of the target based on an HD map (high-precision three-dimensional map data) and calculate the distance to the target.

The processing unit may measure the actual width of the object based on the width of the object relative to the width of the virtual straight line in the image.

The image may be an image acquired as a frame of video information, and the processing unit may further perform at least one process using images of one or more past frames of the image.

At least a portion of the straight line to be detected may be a white line on the road.

At least a portion of the straight line to be detected may be the upper or lower end of the guardrail.

At least a portion of the straight line to be detected may be a road edge.

The subject may be a human or an animal.

The object may be a car.

The object may be an obstacle.

According to one embodiment, the information processing method includes a processing unit that acquires an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction, detects at least a portion of a plurality of straight lines extending upward from a lower end in the second direction of the image with an inclination within a predetermined angle range, acquires virtual straight lines from at least a portion of the detected plurality of straight lines, acquires actual distances between the virtual straight lines, and measures the distance to the object from the position of the object in the image relative to the virtual straight lines and the width between the virtual straight lines at the position of the object.

The information processing method may include at least one step of the method for executing each step of the information processing device described above.

According to one embodiment, when executed by a computer, the program acquires an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction, detects at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range, acquires virtual straight lines from at least a portion of the detected plurality of straight lines, acquires actual distances between the virtual straight lines, and measures the distance to the object from the position of the object in the image relative to the virtual straight lines and the width between the virtual straight lines at the position of the object.

The program may cause a computer to execute a method including at least one of the methods for executing each step of the information processing device described above.

FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to an embodiment. A graph showing camera pitch and vertical misalignment. FIG. 13 is a diagram illustrating obtaining a target position from a reference width according to an embodiment. FIG. 13 is a diagram showing acquisition of a virtual straight line according to an embodiment. 10 is a flowchart showing a process of an information processing device according to an embodiment. FIG. 1 is a block diagram showing a configuration of an information processing device according to an embodiment. FIG. 13 illustrates measuring a width of interest according to one embodiment. FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to an embodiment. FIG. 13 illustrates gradient change detection according to an embodiment. FIG. 13 illustrates a calculation of gradient variation according to an embodiment. FIG. 1 is a block diagram showing a configuration of an information processing apparatus according to an embodiment. FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit.

Below, the embodiments of the present disclosure will be explained with reference to the drawings. The drawings are used for explanatory purposes, and the shape, size, or size ratio of each component in the actual device to other components need not be as shown in the drawings. In addition, since the drawings are simplified, components necessary for implementation other than those shown in the drawings are to be appropriately included. In addition, while the words "more than" and "less than" may be used in the specification, etc., these can be appropriately interpreted as "greater (higher, more)" and "smaller (lower, less)," and vice versa.

The following description will be given using an on-board device for an automobile, but the scope of application of this application is not limited to on-board devices for automobiles. For example, it can also be applied to drones, robots, etc.

(First embodiment)
1 is a block diagram showing a configuration of an information processing device according to an embodiment. The information processing device 1 includes an imaging unit 100, a storage unit 102, an image processing unit 104, a virtual line acquisition unit 106, a reference width acquisition unit 108, and a distance calculation unit 110. The information processing device 1 measures the distance to a target obstacle based on image or video information acquired from the imaging unit 100. The target may be a person, an animal, a car, an obstacle, or the like, but is not limited thereto. For example, a person riding a bicycle may also be the target.

The imaging unit 100 is configured to include, for example, a camera. The camera is mounted on the vehicle to capture, for example, the state in front of the vehicle as image and video information. In FIG. 1, the imaging unit 100 is provided in the information processing device 1, but this is not limiting. For example, the imaging unit 100 may be provided outside the information processing device 1. The imaging unit 100 has light-receiving pixels in an array in a first direction and a second direction intersecting the first direction, and captures an image having pixels in an array in the first direction and the second direction based on signals that photoelectrically convert light received by the light-receiving pixels.

The storage unit 102 stores information acquired by the imaging unit 100 and information necessary for distance measurement. When at least some of the functions of the information processing device 1 are realized by software information processing using hardware resources, the storage unit 102 may store programs, executable files, etc. related to this software. The storage unit 102 may include a non-temporary storage area. The storage unit 102 may also include a temporary storage area such as a frame buffer.

The image processing unit 104 performs appropriate image processing on the images, video information, etc. acquired by the imaging unit 100. The image processing unit 104 corrects, for example, distortion caused by the optical system of the imaging unit 100. The distortion correction may be performed using parameters, etc., that are set in advance by the imaging unit 100. These parameters, etc. may be updated at any timing or at a specified timing.

The virtual line acquisition unit 106 extracts a plurality of lines, half lines, line segments, etc. that run from the bottom to the top of the image processed by the image processing unit 104. For example, the virtual line acquisition unit 106 extracts line segments that exist in the bottom of the image (for example, this may be a predetermined area such as the bottom half or the bottom 1/4 of the image, or another area may be set). The extracted line segments, etc. may be limited to those whose inclination in the horizontal and vertical directions is within a predetermined angle range (for example, 60° to 120°, 70° to 110°, etc., but is not limited to these). The predetermined angle range may be a range that depends on the left and right positions of the image. Based on the extracted line segments, the virtual line acquisition unit 106 acquires virtual lines. Details of this process will be described later with reference to the drawings.

The reference width acquisition unit 108 acquires the ratio between the width between the virtual straight lines and the width between the straight lines in the actual state in front of the vehicle based on the width of the virtual straight line at the bottom edge of the image. The reason for acquiring the reference width ratio at the bottom edge of the image is that the image showing the road is less affected by the pitch direction of the camera of the imaging unit 100 than other areas. Note that if the straight line continues to the bottom edge of the image, the reference width ratio may be acquired based on the image output by the image processing unit 104 before determining the virtual straight line. In addition, the acquisition of the reference width ratio does not need to be performed for each frame, that is, for all input images, and the timing of execution can be changed arbitrarily within an appropriate range, for example, at a predetermined timing or when the virtual straight line changes significantly.

Figure 2 shows the relationship between the actual distance from the camera to an object and the positional deviation of the object captured at that actual distance, based on the pitch deviation. As shown in this figure, the deviation of the actual object's position relative to the pitch deviation becomes smaller the closer the object is to the camera. Generally, the pitch during normal driving deviates by about ±1°, but the corresponding deviation in length at the bottom edge of the image is about ±3-4%, which does not have a significant effect on distance measurement.

Returning to FIG. 1, the distance calculation unit 110 calculates the distance to the target based on the virtual line acquired by the virtual line acquisition unit 106 and the reference width ratio acquired by the reference width acquisition unit 108.

Figure 3 is a diagram showing an example of finding the distance to an object from the reference width ratio. For example, when a person is photographed as the object O, measuring the distance to this object O will be described. A right triangle is formed by the perpendicular line to the imaging plane, a point at a focal length f perpendicular to the imaging plane, and a width y on the imaging plane. As shown in this Figure 3, the distance calculation unit 110 may obtain, for example, the vertical coordinate of the bottom edge of the rectangle in which the object O is detected, or the bottom edge of the area in which the object O is detected. The vertical coordinate of the object O at other positions may also be obtained, but for example, in the case of an information processing device 1 to be mounted on a vehicle, it is desirable to use the bottom position at which the distance is the shortest for safety reasons.

As shown in the figure, a similar right triangle is formed including a straight line parallel to the imaging plane at the distance of the object O and a perpendicular line to the imaging plane. With this definition, the distance to the object O can be expressed as follows:

The distance calculation unit 110 can obtain the distance to the target O based on the width of the virtual line at the position of the target O by fixing Y in this equation (1) as a fixed value, for example, the lane width. For example, it is assumed that the width of the virtual line at the position of the target O (hereinafter, the position of the target O below the image in the vertical direction may be simply referred to as the position) is obtained as u pixels on the imaging surface. In other words, it is assumed that the width of the virtual line at the position of the target O in the image is u pixels. In this case, it can be expressed as y = u × (pixel pitch or pixel width). The pixel pitch or pixel width is a fixed value that can be obtained by the configuration of the imaging unit 100. Therefore, by using equation (1), the distance calculation unit 110 can calculate the distance to the target O.

The information processing device 1 outputs the distance calculated by the distance calculation unit 110 as the distance to the object O. If there are multiple objects, the distance may be measured and output for each object. For example, lane width is often a value determined by law or the like. For this reason, the lane width can be set to a fixed value, for example, 3 m for general roads and 3.5 m for expressways, so that the distance to the object O can be measured appropriately. Note that the object can be extracted using an appropriate method, such as using various feature amounts.

The lane width may be defined as being the same up to the object to which the distance is measured, while the lane width at the bottom of the image may be appropriately acquired. For example, if a part of the hood of the car is captured at the bottom by the vehicle-mounted camera provided in the imaging unit 100, the width of the virtual straight line at the bottom of the image may be calculated based on the size (fixed value) of this part of the hood. On expressways, the lane width may differ depending on the lane in which the vehicle is located. To deal with such cases, the width of the virtual straight line at the bottom of the image may be set depending on the lane in which the vehicle is located. The lane width may also differ depending on the road. In such cases, the lane width may be acquired from a database, etc., based on information received from a GPS (Global Positioning System), etc., or position information acquired via a CAN (Controller Area Network), etc. Of course, it is also possible to acquire the lane width for each lane.

As another example, the reference width acquisition unit 108 may obtain a reference lane width based on the intersection between the bottom edge of the image and a virtual straight line. This is because the position at which the imaging unit 100 is installed is fixed, and therefore the width of the lane at the bottom edge of the image is measurable in the image processed by the image processing unit 104. In this way, the reference width acquisition unit 108 may obtain the lane width or the actual distance between the virtual straight lines, rather than obtaining the ratio between the actual lane and the width between the virtual straight lines. In this case as well, the above formulas etc. can be used to perform similar processing.

Next, the acquisition of a virtual line by the virtual line acquisition unit 106 will be described. When the road is straight and the lanes are straight, a line acquired by various detection methods such as white line detection can be used as equivalent to a virtual line. In such a case, there is no need to acquire a virtual line.

On the other hand, if the road is curved, the result of detecting the white lines will be a curved shape. In such cases, it becomes difficult to detect the distance from the lane width as described above. For example, it becomes necessary to find the normal of the white line that passes through the front side of the area (including horizontal and vertical) where the object O exists. Such complex calculations are often difficult to process in real time, which is limited by the frame rate. In such cases, a method of detecting virtual straight lines is effective.

Figure 4 is a diagram for explaining the calculation of a virtual straight line according to one embodiment. In the following, the first direction and the second direction may be defined as shown in Figure 4. That is, the width of a lane or the like indicates the distance in the first direction having the same coordinate in the second direction. The position of the bottom end or the like indicates the coordinate value in the second direction. For example, the bottom end of the image may have a coordinate value of 0 in the second direction, and based on this coordinate value, the coordinate value may increase for each pixel going upward.

The figure shows an image after distortion and the like have been corrected by the image processing unit 104, for example. The virtual straight line acquisition unit 106 performs white line detection using one of various white line detection methods. White line detection may be performed within a range having a tilt within a predetermined angle range. A virtual straight line is acquired by fitting a straight line in a predetermined region of the detected white line, such as, as a non-limiting example, the bottom 1/4 of the image. Various methods can be used for this fitting as well.

By acquiring virtual straight lines in this way, it becomes possible to measure the distance to an object appropriately even on a curved road. For example, the distance to object O may be measured using equation (1) by using the width between the virtual straight lines at the vertical position of the bottom edge of object O in the image. In this case, similar to the above, it is desirable for the distance calculation unit 110 to acquire the bottom edge of the detected rectangle of object O, or the width between the virtual straight lines based on the bottom edge of object O.

Figure 5 is a flowchart showing an example of processing by the information processing device 1 according to this embodiment.

First, the imaging unit 100 acquires an image (S100). As described above, the imaging unit 100 may be provided in the information processing device 1 or may be provided externally. If the imaging unit 100 is provided externally, the information processing device 1 acquires the image in this step by receiving the image from the imaging unit 100, etc.

Next, the image processing unit 104 executes image processing (S102). Image data may be stored in the storage unit 102 between image acquisition and image processing. In this case, the image processing unit 104 reads the image data stored in the storage unit 102 and executes image processing. The appropriately processed image may be stored again in the storage unit 102.

Next, the virtual straight line acquisition unit 106 acquires a virtual straight line in the processed image (S104). Note that if the white line has been detected as a straight line by white line detection or the like, this step may be omitted and the detected white line may be used in the following processing. In other words, this processing may be omitted at will.

Next, the reference width acquisition unit 108 acquires the width of the lane or the ratio of the lane width to the width between the virtual straight lines (S106). The reference width acquisition unit 108 may acquire the lane width, etc., at the bottom edge of the image based on the results of the white line detection. If a virtual straight line has been acquired, the reference width acquisition unit 108 may acquire the lane width at the bottom edge of the image based on the position of the intersection between the virtual straight line and the bottom edge of the image.

Next, the distance calculation unit 110 detects the target and measures the distance to the target based on the detection result and the data acquired in each step above (S108). For example, the distance calculation unit 110 calculates the distance based on the position of the bottom edge of the rectangular area representing the detected target as described above. Furthermore, when multiple targets are detected, the distance calculation unit 110 may acquire the distances to the multiple targets.

In this way, the image is used to measure the distance to the object. These processes may be repeated for each frame, for example. Also, the process of S106 does not have to be executed for each frame. For example, the process of S106 may be executed for each predetermined frame. As another example, at least one of the processes from S102 to S108 may be executed using information from a past frame. For example, a virtual straight line may be obtained and corrected, a diagonal line width may be obtained and corrected, or a distance may be obtained and corrected based on the difference between the images of the past frame and the current frame.

As described above, according to this embodiment, it is possible to appropriately measure the distance to a target with high accuracy using a monocular camera. As a result, it is possible to reduce the financial and mounting costs, etc., that would otherwise be incurred by using multiple cameras. Furthermore, it is possible to reduce the cumbersome calibration process that would be required when using multiple cameras, as well as the computational costs associated with handling data from multiple cameras.

Second embodiment
By using the same data as the information processing device 1 described above, it is also possible to estimate the width of the target.

Figure 6 is a block diagram showing the information processing device 1 according to this embodiment. In addition to the configuration of Figure 1, the information processing device 1 may include a target width acquisition unit 112. The target width acquisition unit 112 acquires the width of the target based on the lane width acquired by the reference width acquisition unit 108 and the width of the target on the image.

The target width acquisition unit 112 acquires the number of pixels between the lanes in the vertical coordinate of the bottom edge of the target (which may be the bottom edge of the detected rectangle as in the previous embodiment) and the number of pixels of the width of the bottom edge of the target.

Figure 7 shows an example of an image acquired by the information processing device 1 in this embodiment. The distance between lanes is fixed at, for example, 3.5 m in Figure 7. Therefore, the width [m] of an object can be measured based on the width [pixel] of the lane at the object's position and the width [pixel] of the object.

For example, as shown in the figure, if the lane width at the target position is 140 pixels and the target width is 120 pixels, then we can calculate 3.5[m] × 120 / 140 = 3.0[m].

In this way, according to this embodiment, the width of the target can also be obtained in the same steps. For example, by obtaining the width of the target in this way, it is also possible to obtain the type of vehicle of the target that is traveling on the road. By obtaining the type of vehicle, it is also possible to use it as data to be used in ADAS (Advanced Driver-Assistance Systems).

Third embodiment
By using the virtual straight line as described above, the distance to the object can be measured even when there is a change in the slope of the captured image. In this embodiment, the process of the information processing device 1 when the slope of the road changes will be described.

When the gradient changes, the width of the object and the lane width can be obtained in the same way as in the previous embodiment, regardless of the height of the object, but the straight-line distance to the object will differ depending on the gradient. For this reason, it is necessary to obtain information about the gradient and calculate the straight-line distance.

Figure 8 is a block diagram showing the information processing device 1 in this embodiment. In addition to the information processing device 1 of the first embodiment described above, the information processing device 1 includes a vanishing point acquisition unit 114 and a gradient calculation unit 116. Note that the information processing device 1 may also be configured to include a vanishing point acquisition unit 114 and a gradient calculation unit 116 in addition to the information processing device 1 of the second embodiment.

The vanishing point acquisition unit 114 acquires a vanishing point based on lines, half lines, and line segments detected from the image. The vanishing point may be acquired by a general method. Information about virtual lines acquired by the virtual line acquisition unit 106 may be used to acquire the vanishing point. For example, the vanishing point acquisition unit 114 may calculate a point where the virtual lines acquired by the virtual line acquisition unit 106 intersect, and acquire this point as the vanishing point. The vanishing point acquisition unit 114 detects a first vanishing point based on a line from the bottom edge of the image, for example. Then, in the result of the white line detection, a second vanishing point may be detected by detecting a line above a point where the inclination in the image changes by a predetermined value or more.

The gradient calculation unit 116 calculates the position where the gradient changes and the degree of change in the gradient from the acquired vanishing point. The calculation method will be explained in detail later.

Figure 9 is a diagram showing an example in which multiple vanishing points are detected. As shown in this figure, when the gradient varies greatly within an image, vanishing points occur at different positions. Therefore, the virtual line acquisition unit 106 detects the positions of the vanishing points before and after the gradient varies. For example, the virtual line acquisition unit 106 may divide the image into predetermined regions in the second direction and detect line segments, etc. within the regions. Then, when the difference or ratio of the slopes detected in each region exceeds a predetermined value, a virtual line may be acquired for each of them.

The vanishing point acquisition unit 114 acquires multiple vanishing points from the acquired multiple sets of virtual straight lines. In the case of FIG. 9, a first vanishing point formed by a set of virtual straight lines from the bottom end and a second vanishing point formed by a set of virtual straight lines above are extracted. Then, the distance between these vanishing points in the second direction is extracted in pixel units.

f[m]及びピクセルピッチは、カメラの構成により取得できる値であり、diff[pix]は、上記の説明により取得することが可能である。このため、(2)式に基づいて、道路の勾配の変動がどの程度であるかを算出することが可能となる。 FIG. 10 is a schematic diagram showing a cross-sectional view of a road having a gradient variation as shown in FIG. 9. The solid line represents the road, and the dotted line represents the direction of the vanishing point. For example, the imaging surface is shown in this diagram. From this diagram, the gradient variation value can be obtained by the following formula.

f[m] and pixel pitch are values that can be obtained from the camera configuration, and diff[pix] can be obtained from the above explanation. Therefore, it is possible to calculate the degree of change in road gradient based on formula (2).

The results of white line detection and the width of the lane make it possible to determine up to what distance from the camera the gradient does not change and at what point the gradient begins to change. As a result, the distance calculation unit 110 can calculate the distance to the object based on these distances and gradients.

Xは、(1)式に基づいて、対象Oの位置における車線幅から求められた距離であり、θは、(2)式に基づいて求められた勾配である。(3)式により、対象Oまでの直線距離を近似して求める形態としてもよい。 It is also possible to simplify the calculation and calculate the distance to the target as follows:

X is the distance calculated from the lane width at the position of the target O based on the formula (1), and θ is the gradient calculated based on the formula (2). The straight-line distance to the target O may be calculated by approximation based on the formula (3).

As described above, even if there is a gradient change in the image, or if there is no information that requires other sensors such as LiDAR (Light Detection and Ranging), it is possible to achieve accurate distance measurement to the target using a monocular camera.

Note that many methods for detecting vanishing points can result in blurring. To reduce the effects of such blurring, for example, if the second vanishing point in FIG. 9 is not within a predetermined range from the first vanishing point as indicated by the dashed line, the gradient may be found from the second vanishing point. This blurring may be taken into account only in the second direction.

(Fourth embodiment)
In each of the above-described embodiments, distance measurement is performed under the assumption that the lane width is constant. However, the lane width may not be constant depending on the location. In such a case, it is difficult to avoid the occurrence of distance calculation errors using the distance measurement method in each of the above-described embodiments. Therefore, the distance measurement in this embodiment is intended to deal with such a case.

Figure 11 is a block diagram showing an information processing device 1 according to this embodiment. In addition to the configuration of Figure 1, the information processing device 1 further includes a map reading unit 118. Note that the information processing device 1 according to the second or third embodiment may be configured to include the map reading unit 118.

The map reading unit 118 reads a map from the outside or from the memory unit 102. The map read by the map reading unit 118 may be, for example, an HD map. Information on lane width is recorded on this map. The map reading unit 118 reads a surrounding map at an appropriate time based on the current position of the vehicle. The current position of the vehicle may be obtained from a GPS receiver (not shown) provided in the information processing device 1, may be read from an integrated value of the steering wheel, or may be obtained by a technology such as SLAM (Simultaneous Localization and Mapping).

The distance calculation unit 110 checks whether the lane width in the image is constant based on the acquired map, and then measures the distance to the object.

When using an HD map, it is possible to check whether the lane width is constant based on the distance from the vehicle. For this reason, the distance calculation unit 110 measures the distance to the target once, assuming that the lane width is constant. If this distance belongs to a distance where the lane width obtained from the HD map is not constant, the distance is recalculated based on the lane width obtained from the HD map. The distance calculation unit 110 recalculates the distance to the target, for example, based on the ratio between the lane width at the target position and the lane width at the bottom edge of the image. As an example, the distance calculated assuming that the lane is constant can be multiplied by (lane width at the target position) / (lane width at the bottom edge of the image) to correct the distance to one that takes the lane width into account.

In this way, the distance calculation unit 110 can correct the distance based on the lane width read by the map reading unit 118.

When the lane width is narrower at the rear than at the front in the image and the object is at the rear, the object's position is detected as being farther away than its actual position. Therefore, the object's position calculated assuming the lane width to be constant is calculated as being farther away than its actual position, and is detected as being further away than the position where the lane width has changed. In this case, the distance calculation unit 110 detects that the object is on the other side of the changed lane width and recalculates the distance to the object. This calculation allows for highly accurate distance measurement.

Conversely, if the lane width is wider at the back than at the front in the image and the object is at the back, the object's position will be detected as closer than its actual position. In this case, even if it is compared with the position where the lane width changes in the HD map based on the distance detected as if the lane width were constant, it may be determined to be the position before the lane width changed. In such cases, it may not be recalculated.

When determining the accuracy of distance measurement, in the range where lane width may fluctuate, the distance where the lane width is constant and the distance after the lane width has changed may be calculated, and the actual distance may be selected from the multiple distances calculated based on these distances and the distance between the vehicle and the position of the change.

As described above, this embodiment makes it possible to properly measure the distance to an object even when the lane width changes.

When using an HD map in this manner, it is also possible to measure the distance to the target without acquiring the reference lane width by the reference width acquisition unit 108. In other words, it is possible to use the distance between the virtual straight lines at the bottom edge of the image, which serves as the reference, as a value read from the vehicle's position on the HD map.

Although lanes have been described above, they do not have to be lanes. For example, the top or bottom of a guardrail installed on the side of the road may be detected as a straight line. As another example, the edge of the road may be detected as a straight line. The edge of the road may be, for example, the boundary between the center divider or side wall and the road surface on a highway, or the boundary between a side road or pedestrian path on a general road.

The technology disclosed herein can be applied to a variety of products. For example, the technology disclosed herein may be realized as a device mounted on any type of moving object, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, construction machinery, or an agricultural machine (tractor).

FIG. 12 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology disclosed herein can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 12, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside vehicle information detection unit 7400, an inside vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be, for example, an in-vehicle communication network conforming to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).

Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various devices to be controlled. Each control unit includes a network I/F for communicating with other control units via a communication network 7010, and a communication I/F for communicating with devices or sensors inside and outside the vehicle by wired or wireless communication. In FIG. 12, the functional configuration of the integrated control unit 7600 includes a microcomputer 7610, a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I/F 7660, an audio/image output unit 7670, an in-vehicle network I/F 7680, and a storage unit 7690. Other control units also include a microcomputer, a communication I/F, a storage unit, and the like.

The drive system control unit 7100 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle. The drive system control unit 7100 may also function as a control device such as an ABS (Antilock Brake System) or ESC (Electronic Stability Control).

The drive system control unit 7100 is connected to a vehicle state detection unit 7110. The vehicle state detection unit 7110 includes at least one of a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine speed, or the rotation speed of the wheels, for example. The drive system control unit 7100 performs arithmetic processing using the signal input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

The body system control unit 7200 controls the operation of various devices installed in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, tail lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals from various switches transmitted from a portable device that replaces a key can be input to the body system control unit 7200. The body system control unit 7200 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, according to various programs. For example, information such as the battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs calculations using these signals, and controls the temperature regulation of the secondary battery 7310 or a cooling device or the like equipped in the battery device.

The outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of the imaging unit 7410 and the outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc., around the vehicle equipped with the vehicle control system 7000.

The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the degree of sunshine, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or device, or may be provided as a device in which multiple sensors or devices are integrated.

Here, FIG. 13 shows an example of the installation positions of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

In addition, FIG. 13 shows an example of the imaging ranges of each of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, an overhead image of the vehicle 7900 viewed from above is obtained by superimposing the image data captured by the imaging units 7910, 7912, 7914, and 7916.

External information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and upper part of the windshield inside the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. External information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and upper part of the windshield inside the vehicle 7900 may be, for example, LIDAR devices. These external information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

Returning to FIG. 12, the explanation will be continued. The outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside-vehicle information detection unit 7400 also receives detection information from the connected outside-vehicle information detection unit 7420. If the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detection unit 7400 transmits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. The outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface based on the received information. The outside-vehicle information detection unit 7400 may perform environmental recognition processing for recognizing rainfall, fog, road surface conditions, etc. based on the received information. The outside-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, cars, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by synthesizing image data captured by different imaging units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different imaging units 7410.

The in-vehicle information detection unit 7500 detects information inside the vehicle. For example, a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound inside the vehicle. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or a driver holding the steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing. The in-vehicle information detection unit 7500 may perform processing such as noise canceling processing on the collected sound signal.

The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. The input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is realized by a device that can be operated by the passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input by a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an external connection device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the above-mentioned input unit 7800 and outputs the input signal to the integrated control unit 7600. Passengers and others can operate the input unit 7800 to input various data and instruct processing operations to the vehicle control system 7000.

The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The storage unit 7690 may also be realized by a magnetic storage device such as a HDD (Hard Disc Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device, etc.

The general-purpose communication I/F 7620 is a general-purpose communication I/F that mediates communication between various devices present in the external environment 7750. The general-purpose communication I/F 7620 may implement cellular communication protocols such as GSM (registered trademark) (Global System of Mobile communications), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution) or LTE-A (LTE-Advanced), or other wireless communication protocols such as wireless LAN (also called Wi-Fi (registered trademark)) and Bluetooth (registered trademark). The general-purpose communication I/F 7620 may connect to devices (e.g., application servers or control servers) present on an external network (e.g., the Internet, a cloud network, or a carrier-specific network) via, for example, a base station or an access point. The general-purpose communication I/F 7620 may also connect to terminals (e.g., drivers', pedestrians', or store terminals, or MTC (Machine Type Communication) terminals) present in the vicinity of the vehicle, for example, using P2P (Peer To Peer) technology.

The dedicated communication I/F 7630 is a communication I/F that supports a communication protocol developed for use in vehicles. The dedicated communication I/F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), DSRC (Dedicated Short Range Communications), or a cellular communication protocol, which is a combination of the lower layer IEEE 802.11p and the higher layer IEEE 1609. The dedicated communication I/F 7630 typically performs V2X communication, which is a concept that includes one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites) and generates position information including the latitude, longitude, and altitude of the vehicle. The positioning unit 7640 may determine the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

The beacon receiver 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations installed on the road, and acquires information such as the current location, congestion, road closures, and travel time. The functions of the beacon receiver 7650 may be included in the dedicated communication I/F 7630 described above.

The in-vehicle device I/F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I/F 7660 may establish a wireless connection using a wireless communication protocol such as a wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). The in-vehicle device I/F 7660 may also establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I/F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

The in-vehicle network I/F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I/F 7680 transmits and receives signals, etc. in accordance with a specific protocol supported by the communication network 7010.

The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I/F 7660, and the in-vehicle network I/F 7680. For example, the microcomputer 7610 may calculate the control target value of the driving force generating device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 may perform cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may control the driving force generating device, steering mechanism, braking device, etc. based on the acquired information about the surroundings of the vehicle, thereby performing cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation.

The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people based on information acquired through at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I/F 7660, and the in-vehicle network I/F 7680, and may create local map information including information about the surroundings of the vehicle's current position. In addition, the microcomputer 7610 may predict dangers such as vehicle collisions, the approach of pedestrians, or entry into closed roads based on the acquired information, and generate warning signals. The warning signals may be, for example, signals for generating warning sounds or turning on warning lights.

The audio/image output unit 7670 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of FIG. 12, an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may be other devices such as headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, or a lamp, in addition to these devices. When the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, graphs, etc. When the output device is an audio output device, the audio output device converts an audio signal consisting of reproduced audio data or acoustic data into an analog signal and audibly outputs it.

In the example shown in FIG. 12, at least two control units connected via the communication network 7010 may be integrated into one control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by any control unit may be provided by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, a predetermined calculation process may be performed by any control unit. Similarly, a sensor or device connected to any control unit may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

A computer program for implementing each function of the information processing device 1 according to this embodiment described with reference to FIG. 1 and the like can be implemented in any control unit or the like. A computer-readable recording medium on which such a computer program is stored can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. The computer program may also be distributed, for example, via a network, without using a recording medium.

In the vehicle control system 7000 described above, the information processing device 1 according to this embodiment described using FIG. 1 etc. can be applied to at least a part of the outside vehicle information detection unit 7400, the imaging unit 7410, and the outside vehicle information detection unit 7420 of the application example shown in FIG. 12.

In addition, at least some of the components of the information processing device 1 described using FIG. 1 etc. may be realized in a module (e.g., an integrated circuit module configured on a single die) for at least some of the outside vehicle information detection unit 7400, the imaging unit 7410, and the outside vehicle information detection unit 7420 shown in FIG. 12. Alternatively, the information processing device 1 described using FIG. 1 etc. may be realized by multiple control units of the vehicle control system 7000 shown in FIG. 12.

The above-described embodiment may be modified as follows:

(1)
A storage unit and a processing unit are provided,
The processing unit includes:
Acquiring an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction;
Detecting at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range;
acquiring a virtual straight line from at least a portion of the detected straight lines;
Obtaining the actual distance between the imaginary lines;
measuring a distance to the object from a position of the object relative to the virtual line in the image and a width between the virtual lines at the position of the object;
Information processing device.

(2)
The processing unit includes:
acquiring a ratio between a width of the virtual straight line in the first direction at the lower end in the second direction in the image and a width of the actual straight lines based on an intersection point between the virtual straight line and the lower end;
An information processing device as described in (1).

(3)
The processing unit includes:
obtaining a coordinate value of a lower end of the object in the second direction;
calculating a distance to the object based on a ratio between a width in the first direction between the virtual straight lines at a coordinate value of a lower end of the object in the second direction and a width in the first direction of the virtual straight line at a lower end of the image;
An information processing device according to (1) or (2).

(Four)
The processing unit includes:
Calculating the distance to the target assuming that the width of the virtual straight line is a constant value;
(3) An information processing device.

(Five)
The processing unit includes:
Detecting a plurality of vanishing points from at least a portion of the plurality of straight lines;
If the plurality of vanishing points are not within a predetermined range, a point where a slope changes is detected on the way to the object.
An information processing device according to any one of (1) to (4).

(6)
The processing unit includes:
When the plurality of vanishing points are not within a predetermined range, the degree of change in the inclination is calculated from the deviation of the positions of the plurality of vanishing points.
(5) An information processing device according to the present invention.

(7)
The processing unit includes:
and determining a distance to the object based on the change in tilt.
(6) An information processing device.

(8)
The processing unit includes:
Based on an HD map (high-precision three-dimensional map data), a width between the virtual straight lines at the position of the object is corrected, and a distance to the object is calculated.
An information processing device according to any one of (1) to (7).

(9)
The processing unit includes:
measuring a real width of the object based on a width of the object relative to a width of the imaginary line in the image;
An information processing device according to any one of (1) to (8).

(Ten)
the image is an image captured as a frame of video information;
The processing unit further comprises:
performing at least one process using one or more past frame images of the image;
An information processing device according to any one of (1) to (9).

(11)
At least a part of the straight line to be detected is a white line on a road.
An information processing device according to any one of (1) to (10).

(12)
At least a part of the straight line to be detected is the upper end or the lower end of the guardrail.
An information processing device according to any one of (1) to (11).

(13)
At least a part of the straight line to be detected is a road edge.
An information processing device according to any one of (1) to (12).

(14)
The subject is a human or an animal.
An information processing device according to any one of (1) to (13).

(15)
The object is an automobile.
An information processing device according to any one of (1) to (14).

(16)
The object is an obstacle.
An information processing device according to any one of (1) to (15).

(17)
The processing section:
Acquiring an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction;
Detecting at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range;
acquiring a virtual straight line from at least a part of the detected straight lines;
Obtaining the actual distance between the imaginary lines;
measuring a distance to the object from a position of the object relative to the virtual line in the image and a width between the virtual lines at the position of the object;
Information processing methods.

(18)
The method includes at least one step of executing each step of the information processing device according to (2) to (16),
(17) An information processing method according to (17).

(19)
When you run it on your computer,
Acquiring an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction;
Detecting at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range;
acquiring a virtual straight line from at least a part of the detected straight lines;
Obtaining the actual distance between the imaginary lines;
measuring a distance to the object from a position of the object relative to the virtual line in the image and a width between the virtual lines at the position of the object;
program.

(20)
On the computer,
Executing a method including at least one step among the methods for executing each step of the information processing device according to (2) to (16);
(19) The program according to (19).

The aspects of the present disclosure are not limited to the above-described embodiments, but include various conceivable modifications, and the effects of the present disclosure are not limited to the above-described contents. The components in each embodiment may be appropriately combined and applied. In other words, various additions, modifications, and partial deletions are possible within the scope that does not deviate from the conceptual idea and intent of the present disclosure derived from the contents defined in the claims and their equivalents.

1: Information processing device;
100: imaging unit, 102: storage unit, 104: image processing unit, 106: virtual straight line acquisition unit, 108: reference width acquisition unit, 110: distance calculation unit,
112: target width acquisition unit,
114: vanishing point acquisition unit, 116: gradient calculation unit,
118: Map reader

Claims (12)

A storage unit and a processing unit are provided,
The processing unit includes:
Acquiring an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction;
Detecting at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range;
acquiring a virtual straight line from at least a portion of the detected straight lines;
Obtaining the actual distance between the imaginary lines;
measuring a distance to the object from a position of the object relative to the virtual line in the image and a width between the virtual lines at the position of the object;
Information processing device. The processing unit includes:
acquiring a ratio between a width of the virtual straight line in the first direction at the lower end in the second direction in the image and a width of the actual straight lines based on an intersection point between the virtual straight line and the lower end;
The information processing device according to claim 1. The processing unit includes:
obtaining a coordinate value of a lower end of the object in the second direction;
calculating a distance to the object based on a ratio between a width in the first direction between the virtual straight lines at a coordinate value of a lower end of the object in the second direction and a width in the first direction of the virtual straight line at a lower end of the image;
The information processing device according to claim 1. The processing unit includes:
Calculating the distance to the target assuming that the width of the virtual straight line is a constant value;
The information processing device according to claim 3. The processing unit includes:
Detecting a plurality of vanishing points from at least a portion of the plurality of straight lines;
If the plurality of vanishing points are not within a predetermined range, a point where a slope changes is detected on the way to the object.
The information processing device according to claim 1. The processing unit includes:
When the plurality of vanishing points are not within a predetermined range, the degree of change in the inclination is calculated from the deviation of the positions of the plurality of vanishing points.
The information processing device according to claim 5. The processing unit includes:
and determining a distance to the object based on the change in tilt.
The information processing device according to claim 6. The processing unit includes:
Based on an HD map (high-precision three-dimensional map data), a width between the virtual straight lines at the position of the object is corrected, and a distance to the object is calculated.
The information processing device according to claim 1. The processing unit includes:
measuring a real width of the object based on a width of the object relative to a width of the imaginary line in the image;
The information processing device according to claim 1. the image is an image captured as a frame of video information;
The processing unit further comprises:
performing at least one process using one or more past frame images of the image;
The information processing device according to claim 1. The processing section:
Acquiring an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction;
Detecting at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range;
acquiring a virtual straight line from at least a portion of the detected straight lines;
Obtaining the actual distance between the imaginary lines;
measuring a distance to the object from a position of the object relative to the virtual line in the image and a width between the virtual lines at the position of the object;
Information processing methods. When you run it on your computer,
Acquiring an image having pixels arranged in an array in a first direction and a second direction intersecting the first direction;
Detecting at least a portion of a plurality of straight lines extending upward from a lower end of the image in the second direction with an inclination within a predetermined angle range;
acquiring a virtual straight line from at least a part of the detected straight lines;
Obtaining the actual distance between the imaginary lines;
measuring a distance to the object from a position of the object relative to the virtual line in the image and a width between the virtual lines at the position of the object;
program.

Images