WO2023095489A1

WO2023095489A1 - External environment recognition device

Info

Publication number: WO2023095489A1
Application number: PCT/JP2022/038648
Authority: WO
Inventors: 健遠藤; 春樹的野; 健永崎
Original assignee: 日立Astemo株式会社
Priority date: 2021-11-26
Filing date: 2022-10-17
Publication date: 2023-06-01
Also published as: JP2023078934A

Abstract

Provided is an external environment recognition device capable of accurately calculating the distance to an object regardless of the usage environment. The external environment recognition device is mounted on a vehicle. The external environment recognition device has an image obtaining unit that obtains an image of a landmark and an object; a landmark information obtaining unit that obtains three dimensional information about the landmark; and a distance estimation unit that estimates the distance to the object on the basis of the three dimensional information about the landmark and the sizes of the landmark and the object in the image.

Description

External recognition device

The present invention relates to an external world recognition device.

Conventionally, systems are known that use cameras to detect pedestrians crossing the front of a vehicle, warn the driver and automatically brake as necessary. In particular, from the viewpoint of realization of automatic driving and prevention of traffic accidents, there is a great deal of interest in an automatic braking device that detects a pedestrian or the like and performs braking control. In the automatic braking device, it is necessary to calculate the distance from the own vehicle to the pedestrian with high accuracy. In addition, in the automatic braking device, it is necessary to obtain the distance to the pedestrian in a wider field of view in order to cope with the sudden movement of the pedestrian.

For example, Patent Literature 1 describes a pedestrian distance calculation method using a stereo camera that has a field-of-view overlapping area and a field-of-view non-overlapping area. Specifically, Patent Literature 1 describes a stereo camera device that calculates the distance of a moving object in a monocular region using road surface height calculated from parallax information. The stereo camera device obtains coordinates obtained by subtracting (lowering) the height of the stairs relative to the road surface from the vertical coordinates on the image of the feet of the pedestrian on the stairs. Then, in the stereo camera device, the obtained coordinates are extended to the stereo area, and the distance to the pedestrian is obtained from the parallax information of the road surface at that position.

JP 2017-96777 A

By the way, in the stereo camera device described in Patent Document 1, the parallax of the road surface is calculated based on the texture information of the road surface in the acquired image, and this parallax information is used to calculate the distance to the pedestrian. However, in a usage environment where the texture of the road surface is not clear, such as at night or in rainy weather, it may be difficult to obtain the parallax information of the road surface, and the distance to the pedestrian may not be calculated accurately.

The present invention has been made in view of the above problems, and its purpose is to provide an external world recognition device that can accurately calculate the distance to an object regardless of the usage environment.

In order to achieve the above object, an external world recognition device according to the present invention is an external world recognition device mounted on a vehicle, comprising: an image acquisition unit for acquiring an image of a landmark and an object; a landmark information acquisition unit that acquires information; and a distance estimation unit that estimates the distance to the object based on the three-dimensional information of the landmark and the sizes of the landmark and the object in the image. and.

According to the external world recognition device of the present invention, the distance to the object can be accurately calculated regardless of the usage environment. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

1 is a functional block diagram showing a schematic configuration of an external world recognition device according to a first embodiment of the present invention; FIG. FIG. 2 is a plan view showing an example of a state in which the external world recognition device of FIG. 1 is mounted on a vehicle; 2 is a flowchart showing landmark information acquisition processing of the external world recognition device of FIG. 1; An example of the image which the external world recognition apparatus of FIG. 1 acquired by landmark information acquisition processing. 2 is a flowchart showing distance estimation processing of the external world recognition device of FIG. 1; An example of the image which the external world recognition apparatus of FIG. 1 acquired by the distance estimation process. The functional block diagram which shows schematic structure of the external world recognition apparatus of the modification of 1st Embodiment. 8 is a flowchart showing distance estimation processing of the external world recognition device of FIG. 7; FIG. 8 is a view for explaining proximity determination processing of the external world recognition device of FIG. 7; FIG. 8 is a diagram showing another example of proximity determination processing of the external world recognition device of FIG. 7 ; FIG. 5 is a functional block diagram showing a schematic configuration of an external world recognition device according to another modification of the first embodiment; The functional block diagram which shows schematic structure of the external world recognition apparatus which concerns on 2nd Embodiment of this invention. The functional block diagram which shows schematic structure of the external world recognition apparatus which concerns on 3rd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First embodiment>
FIG. 1 is a functional block diagram showing a schematic configuration of an external world recognition device 1 according to the first embodiment of the invention. 2 is a plan view showing an example of a state in which the external world recognition device 1 of FIG. 1 is mounted on a vehicle V. FIG. FIG. 3 is a flowchart showing landmark information acquisition processing of the external world recognition device 1 of FIG. FIG. 4 is an example of an image acquired by the external world recognition device 1 of FIG. 1 in the landmark information acquisition process. FIG. 5 is a flowchart showing distance estimation processing of the external world recognition device 1 of FIG. FIG. 6 is an example of an image acquired by the external world recognition device 1 of FIG. 1 through distance estimation processing.

The external world recognition device 1 is mounted on a vehicle V (hereinafter also referred to as own vehicle V). As shown in FIG. 1 , the external world recognition device 1 has an image acquisition section 10 , a landmark information acquisition section 30 and a distance estimation section 50 . Although illustration is omitted, the external world recognition device 1 has a configuration in which a CPU, a RAM, a ROM, and the like are connected via a bus, and the CPU executes various control programs stored in the ROM to control the entire system. control behavior. In this embodiment, the image acquisition unit 10 acquires images from a pair of

cameras

10a and 10b (FIG. 2) functioning as stereo cameras, and the distance estimation unit 50 uses the images to determine the pedestrian 200 as the target object. and the own vehicle V will be described. A “landmark” used in this specification means each three-dimensional object (for example,

reference numerals

100, 101 and 102 shown in FIG. 4) photographed by the pair of

cameras

10a and 10b. In this specification, "three-dimensional information of landmarks" means including at least one of the height and width of the

landmarks

100, 101, and 102 themselves. In addition to at least one of the height and width of the

landmarks

100, 101, and 102 themselves, the "three-dimensional information of the landmarks" includes the distance between the

landmarks

100, 101, and 102 and the vehicle V ( depth).

As shown in FIGS. 4 and 6, the image acquisition unit 10 acquires images of

landmarks

100, 101 and 102 and a pedestrian (object) 200 from a pair of

cameras

10a and 10b. Specifically, the image acquisition unit 10 captures an image captured by a pair of

cameras

10a and 10b, and divides a compound eye region with overlapping fields of view and a monocular region (a left monocular region and a right monocular region) with non-overlapping fields of view. Get the containing image. Hereinafter, the compound eye area of the image will be referred to as compound eye area image CA, and the left monocular area and right monocular area of the image will be referred to as left monocular area image LA and right monocular area image RA, respectively. As shown in FIG. 2, the pair of

cameras

10a and 10b are configured as one stereo camera mounted on a vehicle V. As shown in FIG. Specifically, the pair of

cameras

10a and 10b are composed of CCD or CMOS image sensors, and are directed to the front of the vehicle V. As shown in FIG. Also, the pair of

cameras

10a and 10b are installed so that each of the

cameras

10a and 10b captures an image in front of the vehicle V at a predetermined depression angle (that is, an imaging area), and the angles of depression overlap with each other. For example, if the camera 10a is the left camera and the camera 10b is the right camera, the left side of the depression angle of the camera 10a and the right side of the camera 10b overlap each other. As a result, in the present embodiment, the center front area of the vehicle V becomes a stereo area (also called a compound eye area) defined by the imaging area of the camera 10a and the imaging area of the camera 10b. Also, the left front area of the vehicle V is a left monocular area defined by the left portion of the photographing area of the camera 10b. The right front area of the vehicle V is a right monocular area defined by the right side of the imaging area of the camera 10a. Images captured by the pair of

cameras

10 a and 10 b are input to the image acquisition section 10 . The image acquisition unit 10 acquires two images captured by a pair of

cameras

10a and 10b, and acquires parallax information by applying a known parallax calculation algorithm to the acquired left and right images.

As shown in FIG. 1, the landmark information acquisition unit 30 includes a compound eye region landmark detection unit 32, a size information measurement unit 33, and a landmark information storage unit 36. A landmark information acquisition unit 30 acquires three-dimensional information of the

landmarks

100 , 101 and 102 . Specifically, the landmark information acquisition unit 30 acquires three-dimensional information of the

landmarks

100, 101, and 102 positioned in the compound eye area image CA.

As shown in FIG. 4, the compound eye area landmark detection unit 32 detects

landmarks

100, 101, and 102 from the compound eye area image CA acquired by the image acquisition unit 10. FIG. For example, the compound eye area landmark detection unit 32 may analyze the texture of the compound eye area image CA transmitted from the image acquisition unit 10 and detect the

landmarks

100, 101 and 102 from this. Moreover, the compound eye area landmark detection unit 32 may detect the

landmarks

100 , 101 , and 102 based on the parallax information acquired by the image acquisition unit 10 . Specifically, the

landmarks

100, 101, and 102 may be detected by clustering disparity information or the like, or the

landmarks

100, 101, and 102 may be detected using a convolutional neural network as statistical machine learning. You may

The size information measuring unit 33 measures the actual sizes of the

landmarks

100, 101, and 102 included in the three-dimensional information from the compound eye area image CA. Specifically, the size information measurement unit 33 calculates the actual heights of the

landmarks

100, 101, and 102 detected by the compound eye area landmark detection unit 32 from the compound eye area image CA. to calculate the Note that the size information measuring unit 33 measures not only the actual vertical size (actual height) of the

landmarks

100, 101, and 102, but also the actual horizontal size (actual height) of the

landmarks

100, 101, and 102. actual width) may be measured. Also, the size information measuring unit 33 may measure the actual size (actual height or actual width) of a part of the

landmarks

100 , 101 , 102 . The landmark information storage unit 36 registers at least one of the size information of the

landmarks

100, 101, and 102 measured by the size information measuring unit 33 as landmark information used for distance calculation, which will be described later. Specifically, the landmark information storage unit 36 stores size information (actual height, actual width, etc.) of the

landmarks

100, 101, and 102 that have been measured, as well as size information of the photographed

landmarks

100, 101, and 102. Stores texture information.

The distance estimation unit 50 estimates the distance to the pedestrian 200 located in the monocular area image (left monocular area image LA, right monocular area image RA). Specifically, the distance estimating unit 50 uses the

landmarks

100, 101, and 102 detected by the landmark information acquiring unit 30 to use the monocular area images (left monocular area image LA, right The distance to the pedestrian 200 included in the monocular area image RA) is estimated. More specifically, the distance estimation unit 50 estimates the distance from the vehicle V to the pedestrian 200 based on the three-dimensional information of the landmark 100 and the sizes of the landmark 100 and the pedestrian 200 in the image. do. In this embodiment, a case where the landmark 100 is used as a landmark for estimating the distance between the pedestrian 200 and the own vehicle V will be described. As shown in FIG. 1 , the distance estimation unit 50 includes a monocular area landmark detection unit 52 , a monocular area object detection unit 54 , a size estimation unit 58 and a distance calculation unit 60 .

As shown in FIG. 6, the monocular area landmark detection unit 52 detects landmarks 100 from the monocular area images (left monocular area image LA, right monocular area image RA) acquired by the image acquisition unit 10 . Specifically, the monocular area landmark detection unit 52 re-detects the landmark 100 based on the texture information of the landmark 100 stored in the landmark information storage unit 36 . For example, the monocular area landmark detection unit 52 may detect by template matching using the texture information of the landmark 100 stored in the landmark information storage unit 36 as a template. Also, the monocular area landmark detection unit 52 may detect the landmark 100 by tracking processing using a convolutional neural network. Further, the monocular area landmark detection unit 52 detects the vertical length of the landmark 100 in the image from the monocular area images (the left monocular area image LA and the right monocular area image RA) of the landmark 100 acquired from the image acquisition unit 10 . height (i.e., image height).

The monocular area object detection unit 54 detects an object for distance estimation from the monocular area images (left monocular area image LA, right monocular area image RA) acquired by the image acquisition unit 10 . The monocular region target object detection unit 54 detects, for example, a pedestrian 200 by known statistical machine learning when the target is the pedestrian 200 . Specifically, the monocular region object detection unit 54 may detect the pedestrian 200 using classical machine learning such as random forest, support vector machine, and real Adaboost, or may detect the pedestrian 200 using a convolutional neural network. A pedestrian 200 may be detected by using this. Further, the monocular area object detection unit 54 calculates the vertical length (that is, the image height) of the pedestrian 200 in the image based on the monocular area image of the pedestrian 200 acquired from the image acquisition unit 10 .

The size estimation unit 58 estimates the landmark 100 based on the image size of the landmark 100 in the monocular area image (left monocular area image LA, etc.) and the image size of the pedestrian 200 in the monocular area image (left monocular area image LA, etc.). The size of pedestrian 200 is estimated from the size of . The size estimation unit 58 estimates the size (height, etc.) of the pedestrian 200 using the size (height, etc.) of the landmark 100 stored in the landmark information storage unit 36 . Specifically, based on the image height of the landmark 100 and the image height of the pedestrian 200, the size estimation unit 58 calculates the height of the pedestrian 200 from the actual height of the landmark 100 (hereinafter also referred to as height). ). More specifically, the size estimation unit 58 multiplies the actual height of the landmark 100 by the ratio of the height of the image of the pedestrian 200 to the height of the image of the landmark 100 to obtain the height of the pedestrian 200. to estimate The size information of pedestrian 200 may be estimated only once.

Based on the actual size of the pedestrian 200 estimated by the size estimation unit 58 and the image size of the pedestrian 200 in the monocular area images (left monocular area image LA, right monocular area image RA), the distance calculation unit 60 A distance to the pedestrian 200 is calculated. Specifically, the distance calculation unit 60 calculates the distance from the own vehicle V to the pedestrian 200 based on the height of the pedestrian 200 estimated by the size estimation unit 58 . More specifically, the distance calculation unit 60 calculates the size of the vehicle based on the height of the pedestrian 200 estimated by the size estimation unit 58 and the image height of the pedestrian 200 detected by the monocular area object detection unit 54. The distance between V and pedestrian 200 is calculated.

Next, with reference to FIGS. 3 to 6, an operation example of the external world recognition device 1 of this embodiment will be described in landmark information acquisition processing (FIGS. 3 and 4) and distance estimation processing (FIGS. 5 and 6). I will explain separately. Here, as shown in FIG. 2, the external world recognition device 1 using a pair of

cameras

10a and 10b functioning as stereo cameras installed to monitor the front of the vehicle V will be described. Also, a case of calculating the distance between the own vehicle V and the pedestrian 200 based on the height of the pedestrian 200 photographed in the left monocular area image LA shown in FIG. 6 will be described.

First, the landmark information acquisition process will be described with reference to FIGS.

As shown in FIG. 3, the external world recognition device 1 performs image acquisition processing (P101), parallax calculation processing (P102), solid object detection processing (P103), and three-dimensional information acquisition processing (P104) in the landmark information acquisition processing. , the landmark registration process (P105) is executed in order.

In the image acquisition process (P101), the image acquisition unit 10 acquires images captured by a pair of

cameras

10a and 10b as shown in FIG. Acquire an area image RA).

Next, in the parallax calculation process (P102), the image acquisition unit 10 calculates the parallax in the compound eye area image CA. For example, a window area of 5×5 pixels is set for the parallax calculation. Then, the parallax is calculated by horizontally scanning the image of the right camera 10b using the SAD as an evaluation value, with the image of the left camera 10a of the pair of

cameras

10a and 10b as a reference.

Next, in the three-dimensional object detection process (P103), the compound eye region landmark detection unit 32 uses the parallax calculated in the parallax calculation process (P102) to detect

landmarks

100, 101, and 102 as shown in FIG. To detect. Specifically, since the areas of the

landmarks

100, 101, and 102 are at the same distance, clustering processing is performed on the parallax to roughly extract the areas where the

landmarks

100, 101, and 102 exist. be done. Next, since the distance value changes between each of the

landmarks

100, 101, and 102 and the boundary of the background, by detecting the parallax change point in the roughly extracted region, the

landmarks

100, 101, and 102 The existing areas are detected in detail. FIG. 4 shows the detection result of the three-dimensional object detection process (P103) for the landmark 100 among the plurality of

landmarks

100, 101, and 102. FIG. In the three-dimensional object detection process (P103), a rectangle B100 as shown in FIG. 4 is the detection result.

Next, in the three-dimensional information acquisition process (P104), the size information measuring unit 33 measures the height direction size ( That is, the actual height) is calculated. A method of estimating the size (actual height) with respect to the landmark 100 among the plurality of

landmarks

100, 101, and 102 will be described below. For estimating the actual height of the landmark 100, the rectangle B100, which is the detection result of the three-dimensional object detection process (P103), is used. T100 in FIG. 4 is the pixel position of the upper edge of the rectangle B100, and L100 in FIG. 4 indicates the pixel position of the lower edge of the rectangle B100. Also, let D_med be the disparity median value at the landmark 100 . D_med is obtained by calculating the median value of disparities contained within rectangle B100. The actual height Height of the landmark 100 is calculated by calculating "Height=BaseLine×abs(T100-L100)/D_med" (formula (1)). Here, BaseLine in Expression (1) is the base line length of the camera 10a and the camera 10b, and abs indicates an absolute value operator.

Next, in the landmark registration process (P105), the landmark information storage unit 36 registers the actual height information of the landmark 100 calculated in the three-dimensional information acquisition process (P104). Also, the landmark information storage unit 36 registers the texture information of the compound eye area image CA analyzed by the compound eye area landmark detection unit 32 . The landmark information storage unit 36 also stores a compound eye area image CA for the rectangle B100 detected in the three-dimensional object detection process (P103).

Next, the distance estimation process will be described with reference to FIGS. 5 and 6. FIG.

As shown in FIG. 5, in the distance estimation process, the external world recognition device 1 performs image acquisition processing (P111), landmark detection processing (P112), pedestrian detection processing (P113), size estimation processing (P115), distance The estimation process (P116) is executed in order.

In the image acquisition process (P111), the image acquisition unit 10 acquires images captured by a pair of

cameras

10a and 10b as shown in FIG. Acquire an area image RA).

Next, in the landmark detection process (P112), the monocular area landmark detection unit 52 detects the landmark 100 again. The landmark 100 was photographed in the compound eye area image CA in FIG. 4, but as the vehicle V moves, it is photographed in the left monocular area image LA in FIG. Here, by executing template matching using the texture information of the landmark 100 stored in the landmark information storage unit 36 as a template, the landmark 100 photographed in the left monocular area image LA shown in FIG. 6 is re-detected. do. The area detected in this process is represented by a rectangle B101 as shown in FIG.

Next, in the pedestrian detection process (P113), the monocular area object detection unit 54 detects the pedestrian 200 photographed in the left monocular area image LA of FIG. A convolutional neural network is used for this detection. The convolutional neural network used detects the pedestrian 200 included in the input predetermined image. In addition, the convolutional neural network not only locates the pedestrian 200, but also calculates the identification score at the same time. The identification score is a point for the type of pedestrian 200, and a high point for a certain type indicates that the region is likely to be of that type. In the pedestrian detection process (P113), the pedestrian 200 is detected by inputting the left monocular area image LA into the convolutional neural network and adopting only detection results with a high identification score for the type of pedestrian. The detection result is represented by a rectangle B201 as shown in FIG.

In the size estimation process (P115), the size estimation unit 58 estimates the height of the pedestrian 200 using the landmark 100 information. In the landmark registration process (P105) described above, the actual height of the landmark 100 is stored in the landmark information storage unit . Further, in the landmark detection process (P112), the monocular area landmark detection unit 52 detects the image height of the landmark 100 (that is, the height of the rectangle B101 on the image) IMG_Height_Land(pix). Also, in the pedestrian detection process (P113), the monocular area object detection unit 54 detects the image height of the pedestrian 200 (that is, the height of the rectangle B201 on the image) IMG_Height_Ped(pix). In the size estimation process (P115), the height Height_Ped of the pedestrian 200 is estimated by "Height_Ped=(IMG_Height_Ped/IMG_Height_Land)×Height" (formula (2)). That is, by applying the ratio of the image height (IMG_Height_Land) of the landmark 100 and the image height (IMG_Height_Ped) of the pedestrian 200 to the actual height (Height) of the landmark 100, the height of the pedestrian 200 is estimated. do.

Next, in the distance estimation process (P116), the distance calculation unit 60 calculates the height (Height_Ped) of the pedestrian 200 and the image height (IMG_Height_Ped(pix)) of the rectangle B201 detected in the pedestrian detection process (P113). , is used to obtain the distance from the own vehicle V to the pedestrian 200 . The distance Dist from the host vehicle V to the pedestrian 200 is calculated based on "Dist=f×(Height_Ped/IMG_Height_Ped)" (Formula (3)). Here, f is the focal length (pix) of the pair of

cameras

10a and 10b. Note that once the height of the pedestrian is calculated in the size estimation process (P115), the distance estimation process can be performed using the height of the pedestrian 200 in another image frame after the image shown in FIG. At (P116), the distance between the vehicle V and the pedestrian 200 can be obtained.

In this way, the external world recognition device 1 of this embodiment utilizes the actual height of any landmark 100 photographed in the compound eye area image CA to perform walking in the monocular area image (for example, the left monocular area image LA). The distance between the person 200 and the own vehicle V is calculated. Since the landmark 100 is a three-dimensional object and has texture information, parallax can be obtained more stably than the road surface. Therefore, it is possible to calculate the distance to the pedestrian 200 with higher accuracy even in a situation where it is difficult to obtain the parallax with respect to the road surface, such as at night or in rainy weather.

In addition, the external world recognition device 1 of this embodiment measures the actual height of the landmark 100 in the compound eye area image CA, and uses the result to estimate the height of the pedestrian 200 . Pedestrian 200 changes its size in the horizontal direction according to the movement of the hand during walking. On the other hand, the height of the pedestrian 200 does not change or fluctuates little even when the pedestrian 200 walks. Therefore, by estimating the height of pedestrian 200 and obtaining the distance between pedestrian 200 and own vehicle V based on the estimated height, the distance to pedestrian 200 can be obtained with high accuracy. Further, when a method of estimating the distance between the pedestrian 200 and the own vehicle V from the contact position of the pedestrian 200 on the road surface and the mounting posture of the camera on the road surface is adopted, the mounting posture of the camera on the road surface can be accurately estimated. There is a need. In contrast, by estimating the height of the pedestrian 200 and obtaining the distance as in the present embodiment, the distance between the own vehicle V and the pedestrian 200 can be calculated without being affected by changes in the posture of the camera. can be estimated.

<Modification 1>
Next, Modification 1 of the first embodiment of the present invention will be described.

FIG. 7 is a functional block diagram showing a schematic configuration of an external world recognition device 1d according to Modification 1 of the first embodiment. FIG. 8 is a flowchart showing distance estimation processing of the external world recognition device 1d of FIG. FIG. 9 is a diagram for explaining proximity determination processing of the external world recognition device 1d of FIG. The external world recognition device 1d according to Modification 1 differs from the external world recognition device 1 described above in that it includes a proximity determination unit 56, which will be described later. Hereinafter, configurations having the same or similar functions as those of the external world recognition device 1 described above are denoted by the same reference numerals, and descriptions thereof are omitted, and different portions are described.

As shown in FIG. 7, the distance estimation unit 50 includes a proximity determination unit 56. The proximity determination unit 56 determines whether or not the landmark 100 detected by the monocular area landmark detection unit 52 and the pedestrian 200 detected by the monocular area object detection unit 54 are three-dimensionally close. Specifically, as shown in FIG. 9, the proximity determination unit 56 determines whether the lower end of the landmark 100 in the left monocular area image LA (monocular area image) and the pedestrian 200 in the left monocular area image LA (monocular area image) Calculate the difference in the height direction from the bottom edge. If the difference calculated by the proximity determination unit 56 is within a margin (proximity determination threshold value) δ described later, and the size estimation unit 58 determines that the landmark 100 and the pedestrian 200 are close to each other, , to estimate the height of the pedestrian 200 .

As shown in FIG. 8, the external world recognition device 1d according to Modification 1 performs proximity determination processing ( P114) is executed. In the proximity determination process (P114), the proximity determination unit 56 spatially determines the landmark 100 detected by the monocular area landmark detection unit 52 and the pedestrian 200 detected by the monocular area object detection unit 54. It is determined whether or not it exists at a close position. For example, in the proximity determination process (P114), the proximity determination unit 56 uses the lower end of the landmark 100. FIG. As shown in FIG. 9, the proximity determination unit 56 sets a predetermined margin δ (for example, 5 pix) in the height direction around the lower end of the landmark 100, and the lower end of the pedestrian 200 is included in the margin δ. determine whether or not When the lower end of the pedestrian 200 is included in the margin δ, the proximity determination unit 56 determines that the landmark 100 and the pedestrian 200 are at approximately the same distance from the host vehicle V, and determines that the two are close to each other. It is determined that When the proximity determination unit 56 determines that the landmark 100 and the pedestrian 200 are close to each other, in the size estimation process (P115), the size estimation unit 58 determines the size of the landmark 100 stored in the landmark information storage unit 36. The height of the pedestrian 200 is estimated using the actual height. Note that the above margin δ (eg, 5 pix) may be stored in the landmark information storage unit 36 .

In the external world recognition device 1d of Modification 1, the proximity determination unit 56 determines whether or not the landmark 100 is used for estimating the height of the pedestrian 200. When estimating the height of the pedestrian 200, the ratio of the size (ie, image height) of the pedestrian 200 and the landmark 100 on the image is used. However, when the pedestrian 200 and the landmark 100 exist at distant positions, the ratio of the image heights of the two may deviate from the ratio of the actual heights of the two, and the height of the pedestrian 200 cannot be accurately estimated. Sometimes. Therefore, the height of the pedestrian 200 can be estimated more accurately by the proximity determination unit 56 performing the proximity determination process (P114) as in the first modification.

Next, another example of the external world recognition device 1d according to Modification 1 will be described. FIG. 10 is a diagram showing another example of the proximity determination process (P114) of the external world recognition device 1d of FIG. In FIG. 10, the landmark registration process (P105) is described in the upper part, and the proximity determination process (P114) is described in the lower part. The example shown in FIG. 10 differs from the external world recognition device 1d according to Modification 1 described above in terms of landmark registration processing (P105) and proximity determination processing (P114). In the example shown in FIG. 10, the same reference numerals as those of the external world recognition device 1d (FIG. 7) according to Modification 1 are attached and the description thereof will be made.

In the example shown in FIG. 10, the proximity determination unit 56 executes the proximity determination process (P114) based on the parallax around the landmark 100 detected by the compound eye region landmark detection unit 32. For example, as shown in the upper part of FIG. 10, the compound eye area landmark detection unit 32 detects the inclination of the surface on which the landmark 100 is on the ground based on the parallax information around the surface on which the detected landmark 100 is on the ground. change may be calculated. As a result, the location where the inclination of the surface on which the landmark 100 is grounded is identified. After that, the size information measurement unit 33 measures the image height (Obj_Height) of the landmark 100 detected by the compound eye area landmark detection unit 32 and the image from the center position of the landmark 100 to the point where the inclination changes. The ratio Plane_rate to the image length (Plane_Distance) of is calculated by "Plane_rate=(Plane_Distance/Obj_Height)" (equation (4)). Then, the landmark information storage unit 36 stores, for example, the value of Plane_rate as the proximity determination radius in addition to the predetermined margin δ (landmark registration process (P105)).

As shown in the lower part of FIG. 10, the proximity determination unit 56 uses the image height (Obj_Height2) of the landmark 100 re-detected by the monocular area landmark detection unit 52 in the proximity determination process (P114), Calculate the margin δ2 (Plane_rate×Obj_Height2). Then, the proximity determination unit 56 determines that the pedestrian 200 is close to the landmark 100 when the pedestrian 200 approaches within δ2 (pix) in the horizontal direction in addition to the margin δ (pix) in the vertical direction. judge.

In the size estimation process (P115), the height of the pedestrian 200 is estimated on the assumption that the pedestrian 200 and the landmark 100 are at the same distance from the own vehicle V. However, if the inclination of the surface where the landmark 100 touches the ground changes in the middle, if only the vertical margin δ is referred to in the proximity determination process (P114), the pedestrian 200 and the landmark 100 are not at the same distance. In spite of this, it may be determined that the landmark 100 and the pedestrian 200 are close to each other. Therefore, based on the parallax information, the area (δ2 in the lower part of FIG. 10) up to the point where the inclination of the road surface does not change is calculated, and based on this result, the proximity determination process (P114) is performed, so that both are at equal distances or not. Therefore, the height of pedestrian 200 can be estimated more accurately.

<Modification 2>
Next, Modification 2 of the first embodiment of the present invention will be described.

The external world recognition device 1e according to Modification 2 differs from the external world recognition device 1 described above in that it includes a landmark information measurement unit 34, which will be described later. Hereinafter, configurations having the same or similar functions as those of the external world recognition device 1 described above are denoted by the same reference numerals, and descriptions thereof are omitted, and different portions are described.

FIG. 11 is a functional block diagram showing a schematic configuration of an external world recognition device 1e according to modification 2 of the first embodiment. As shown in FIG. 11 , the landmark information acquisition section 30 includes a landmark information measurement section 34 .

The landmark information measurement unit 34 determines whether the

landmarks

100, 101, and 102 detected by the compound eye area landmark detection unit 32 have moved from the compound eye area image CA. judge. Then, the landmark information storage unit 36 stores the landmarks determined to be moving by the landmark information measurement unit 34 among the

landmarks

100, 101, and 102 included in the compound eye area image CA (for example, the landmarks in FIG. The mark 101) is stored as a landmark for estimating the distance to the pedestrian 200. FIG.

In the external world recognition device 1e according to Modification 2, not only can stationary three-dimensional objects be registered as landmarks, but also three-dimensional objects such as other pedestrians and vehicles can be used as landmarks as mobile objects. Since the landmark 100 shown in FIG. 4 is a stationary object, if the pedestrian 200, who is the target of distance calculation with respect to the own vehicle V, is stationary, the two do not approach each other. Therefore, the height of stationary pedestrian 200 cannot be estimated based on landmark 100, which is a stationary object. On the other hand, by using a moving object (for example, the landmark 101) as in Modification 2, even when the pedestrian 200, which is the object of distance calculation, is stationary, the landmark 101 is the pedestrian. It can get close to 200. Therefore, it becomes possible to estimate the height of the stationary pedestrian 200 as well.

<Modification 3>
Next, Modification 3 of the first embodiment of the present invention will be described.

The external world recognition device according to Modification 3 differs from the external world recognition device described above in terms of the function of the landmark information storage unit 36 . Hereinafter, the external world recognition device 1 according to Modification 3 will be described with the same reference numerals as the external world recognition device 1 described above, with reference to FIGS.

In modification 3, the landmark information storage unit 36 shown in FIG. ) above (for example, the

landmarks

100 and 102 in FIG. 4) are stored as landmarks for estimating the distance to the pedestrian 200 . Thus, in Modification 3, among the

landmarks

100, 101, and 102 measured in the three-dimensional information acquisition process (P104), the actual height of the landmark (eg , 100 and 102 in FIG. 4) are registered in the landmark information storage unit 36 in the landmark registration process (P105). Then, in the landmark detection process (P112), the landmarks registered in the landmark registration process (P105) and having an actual height equal to or greater than the height threshold (for example, the

landmarks

100 and 102 in FIG. 4) are detected again. .

If a process such as template matching is used when re-detecting a landmark with a low actual height in the landmark detection process (P112), the detected position may be misaligned. When the detection position is misaligned, the image height of the landmark detected by the monocular area landmark detection unit 52 changes from the true image height of the landmark. The estimated height of pedestrian 200 may be deviated. In this regard, the external world recognition device according to Modification 3 uses only landmarks having a height equal to or greater than a predetermined height threshold (for example, 1 m). Therefore, the difference between the height of the detected landmark image and the true height of the image can be reduced, and the height of the pedestrian can be estimated more accurately.

As another example of Modified Example 3, the landmark information storage unit 36 selects the plurality of

landmarks

100, 101, and 102 measured by the size information measuring unit 33 in descending order of actual height. (for example, the number corresponding to 60% of the plurality of landmarks) (for example, the landmarks 100 and 102) may be stored as landmarks for estimating the distance to the pedestrian 200. FIG. As a result, many options of landmarks used for distance estimation to the pedestrian 200 can be held.

<Modification 4>
Next, Modification 4 of the first embodiment of the present invention will be described.

The external world recognition device according to Modified Example 4 differs from the external world recognition device 1e (FIG. 11) according to Modified Example 2 in terms of the functions of the landmark information measurement unit 34 and the landmark information storage unit 36. Hereinafter, the external world recognition device according to Modified Example 4 will be described with reference to FIGS.

In Modified Example 4, the landmark information measurement unit 34 calculates the contrast information (edge component , color, luminance difference, etc.). Then, of the

landmarks

100, 101, and 102 included in the compound eye area image CA, the landmark information storage unit 36 selects a landmark having contrast information equal to or greater than a predetermined threshold value (for example, the landmark 100 in FIG. 4), It is stored as a landmark for estimating the distance to pedestrian 200 .

Specifically, in Modified Example 4, the landmark information measurement unit 34 applies a Sobel filter to each of the

landmarks

100, 101, and 102 in the landmark registration process (P105) to obtain edge components (contrast information). In this process, the landmark information storage unit 36 selects landmarks (for example, the landmark 100) whose density of edge components (the number of edges) is equal to or greater than a predetermined value within the areas of the

landmarks

100, 101, and 102. register. Alternatively, a landmark (for example, landmark 100) whose color information (contrast information) is equal to or greater than a predetermined value may be registered. Specifically, pixels with saturation and brightness equal to or greater than a predetermined value are counted in a hue region that can be regarded as the red component of an image, and if the number of pixels is equal to or greater than a predetermined value, it is registered as a landmark. good too. As a result, in the landmark detection process (P112), the landmark 100 can be re-detected more stably, and erroneous detection of other three-dimensional objects as landmarks can be avoided. Thereby, the height of pedestrian 200 can be estimated with higher accuracy.

Also, in the landmark registration process (P105), the landmark information measurement unit 34 analyzes the luminance difference between the lower ends of the

landmarks

100, 101, and 102 and the road surface. In this process, the landmark information storage unit 36 stores landmarks (for example, landmarks) in which the number of pixels whose brightness difference between the lower end and the road surface is equal to or greater than a predetermined value among the

landmarks

100, 101, and 102 is equal to or greater than a predetermined value. store the mark 100). This makes it easier for the monocular area landmark detection unit 52 to re-detect the same area as the area detected by the compound eye area landmark detection unit 32, thereby estimating the height of the pedestrian 200 with higher accuracy. can be done. Further, when the external world recognition device according to the fourth modification includes the proximity determination unit 56, when using the lower end of the landmark 100 in the proximity determination process (P114), it is possible to register the landmark 100 that is highly separable from the road surface. , the proximity determination can be performed more accurately.

Also, the landmark information storage unit 36 can register a specific part of the landmark in the landmark registration process (P105). Specifically, the landmark information measuring unit 34 measures, for example, a portion of the landmark 100 where the edge component is strong as the upper end of the landmark. Then, in the processing, the landmark information storage unit 36 can register the portion as the upper end of the landmark. This makes it easier for the monocular area landmark detection section 52 to re-detect the same portion as the portion detected by the compound eye area landmark detection section 32 . In this way, by registering, for example, a specific portion of the landmark 100 based on a portion with a strong edge component, the landmark can be stably re-detected by the monocular area landmark detection unit 52. Pedestrian distance can be estimated with high accuracy.

<Modification 5>
Next, Modification 5 of the first embodiment of the present invention will be described.

The external world recognition device according to modification 5 differs from the external world recognition device 1 described above in terms of the function of the size estimation unit 58 . Hereinafter, the external world recognition device according to Modification 5 will be described with the same reference numerals as the external world recognition device 1 described above, with reference to FIGS.

In the external world recognition device according to Modification 5, when a plurality of

landmarks

100, 101, and 102 are registered in the landmark information storage unit 36, the size estimation unit 58 uses the plurality of landmarks to estimate the pedestrian 200. Estimate the size of Specifically, the size estimation unit 58 estimates the height (Height_Ped) of the pedestrian 200 when approaching the landmark 100 in a certain image frame. Then, the distance calculation unit 60 calculates the distance between the vehicle V and the pedestrian 200 based on the Height_Ped. After that, in another image frame, when the pedestrian 200 approaches the landmark 101, the size estimator 58 calculates the pedestrian's Update height Height_Ped. Here, Height_Ped_2 is the height of the pedestrian 200 estimated from the actual height of the landmark 101, the image height of the landmark 101, and the height of the pedestrian 200 image. In the size estimation process (P115), size estimation unit 58 updates the height of pedestrian 200 each time pedestrian 200 approaches a plurality of landmarks. As a result, even if the initial height estimation result contains an error, the height can be updated so as to approach the correct value each time it approaches each landmark, and the distance of the pedestrian can be estimated more accurately. can.

<Second embodiment>
Next, a second embodiment of the invention will be described.

FIG. 12 is a functional block diagram showing a schematic configuration of the external world recognition device 1a according to the second embodiment of the present invention. The external world recognition device 1a according to the second embodiment differs from the external world recognition device 1d (Modification 1) in that it includes a landmark selection unit 40 and a course estimation unit 70, which will be described later. Configurations having the same or similar functions as those of the external world recognition device 1d described above are denoted by the same reference numerals as those of the external world recognition device 1d, and descriptions thereof are omitted. Different parts will be explained.

As shown in FIG. 12, the external world recognition device 1a includes a landmark selection unit 40 and a traveling route estimation unit 70. Landmark selection unit 40 is included in landmark information acquisition unit 30 . When the landmark information acquisition unit 30 of the external world recognition device 1a according to the second embodiment acquires the three-dimensional information of the plurality of

landmarks

100, 101, and 102, the landmark information acquisition unit 30 determines the location of the plurality of landmarks according to the movement of the vehicle V. Select a landmark (eg, landmark 100) to be used for distance estimation. Specifically, the landmark selection unit 40 of the landmark information acquisition unit 30 selects the landmark (for example, the landmark 100) used for distance estimation from a plurality of landmarks. More specifically, the landmark selection unit 40 selects a landmark 100 existing in the traveling direction of the vehicle V estimated by the traveling path estimation unit 70 described later as a landmark used for estimating the distance to the pedestrian 200. select. The landmark information storage section 36 stores the landmark 100 selected by the landmark selection section 40 .

The traveling path estimation unit 70 estimates the traveling direction of the vehicle V from the movement of the vehicle V (that is, behavior information). Specifically, the traveling route estimation unit 70 predicts the traveling route of the own vehicle V based on the speed of the vehicle V, the yaw rate, and the like. For example, the traveling path estimator 70 estimates the turning radius of the vehicle V from the values of the vehicle speed and the yaw rate, and sets the locus on the circumference as the traveling path. Note that, when the vehicle V is traveling straight ahead, the traveling path estimation unit 70 may estimate a trajectory along the straight traveling direction as the traveling path.

As described above, in the external world recognition device 1a according to the second embodiment, in the landmark registration process (P105), only the landmark (for example, the landmark 100) existing in the traveling direction estimated by the traveling path estimation unit 70 is registered as a landmark. Registered in the mark information storage unit 36 . For example, the landmark information storage unit 36 may store landmarks 100 that are within a predetermined distance (20 m) from the course of the vehicle V. FIG. By selecting the landmarks to be registered based on the traveling route information of the vehicle V in this manner, it is possible to register only the landmarks used for estimating the distance between the pedestrian 200 who is at risk of colliding with the vehicle V. can. As a result, by reducing the number of registered landmarks, it is possible to limit the amount of memory used and the number of landmarks detected in the landmark detection process (P112), thereby reducing the processing load on the external world recognition device 1a. can be done.

<Third Embodiment>
Next, a third embodiment of the invention will be described.

FIG. 13 is a functional block diagram showing a schematic configuration of an external world recognition device 1b according to the third embodiment of the present invention. The external world recognition device 1b according to the third embodiment differs from the external world recognition device 1d (Modification 1) in that it includes a landmark selection unit 40 and a surrounding environment recognition unit 90, which will be described later. Configurations having the same or similar functions as those of the external world recognition device 1d described above are denoted by the same reference numerals as those of the external world recognition device 1d, and descriptions thereof are omitted. Different parts will be explained.

As shown in FIG. 13, the external world recognition device 1b includes a landmark selection unit 40 and a surrounding environment recognition unit 90 that recognizes the surrounding environment of the vehicle V. Landmark selection unit 40 is included in landmark information acquisition unit 30 . The landmark information acquisition unit 30 of the external world recognition device 1b according to the third embodiment recognizes the landmarks (for example, the landmark 100) existing in the area recognized as the sidewalk by the surrounding environment recognition unit 90 up to the pedestrian 200. as landmarks for distance estimation. Specifically, the landmark selection unit 40 of the landmark information acquisition unit 30 selects the landmark (for example, the landmark 100) used for estimating the distance to the pedestrian 200. FIG. The landmark information storage unit 36 stores the landmarks selected by the landmark selection unit 40 .

The surrounding environment recognition unit 90 estimates type information for each pixel of the images of the pair of

cameras

10 a and 10 b acquired by the image acquisition unit 10 . Here, the type information includes not only information on objects such as vehicles and pedestrians, but also information on road surfaces, sidewalks, and the like. A convolutional neural network is used to estimate the type information for each pixel. The surrounding environment recognition unit 90 estimates the type information corresponding to each pixel by using a model learned from the correct value data to which the type information for each pixel is added. As a result, the surrounding environment recognition unit 90 can recognize the sidewalk area from the images captured by the pair of

cameras

10a and 10b.

Thus, in the external world recognition device 1b according to the third embodiment, in the landmark registration process (P105), the landmark registration process is executed using the type information estimated by the surrounding environment recognition unit 90. Specifically, the landmark information storage unit 36 stores the landmark 100 that exists on the pixel that the surrounding environment recognition unit 90 has estimated as the sidewalk. As a result, only landmarks 100 that are likely to be close to pedestrian 200 can be registered, and the processing load on external world recognition device 1b can be reduced.

In the above-described embodiment, the pair of

cameras

10a and 10b are used as one stereo camera in order to acquire the three-dimensional information (size information) of the

landmarks

100, 101, and 102 in the landmark information acquisition unit 30. ing. However, a three-dimensional sensor (for example, lidar, millimeter wave radar, ultrasonic sensor, etc.) may be used to acquire the three-dimensional information (size) of the landmark. In this case, the pair of

cameras

10a and 10b may be installed so as not to have a stereo area. For example, the camera 10 a may capture only the left monocular region, the camera 10 b may capture only the right monocular region, and the images captured in these regions may be input to the image acquisition unit 10 .

Also, the distance between the vehicle V and the pedestrian 200 may be estimated using specific landmarks described on a digital map or the like and using three-dimensional information registered in the digital map. . Further, in the above-described embodiment, a detailed description was given from the viewpoint of estimating the height of pedestrian 200, but based on the width of pedestrian 200, the distance between own vehicle V and pedestrian 200 is calculated. good too. For example, the distance estimating unit 50 estimates the distance to the pedestrian 200 based on the actual width of the landmark 100 and the lateral length (i.e. image width) of the landmark 100 and the pedestrian 200 in the image. may In addition, the method of estimating the distance to the pedestrian 200 has been described by taking the pedestrian 200 as an example of the object, but various objects such as vehicles and animals can be treated as objects for distance estimation. The distance between these and the host vehicle V can be estimated based on the heights of the V and the like.

It should be noted that the present invention is not limited to the above embodiments, and includes various modifications. For example, the above embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Moreover, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

In addition, each of the above configurations, functions, processing units, processing means, etc., may be realized by hardware, for example, by designing them in integrated circuits, in part or in whole. Moreover, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tapes, and files that implement each function can be stored in recording devices such as memories, hard disks, SSDs (solid state drives), or recording media such as IC cards, SD cards, and DVDs.

In addition, the control lines and information lines indicate what is considered necessary for explanation, and not all control lines and information lines are necessarily indicated on the product. In practice, it may be considered that almost all configurations are interconnected.

1, 1a, 1b, 1d, 1e external world recognition device, 10 image acquisition unit, 10a, 10b pair of cameras (stereo cameras), 30 landmark information acquisition unit, 32 compound eye region landmark detection unit, 33 size information measurement unit , 34 landmark information measurement unit, 36 landmark information storage unit, 40 landmark selection unit, 50 distance estimation unit, 52 monocular area landmark detection unit, 54 monocular area object detection unit, 56 proximity determination unit, 58 size Estimation unit 60 Distance calculation unit 70 Course estimation unit 90 Surrounding

environment recognition unit

100, 101, 102 Landmark 200 Pedestrian (object) CA Compound eye area image LA Left monocular area image (monocular area image ), RA right monocular area image (monocular area image), V vehicle, δ margin (proximity judgment threshold)

Claims

An external world recognition device mounted on a vehicle,
an image acquisition unit that acquires images of landmarks and objects;
a landmark information acquisition unit that acquires three-dimensional information of the landmark;
and a distance estimating unit that estimates a distance to the object based on three-dimensional information of the landmark and sizes of the landmark and the object in the image. Device.
In the external world recognition device according to claim 1,
The image acquisition unit acquires an image captured by a stereo camera and including a compound eye region with overlapping fields of view and a monocular region with non-overlapping fields of view,
The landmark information acquisition unit acquires three-dimensional information of landmarks located in the compound eye region,
The external world recognition device, wherein the distance estimation unit estimates a distance to the object located in the monocular area.
In the external world recognition device according to claim 2,
The landmark information acquisition unit
a compound eye area landmark detection unit that detects the landmark from the compound eye area;
a size information measuring unit that measures the size of the landmark included in the three-dimensional information from the compound eye region;
The distance estimation unit
a monocular area landmark detection unit that detects the landmark from the monocular area;
a monocular area object detection unit that detects the object from the monocular area;
a size estimation unit that estimates the size of the object from the size of the landmark based on the image size of the landmark in the monocular region and the image size of the object in the monocular region;
and a distance calculation unit that calculates the distance based on the size of the object estimated by the size estimation unit and the image size of the object in the monocular region.
In the external world recognition device according to claim 3,
The size information measuring unit calculates an actual height of the landmark from the compound eye area,
The monocular area landmark detection unit calculates an image height of the landmark from the monocular area,
The monocular area object detection unit calculates an image height of the object from the monocular area,
The size estimating unit estimates the height of the object from the actual height of the landmark based on the image height of the landmark and the image height of the object,
The external world recognition device, wherein the distance calculation unit calculates the distance based on the estimated height.
In the external world recognition device according to claim 4,
The size estimation unit estimates the height of the object by multiplying the actual height of the landmark by a ratio of the image height of the object to the image height of the landmark. An external world recognition device characterized by
In the external world recognition device according to claim 4,
The distance estimating unit includes a proximity determining unit that calculates a height difference between the lower end of the landmark in the monocular region and the lower end of the object in the monocular region,
The external world recognition device, wherein the size estimation unit estimates the height of the object when the difference calculated by the proximity determination unit is within a proximity determination threshold value.
In the external world recognition device according to claim 2,
The landmark information acquisition unit
a landmark information measuring unit that determines whether or not the landmark has moved from the compound eye area;
a landmark information storage unit for storing, among landmarks included in the compound eye region, landmarks determined to be moving by the landmark information measurement unit as landmarks for estimating a distance to the object; An external world recognition device comprising:
In the external world recognition device according to claim 4,
The landmark information acquisition unit
An external world recognition device, comprising: a landmark information storage unit that stores a landmark whose actual height is equal to or greater than a height threshold as a landmark for estimating a distance to the object.
In the external world recognition device according to claim 2,
The landmark information acquisition unit
a landmark information measuring unit that calculates contrast information of the landmark from the compound eye area;
and a landmark information storage unit that stores, among the landmarks included in the compound eye area, landmarks having the contrast information equal to or greater than a predetermined threshold value as landmarks for estimating the distance to the object. Characteristic external recognition device.
In the external world recognition device according to claim 1,
When the landmark information acquisition unit acquires the three-dimensional information of a plurality of landmarks, the landmark information acquisition unit selects a landmark to be used for estimating the distance from the plurality of landmarks according to the movement of the vehicle. An external world recognition device characterized by
In the external world recognition device according to claim 10,
A traveling path estimation unit for estimating the traveling direction of the vehicle from the movement of the vehicle,
The landmark information acquiring unit selects a landmark existing in the traveling direction of the vehicle estimated by the traveling route estimating unit as a landmark used for estimating the distance to the object. A device that recognizes the external world.
In the external world recognition device according to claim 1,
A surrounding environment recognition unit that recognizes the surrounding environment of the vehicle,
The landmark information acquisition unit selects a landmark existing in an area recognized as a sidewalk by the surrounding environment recognition unit as a landmark used for estimating the distance to the object. A device that recognizes the external world.