JP2023109519A - Display control device and display control program - Google Patents

Abstract

To provide a technology that enables superimposed display with fewer displacements or senses of discomfort in a display control device and a display control program for controlling display operation in a display device that, by being mounted to a host vehicle, superimposedly displays a display image in the foreground of the host vehicle.SOLUTION: A recognition result acquisition unit (121) acquires recognition result information that corresponds to the image recognition result of a target based on the foreground image captured by an image capturing device (13). A reliability calculation unit (125) calculates the reliability of recognition result information on the basis of feature quantity information that represents intermediate information in a process in which recognition result information is acquired or the recognition result information. A display state setup unit (126) sets, on the basis of the reliability calculated by the reliability calculation unit, a display state that represents the status of a display image being displayed or not, or the visibility of the display image.SELECTED DRAWING: Figure 2

Description

The present invention relates to a display control device and a display control program that control display operations in a display device that is mounted on a vehicle and displays a display image superimposed on the foreground of the vehicle.

The driving assistance system described in Patent Document 1 superimposes predetermined image information on the windshield of the vehicle, and the length of the arrow indicating the guidance direction in the image information is determined according to the likelihood of the vehicle position information. is characterized by changing Such a driving assist system prevents the driver from being unnecessarily confused by displaying guidance using an arrow with a short arrowhead when the certainty of the vehicle position information is low.

JP 2011-203053 A

As described in Japanese Unexamined Patent Application Publication No. 2002-100002, conventionally, a technique for performing various displays such as route guidance using a so-called head-up display is known. Furthermore, in recent years, in this type of display technology, target objects such as white lines are recognized using image processing technology used for ADAS, and the position information of the vehicle posture and target target is used to enable the driver to visually recognize the target. There is a demand for presenting information without positional deviation in the scene where the user is present, such as a display that assists recognition, judgment, or the like. ADAS stands for Advanced Driver Assistance System.

Here, for example, in image recognition technology, the accuracy of position detection by recognition deteriorates under bad conditions such as a situation in which white lines are blurred and difficult to distinguish. Also, with the positional accuracy of in-vehicle navigation in the widespread range, there is a possibility that the front, rear, left, and right positions to be guided may deviate by about 10 m. Specifically, such positional deviation occurs in situations where positioning signals from satellites such as GPS cannot be received or are difficult to receive, for example, due to the influence of high-rise buildings, tunnels, multi-story parking lots, and the like. Further, such a positional deviation occurs, for example, when the actual road shape is changed from the map information and the position of the vehicle is estimated by deviation when performing map matching with the map.

The present invention has been made in view of the circumstances exemplified above. That is, the present invention provides, for example, a technology that enables superimposed display with less misalignment and discomfort.

A display control device (12) according to claim 1 controls the display operation of a display device (11) that superimposes and displays a display image (IM) on the foreground of a vehicle (1) by being mounted on the vehicle (1). is configured to
This display control device
a recognition result acquisition unit (121) for acquiring recognition result information corresponding to a target image recognition result based on the foreground captured image by an imaging device (13); a reliability calculation unit (125) that calculates based on feature amount information, which is intermediate information in the process of acquiring information or the recognition result information;
a display state setting unit (126) for setting a display state in the display image based on the reliability calculated by the reliability calculation unit;
It has
The display control program according to claim 9 is a program executed by the display control device,
The processing executed by the display control device includes:
a recognition result acquisition process for acquiring recognition result information corresponding to a target image recognition result based on the foreground captured image by the imaging device (13);
Reliability calculation processing for calculating the reliability of the recognition result information based on feature amount information, which is intermediate information in the process of acquiring the recognition result information or the recognition result information;
display state setting processing for setting a display state in the display image based on the reliability calculated by the reliability calculation processing;
including.

In the application documents, each element may be given a reference sign with parentheses. However, even in this case, such reference numerals merely indicate an example of the correspondence relationship between each element and specific means described in the embodiments described later. Therefore, the present invention is not limited in any way by the above reference numerals.

It is a figure showing a schematic structure of an in-vehicle system containing a display control device concerning one embodiment of the present invention. 2 is a block diagram showing a schematic functional configuration of the in-vehicle system shown in FIG. 1; FIG. 3 is a block diagram showing an overview of reliability calculation according to the first embodiment of the display control device shown in FIG. 2; FIG. It is a schematic diagram showing a display example according to a comparative example. It is a schematic diagram showing an example of a display concerning a first embodiment. 3 is a block diagram showing an overview of reliability calculation according to the second embodiment of the display control device shown in FIG. 2; FIG. It is a schematic diagram showing a display example according to a comparative example. It is a schematic diagram showing a display example according to the second embodiment. 3 is a diagram showing an overview of reliability calculation according to the third embodiment of the display control device shown in FIG. 2; FIG. FIG. 10 is a diagram showing an outline of reliability calculation according to the fourth embodiment of the display control device shown in FIG. 2; FIG. 10 is a diagram showing an outline of reliability calculation according to the fifth embodiment of the display control device shown in FIG. 2; FIG. 10 is a diagram showing an overview of reliability calculation according to the sixth embodiment of the display control device shown in FIG. 2; FIG. 11 is a diagram showing an overview of reliability calculation according to the seventh embodiment of the display control device shown in FIG. 2; FIG. 11 is a block diagram showing an outline of reliability calculation according to the eighth embodiment of the display control device shown in FIG. 2; FIG. 11 is a time chart showing an operation example according to the eighth embodiment of the display control device shown in FIG. 2; FIG. FIG. 11 is a diagram showing an outline of reliability calculation according to the eighth embodiment of the display control device shown in FIG. 2;

(embodiment)
BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described based on the drawings. It should be noted that if various modifications applicable to one embodiment are inserted in the middle of a series of explanations related to the embodiment, there is a risk that the understanding of the embodiment will be hindered. Therefore, the modified examples will be collectively described after the series of descriptions regarding the embodiment, not in the middle of the description.

(in-vehicle system)
Referring to FIG. 1, a vehicle 1 is a so-called four-wheeled vehicle, and includes a substantially rectangular body 2 in plan view when the vehicle 1 is viewed from vertically above. For convenience of the following explanation, the concepts of "front", "rear", "left", "right", "up" and "down" in the vehicle 1 are defined as indicated by arrows in FIG. These directional concepts are based on the driver D who is an occupant in the driver's seat of the vehicle 1 traveling forward (that is, traveling in a shift range other than reverse). The longitudinal direction is synonymous with the longitudinal direction of the vehicle. Moreover, the left-right direction is synonymous with the vehicle width direction. Further, the vertical direction is synonymous with the vehicle height direction. The vehicle length direction is a direction orthogonal to the vehicle width direction and orthogonal to the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the vehicle 1, and is a direction parallel to the direction of gravity when the vehicle 1 is stably placed on a horizontal plane in a drivable state.

A vehicle interior 3, which is an internal space in which an occupant rides in the vehicle body 2, is covered in front by a front windshield 4. - 特許庁The front windshield 4 is formed in a plate shape from translucent glass or synthetic resin. The front windshield 4 is slanted so that the upper end is located rearward than the lower end when viewed from the side in a line of sight parallel to the vehicle width direction.

A dashboard 5 is arranged below the front windshield 4 in the passenger compartment 3 . A meter panel 6 is provided on the dashboard 5 . A steering column 7 extends from the dashboard 5 toward the driver D. A steering wheel 8 is attached to the steering column 7 .

The vehicle 1 is equipped with an in-vehicle system 10 . The vehicle 1 equipped with the in-vehicle system 10 is hereinafter referred to as "own vehicle". The in-vehicle system 10 is configured to be capable of executing various controls during driving of the own vehicle and various display operations associated therewith. The in-vehicle system 10 is provided to configure an in-vehicle network conforming to a predetermined communication standard such as CAN (International Registered Trademark: International Registration No. 1048262A). CAN (International Registered Trademark) is an abbreviation for Controller Area Network.

The in-vehicle system 10 includes a head-up display device 11 that forms a display image IM using AR technology, and a display control device 12 configured to control the display operation of the head-up display device 11 . AR is an abbreviation for Augmented Reality. The head-up display device 11 is a display device that superimposes a display image IM, which is a virtual image, on the foreground (for example, the road surface) of the vehicle that is viewed through the front windshield 4 by being mounted on the vehicle. , is housed in the dashboard 5 below the front windshield 4 .

FIG. 2 schematically shows a functional block configuration related to operation control of the head-up display device 11 in the in-vehicle system 10. As shown in FIG. Hereinafter, a general head-up display display control configuration in the in-vehicle system 10 will be described with reference to FIGS. 1 and 2. FIG. The in-vehicle system 10 includes an imaging device 13 , a recognition unit 14 , a vehicle state sensor 15 , and a navigation device 16 in addition to a head-up display device 11 and a display control device 12 .

The imaging device 13 is a so-called front camera, and is provided so as to be able to capture images in front of and to the sides of the vehicle. Specifically, the image pickup device 13 is a digital camera device and includes an image sensor such as a CCD or CMOS. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor. Hereinafter, the imaging device 13 may be abbreviated simply as a "camera". The recognition unit 14 is configured to image-recognize a target (for example, a white line) included in the foreground based on the image of the foreground captured by the imaging device 13 . The "target" includes not only road signs such as the intersection IS, road division lines LD, stop lines LL, and pedestrian crossings ZC shown in FIG. 4A, but also three-dimensional objects such as traffic lights TL. In this embodiment, the imaging device 13 and the recognition unit 14 are configured to be able to detect the three-dimensional position of the target relative to the vehicle using monocular stereo or compound stereo technology. The recognition unit 14 is provided so as to be able to provide the target image recognition result to each unit including the display control device 12 .

The vehicle state sensor 15 is provided to detect the driving state and driving environment of the own vehicle. The "driving state" includes various quantities related to the driving operation state by the driver D or the driving automation system, such as accelerator opening, braking amount, shift position, steering angle, and the like. Also, the "driving state" includes physical quantities related to the behavior of the vehicle, such as vehicle speed (that is, running speed of the vehicle), angular velocity, longitudinal acceleration, lateral acceleration, and the like. The "driving environment" includes natural environment such as outside temperature, amount of rainfall, and illuminance. Also, the "driving environment" includes the state of presence of moving and stationary objects within a predetermined detection range around the vehicle. "Moving objects" include pedestrians, cyclists, animals, and other vehicles in motion. "Stationary objects" include structures such as walls and buildings, in addition to objects falling on the road, guardrails, curbs, other parked and stopped vehicles, and road signs. Moving and stationary objects are included in "targets". That is, the vehicle state sensor 15 includes various sensors such as an accelerator opening sensor, a steering angle sensor, a wheel speed sensor, an angular velocity sensor, an acceleration sensor, an outside temperature sensor, a raindrop sensor, an illuminance sensor, a radar sensor, an ultrasonic sensor, and the like. Those other than the imaging device 13 are collectively named for the sake of illustration and explanation simplification.

The navigation device 16 is configured to calculate the position of the vehicle and the traveling route of the vehicle based on the map information and the positioning information obtained from the positioning satellites. "Vehicle position" is the current position of the vehicle on the map information. The "moving route" is the planned travel route of the vehicle from the position of the vehicle to the destination. Further, the navigation device 16 has, as branch information, links connecting nodes indicating intersections, and is configured to be able to calculate the angle formed by the traveling direction of the own vehicle and the links.

(Display control device)
The display control device 12 has a configuration as an HCU. HCU is an abbreviation for HMI Control Unit. HMI stands for Human Machine Interface. The display controller 12 is configured as a so-called on-board computer with a processor and memory programmed to perform one or more functions embodied by a computer program. The display control device 12 includes a recognition result acquisition unit 121, an own vehicle position acquisition unit 122, a route information acquisition unit 123, a display information generation unit 124, as functional configurations or functional units realized on a microcomputer. A reliability calculation unit 125 and a display state setting unit 126 are provided.

The recognition result acquisition unit 121 acquires from the recognition unit 14 the image recognition result of the target based on the image of the foreground captured by the imaging device 13 (that is, recognition result information that is information corresponding to the image recognition result). ing. The own vehicle position acquisition unit 122 acquires the own vehicle position from the navigation device 16 . The route information acquisition unit 123 acquires the traveling route from the navigation device 16 .

The display information generation unit 124 is based on the recognition result information acquired by the recognition result acquisition unit 121, the own vehicle position acquired by the own vehicle position acquisition unit 122, and the traveling route acquired by the route information acquisition unit 123. Then, display information for displaying the display image IM on the head-up display device 11 is generated. The reliability calculation unit 125 calculates the reliability of the recognition result information based on the feature amount information. Here, the "feature amount information" includes recognition result information and/or intermediate information in the process of acquiring the recognition result information. The display state setting section 126 sets the display state of the display image IM based on the reliability calculated by the reliability calculation section 125 . The “display state” referred to here includes presence/absence of display and display form. The “display form” includes various elements (for example, brightness, saturation, hue, line diameter, display size, etc.) that affect the visibility of the display image IM.

That is, the processing executed by the display control device 12 includes recognition result acquisition processing, reliability calculation processing, and display state setting processing. The recognition result acquisition process is a process of acquiring recognition result information corresponding to the target image recognition result based on the foreground captured image by the imaging device 13 . The reliability calculation process calculates the reliability of the recognition result information acquired in the recognition result acquisition process, based on feature amount information, which is intermediate information in the process of acquiring the recognition result information or the recognition result information. processing. The display state setting process is a process of setting the display state in the display image IM based on the reliability calculated by the reliability calculation process.

(Overview of operation)
The operations of the in-vehicle system 10 and the display control device 12 according to the present embodiment, and the overview of the display control method and display control program executed thereby will be described below together with the effects achieved by the present embodiment. In the following description, the configurations of the in-vehicle system 10 and the display control device 12 according to this embodiment, and the series of processes of the display control method and display control program executed thereby are collectively referred to as "this embodiment". I have something to do.

Based on the image of the foreground captured by the imaging device 13, the recognition unit 14 recognizes an image of a target included in the foreground that can be visually recognized by the driver D through the front windshield 4 in front of and on the front side of the own vehicle. The target image recognition result by the recognition unit 14 is used for the automatic driving operation or the ADAS operation of the own vehicle, and is also used for the display operation by the head-up display device 11 .

The display control device 12 generates a display image IM, which is a virtual image, based on signals or information acquired from the recognition unit 14, the vehicle state sensor 15, and the navigation device 16, and uses this to superimpose various information on the foreground. indicate. Specifically, the recognition result acquisition unit 121 acquires from the recognition unit 14 the target image recognition result based on the captured image of the foreground captured by the imaging device 13 . The own vehicle position acquisition unit 122 acquires the own vehicle position from the navigation device 16 . The route information acquisition unit 123 acquires the traveling route from the navigation device 16 . The display information generating unit 124 generates the target image recognition result obtained by the recognition result obtaining unit 121, the own vehicle position obtained by the own vehicle position obtaining unit 122, and the traveling route obtained by the route information obtaining unit 123. and display information for superimposing the display image IM on the foreground.

Here, in the image recognition technology, the recognition accuracy greatly depends on the environment in which the image is captured and the state of the target as the image capturing target. Therefore, it is not always possible to recognize images with the same accuracy. Specifically, for example, the accuracy of position detection by recognition deteriorates under bad conditions such as a situation in which white lines are blurred and it is difficult to distinguish. Therefore, if the image recognition result is simply used, display control is performed using the image recognition result as it is even if the recognition accuracy is degraded. Also, with the positional accuracy of in-vehicle navigation in the widespread range, there is a possibility that the front, rear, left, and right positions to be guided may deviate by about 10 m. Specifically, such positional deviation occurs in situations where positioning signals from satellites such as GPS cannot be received or are difficult to receive, for example, due to the influence of high-rise buildings, tunnels, multi-story parking lots, and the like. Further, such a positional deviation occurs, for example, when the actual road shape is changed from the map information and the position of the vehicle is estimated by deviation when performing map matching with the map. As a typical example, the accuracy of image recognition tends to decrease at a distance (for example, about 100 m ahead), and correspondingly, the superimposed display corresponding to the distant target also causes display deviation and uncomfortable display. Cheap.

In this respect, the inventor has proposed that by expressing image recognition accuracy in the form of reliability and performing display control of the display image IM according to the reliability, it is possible to suppress display deviation and discomfort as much as possible. I found out to be Specifically, for example, by correcting (for example, multiplying) the calculated value or estimated value of the target information corresponding to the image recognition result by the reliability, or performing the display control threshold determination according to the reliability, The display state of the display image IM can be set according to the reliability. This reliability can be calculated using the image recognition result and feature amount information, which is intermediate information generated when deriving this. Further, for example, by using an integrated reliability obtained by integrating reliability obtained from different feature amount information by arbitrary integration means (for example, Bayesian estimation, etc.), it is possible to further suppress display deviation and discomfort. . As described above, according to the present embodiment, superimposed display can be performed with less display deviation and discomfort.

(First embodiment: road gradient)
3, 4A, and 4B show an example of technology for suppressing road gradient deviation in route display. That is, FIG. 3 shows an operation example of the reliability calculation unit 125 shown in FIG. FIG. 4A shows, as a comparative example, a display example in which a display image IM for displaying route information in consideration of road gradients is simply displayed based on recognition results without using reliability. FIG. 4B shows a display example according to this embodiment. In FIGS. 4A and 4B, it is assumed that the own vehicle is approaching the intersection IS ahead while traveling straight on the running lane LR, which is the right lane of the running road RR with two lanes in each direction. Display examples according to the present embodiment will be described below with reference to FIGS. 1 to 3 and FIGS. 4A and 4B.

In this embodiment, the recognition unit 14 recognizes the shape of the road ahead of the vehicle. That is, the recognition unit 14 calculates, or estimates, the road gradient of the destination road. The recognition result acquisition unit 121 acquires the estimation result of the road gradient of the destination road from the recognition unit 14 . The display information generation unit 124 generates display information corresponding to the route display image, which is the display image IM for displaying the traveling route of the vehicle, taking into account the road gradient estimation result. The reliability calculation unit 125 calculates the reliability of the estimation result of the road gradient. Then, the display state setting unit 126 sets the display state in the route display image based on the reliability of the estimation result of the road gradient. Specifically, for example, the display state setting unit 126 sets the display state so that the visibility of the route display image increases as the reliability increases. Alternatively, for example, the display state setting unit 126 sets the display area and the non-display area depending on whether the reliability exceeds the reliability threshold.

Furthermore, in the present embodiment, as shown in FIG. 3, the reliability calculation unit 125 integrates a plurality of reliability (that is, first reliability P1, second reliability P2, . . . ) to obtain , the integrated reliability P is calculated. The first reliability P1 is obtained using the distribution distance, which is the first feature amount information, which is one type of the plurality of types of feature amount information, and the prepared first reliability map corresponding to the distribution distance. be done. The first reliability map as the first correspondence information is a map that defines the correspondence between the distribution distance, which is the first feature amount information, and the first reliability P1, and is expressed by P1=f1 (distribution distance). . The second reliability P2 is obtained by using the distance points, which is the second feature amount information, which is another type of the plurality of types of feature amount information, and the prepared second reliability map corresponding to the distance points. obtained by The second reliability map as the second correspondence information is a map that defines the correspondence between the distance score, which is the second feature quantity information, and the second reliability P2, and is indicated by P2=f2 (distance score). . The number of distance points, which is one of the intermediate information, is the number of distance points used for the quadratic curve approximation of the road gradient, that is, the number of feature points for which the distance, that is, the relative three-dimensional position from the own vehicle is calculated by image recognition. is. One of the intermediate information, the distribution distance, is the 95% tile distance of the number of distance points used for gradient estimation. For example, the distribution distance indicates the spread in the depth direction of the distance points used for estimation, and as shown in FIG. 3, the longer the distribution distance, the more the estimation accuracy tends to improve. If the number of distance points is equal to or greater than a certain number, estimation becomes possible. In addition, there is a tendency that the greater the number of distance points, the higher the estimation accuracy. In addition to the above-described distribution distance and distance points, estimation error, vehicle speed, day and night, etc., can be cited as feature amount information that can be used when calculating the reliability of the estimation result of the road gradient. The above maps are created by experiments, computer simulations, etc., and stored in advance in the memory.

The integration of multiple degrees of reliability is performed using, for example, Equation 1 below. Specifically, the integrated reliability _P12 , which integrates the first reliability P1 and the second reliability P2, is given by (a,b)=(1,2), that is, Pa=P1 and Pb=P2 in Equation 1. It is calculated by An integrated reliability _P123 obtained by integrating the first reliability P1, the second reliability P2, and the third reliability P3 is calculated by setting Pa= _P12 and Pb=P3 in Equation (1). An integrated reliability P ₁₂₃₄ obtained by integrating the first reliability P1 to the fourth reliability P4 is calculated by setting Pa=P ₁₂₃ and Pb=P4 in Equation (1). Consolidation of 5 or more confidences is done in a similar way. That is, integration of more than two confidences can be done by repeating integration of two confidences according to Eq.
Formula 1: P _ab (Pa, Pb)
= Pa · Pb / {(Pa · Pb) + (1-Pa · (1-Pb)}

In this way, by mapping the accuracy (that is, reliability) of gradient estimation according to the number of distance points and the distribution distance, and integrating these reliability levels and reflecting them in the gradient display mode in route display, deviation It is possible to display gradients with less sense of incongruity. Specifically, the estimation result of the road gradient is affected by the recognition result of the white line, that is, the road division line LD. That is, for example, if the road marking line LD is blurred or the road marking line LD is cut off by the intersection IS, the accuracy of the road gradient estimation result based on the calculation result of the curvature of the road marking line LD decreases. In addition, the estimation result of the road gradient based on the curvature of the road division line LD that is accurately recognized relatively close to the vehicle does not necessarily match the actual road gradient at a distance. Therefore, unless the road division lines LD in the distance are recognized satisfactorily, a large error occurs in the estimation result of the road gradient in the distance. Then, as shown in FIG. 4A, a large deviation occurs in the route display in the far area. Therefore, by hiding the display or reducing the visibility in the distant region where the calculated value of the integrated reliability P is low, it is possible to display the route with less misalignment and discomfort. In addition, in the display example shown in FIG. 4B, the distant area where the reliability, that is, the integrated reliability P is less than the threshold value Pth is not displayed, but the present invention is not limited to such an aspect. That is, for example, the display image IM may be displayed such that the visibility such as brightness changes stepwise according to the value of the reliability.

(Second embodiment: road curvature)
5, 6A, and 6B show an example of technology for suppressing deviation of road curvature (that is, curve curvature) in route display. That is, FIG. 5 shows an operation example of the reliability calculation unit 125 shown in FIG. FIG. 6A shows, as a comparative example, a display example in which a display image IM for displaying route information with road curvature added is simply displayed based on recognition results without using reliability. FIG. 6B shows a display example according to this embodiment.

In this embodiment, the recognition unit 14 recognizes the shape of the road ahead of the vehicle. That is, the recognition unit 14 calculates, or estimates, the road curvature of the destination road. The recognition result acquisition unit 121 acquires from the recognition unit 14 the estimation result of the road curvature of the destination road. The display information generation unit 124 generates display information corresponding to the route display image, which is the display image IM for displaying the traveling route of the vehicle, taking into account the road curvature estimation result. The reliability calculation unit 125 calculates the reliability of the road curvature estimation result. Then, the display state setting unit 126 sets the display state in the route display image based on the reliability of the road curvature estimation result. Specifically, for example, the display state setting unit 126 sets the display state so that the visibility of the route display image increases as the reliability increases. More specifically, for example, if the reliability is high, the display is dark, and if the reliability is low, the display is light or blank.

Relying only on the recognition result cannot reduce the display deviation. On the other hand, if the navigation device 16 is of a popular type, the map information and the own vehicle position information used by the navigation device 16 do not have enough position accuracy to display the route without deviation. Even if the navigation device 16 uses high-precision digital map information and high-precision positioning information conforming to the ADASIS standard, as described above, positioning signal reception failure or the like may occur while the vehicle is running. ADASIS stands for Advanced Driver Assistance Systems Interface Specification.

In this regard, the inventor has focused on the fact that by focusing on the intermediate information of the image recognition result, it is possible to predict a state in which display deviation or discomfort is likely to occur. Therefore, in the present embodiment, the reliability of lateral deviation in the depth direction is calculated based on the road shape (ie, Bspline model, for example) estimated by KF and the DNN edge points, and display is performed according to the reliability. KF is an abbreviation for Kalman filter. DNN stands for Deep Neural Network. An edge point is a characteristic point on the boundary between the road division line LD and the other portion. A DNN edge point is an edge point recognized using the DNN technique. Specifically, for example, the longer the recognition distance, which is the position in the depth direction for recognition or calculation of the road curvature, the lower the estimation accuracy. For this reason, confidence can be obtained using a map created such that the longer the recognition distance, the lower the estimation accuracy, as shown in FIG.

Furthermore, as shown in FIG. 5, in the present embodiment, the reliability calculation unit 125 calculates an integrated reliability P by integrating a plurality of reliability as in the first embodiment. do. Integration of multiple confidences is performed using, for example, Equation 1 above. Feature amount information that can be used when calculating the reliability of road curvature estimation results includes recognition distance, edge point lateral position error, preceding vehicle distance, lane width, road edge lateral position error, and the like. The edge point lateral position error is the lateral position difference between the road shape estimated by KF and the DNN edge point, and is shown as "lateral position difference" in FIG. 5 for the sake of simplification of illustration. The preceding vehicle distance is the distance between the own vehicle and the preceding vehicle. That is, when the calculated value of the lateral position based on the image recognition result of the preceding vehicle is used for estimating the road curvature, the precision of the estimation result decreases when the preceding vehicle is far from the own vehicle. Therefore, by reflecting the preceding vehicle distance in the reliability, it becomes possible to calculate the reliability with higher accuracy.

As shown in FIG. 6A, if the image recognition result is used as it is to display the display image IM for the route display reflecting the road curvature, the deviation and discomfort will increase. On the other hand, as shown in FIG. 6B, by hiding the display or reducing the visibility in the distant region where the calculated value of the integrated reliability P is low, the shift and the sense of incongruity become more noticeable. It is possible to display a small number of routes. In addition, in the display example shown in FIG. 6B, the distant area where the reliability, that is, the integrated reliability P is less than the threshold value Pth is not displayed, but the present invention is not limited to such an aspect. That is, for example, the display image IM may be displayed such that the visibility such as brightness changes stepwise according to the value of the reliability.

(Third embodiment: POI)
This embodiment shows an example of the display of the display image IM for emphasizing and notifying the driver D of the information on the POI, which is the target to which the driver D should pay attention. POI is an abbreviation for Point Of Interest, and includes, for example, pedestrians, preceding vehicles, traffic signs, road signs, and the like.

FIG. 7 shows a reliability map corresponding to each of the recognition distance, the DNN recognition score, the tracking state, and the cumulative number of times of tracking as feature amount information. As shown in the map of (A) in FIG. 7, the shorter the recognized distance, the higher the reliability. Further, as shown in the map of (B) in FIG. 7, the higher the score value obtained as the identification result by DNN, the higher the reliability. Further, as shown in the map of (C) in FIG. 7, the reliability is lowered in a ghost state, that is, in a temporary non-recognition state due to an obstacle or the like. Also, as shown in the map of (D) in FIG. 7, the reliability is increased by the cumulative number of detections. Confidence in this embodiment can be obtained using at least one of these confidence maps. The processing for calculating the integrated reliability P using a plurality of reliability maps can be performed in the same manner as in each of the above-described embodiments.

Furthermore, when the superimposed display target of the display image IM is a pedestrian and any of the following situations apply, the luminance may be increased more than usual or the blinking control may be performed.
・When you are near a pedestrian crossing ・When you are at a traffic light, a stop line, or an intersection (between the stop line and a traffic signal located farther than a predetermined distance from the stop line) In addition, when the superimposed display target of the display image IM is a traffic sign In particular, at the time of restriction display such as temporary stop, entry prohibition, speed display, etc., which require special attention, the brightness may be increased more than usual or the blinking control may be performed.

(Fourth Embodiment: Vehicle Position in Lane)
This embodiment shows an example of calculating the reliability of the estimation result of the lateral position information corresponding to the in-lane vehicle position. The in-lane own vehicle position is the position of the own vehicle in the road width direction within the traveling lane LR. The route display is displayed with reference to the center of the running lane LR, not the center of the vehicle. Therefore, by performing display according to the recognition reliability of the in-lane vehicle position, it is possible to display a route with little sense of discomfort. Specifically, for example, by setting a threshold for reliability, display control to cancel display can be performed.

FIG. 8 shows a reliability map corresponding to each of the total extended distance of white lines and the state of recognition of white lines as feature amount information. "White line" is an abbreviation for road division line LD. The lateral position of the vehicle within the lane depends on the state of recognition of the left and right white lines. Therefore, as shown in the map of (A) in FIG. 8, the longer the total extension distance of the white lines, the higher the reliability. Further, as shown in the map of FIG. 8B, the reliability is set as follows according to the recognition state of the left and right white lines of the own vehicle.
・Recognition on both left and right … High reliability (eg 90%)
・Recognize only left or right … Lane width cannot be detected and relies on history, so reliability is medium (eg 25%)
・ Both left and right cannot be recognized … Position calculation is impossible, so the reliability is the lowest (eg 1%)

(Fifth embodiment: lane number)
This embodiment shows an example of calculating the reliability of the estimation result of the lateral position information corresponding to the lane number. The lane number is an ID number indicating the position of the lane LR in which the vehicle is traveling in the road width direction on the road RR on which the vehicle is traveling. For example, the lane numbers are numbered in ascending order from right to left starting with a value of 1. In order to guide the route smoothly and without difficulty, the lane number prompts the user to move to the appropriate lane position in advance, and displays guidance information on features such as shops that serve as landmarks at an appropriate lateral position from the vehicle. It is necessary for

Lane number identification that can be realized only by camera recognition is based on (1) the line type pattern of the white line recognized by the camera (for example, the left is a solid line and the right is a broken line) and (2) the road edge recognized by the camera (for example, the lower end of the shoulder). This can be done by considering the width of the lane and the width of the shoulder based on the lateral offset distance to the own vehicle. The actual situation is that (1) line type results output in white line recognition are discrete information and unstable, while (2) road edge recognition may or may not be recognized because it may be blocked by surrounding vehicles. Therefore, as shown in the map of (A) in FIG. 9, the standard deviation is taken with respect to the values for which the lane can be identified within a certain period of time, and the temporal fluctuation of the lane number is taken as the reliability. If there is a time when the lane cannot be identified within a certain period of time, a standard deviation of a predetermined value is added according to the number of times, and the fluctuation is large, that is, the lane number is changed to an unreliable state. In addition, road edge recognition is often hindered by the number of surrounding vehicles within a predetermined distance (for example, 30 m) in front of the left and right adjacent lanes of the own vehicle. Similarly, the number of adjacent vehicles on the left and right within a predetermined distance is also included in the reliability map. Confidence in this embodiment can also be obtained using at least one of these confidence maps. The processing for calculating the integrated reliability P using a plurality of reliability maps can be performed in the same manner as in each of the above-described embodiments.

(Sixth embodiment: intersection position)
This embodiment shows an example of calculating the reliability of the estimation result of the intersection position. The intersection position is the relative position of the center of the intersection IS with respect to the host vehicle. The estimated intersection position is typically a forward/backward position, that is, a planned traveling distance from the current position of the vehicle in the current traveling direction of the vehicle.

With the positional accuracy of conventional in-vehicle navigation, the front and rear positions of the intersection IS, which is the target of route or traveling direction guidance, may deviate. The cause of this is, for example, a decrease in the accuracy of vehicle position estimation when positioning signals such as GPS cannot be received or are difficult to receive in high-rise buildings, tunnels, multi-story parking lots, and the like. Alternatively, the cause of this is, for example, an error in self-vehicle position estimation due to a change in the actual road shape from the map when performing map matching with the map. Thus, if an error occurs in the estimation or calculation of the own vehicle position, an error also occurs in the relative positional relationship from the own vehicle position to the intersection IS for which the route or direction of travel is to be guided. As a result, display elements for guiding the route or traveling direction, which should be displayed corresponding to the intersection IS, are displayed at positions shifted from the intersection IS. Therefore, by estimating or calculating the position of the intersection using the camera recognition result, it is possible to display the route without deviation or discomfort.

Analyzing an intersection IS with traffic lights, there are a traffic light TL and a stop line LL that can be recognized by a camera. There is a feature. Therefore, it is assumed that the middle point of the traffic light TL on the far side of the intersection IS and the stop line LL on the front side of the intersection IS is assumed to be the center of the intersection IS, and the logic is calculated as the front and rear positions. Also, the lateral position of the center of the intersection IS is calculated based on the lane ID, which is the lane number in the lane LR in which the vehicle is traveling. If the traffic light TL is also placed in front of the intersection IS, the estimated position based on the image recognition result may be reflected in the estimation of the intersection position.

Here, after the intersection IS has been detected once, if it cannot be detected, it is a ghost, and if it cannot be detected more than a predetermined number of times (for example, four times) in succession, the intersection position is reset. When the intersection IS is too close to be recognized, the vehicle speed and yaw rate are complementarily displayed as the display of the own vehicle.

As described above, the front and rear positions of the intersection IS have characteristics that depend on the state of recognition of the traffic light TL and the stop line LL. Therefore, the reliability is estimated by a reliability map using the following feature amount information. FIG. 10 shows an example of these confidence maps. Note that the reliability of the vehicle position by the navigation device 16 may be reflected in addition to the reliability of camera recognition. Confidence in this embodiment can also be obtained using at least one of these confidence maps. The processing for calculating the integrated reliability P using a plurality of reliability maps can be performed in the same manner as in each of the above-described embodiments.
・Reliability based on the DNN recognition score value of the stop line LL and the tracking state ・Reliability based on the DNN recognition score value of the traffic light TL and the tracking state ・Reliability based on the recognition distance: The closer, the more reliable ・Cumulative detection count: The higher, the more reliable

(Seventh embodiment: Reliability setting by scene)
This embodiment shows an example in which the driving scene, that is, the driving condition of the own vehicle is reflected in the reliability calculation. When the route display by the head-up display device 11 is controlled by camera recognition, there is a possibility that the display may be shifted in a scene where the camera recognition is not good. Therefore, in such a scene, the reliability is lowered in advance before display deviation occurs so as to prevent erroneous display. That is, in the present embodiment, the reliability calculation unit 125 corrects the reliability so as to decrease when the image recognition result includes the driving situation of the own vehicle that causes the reliability to be low.

Typical scenes in which errors are likely to occur in recognizing road shapes (that is, road gradients and road curvatures) by a camera are, for example, approaching an intersection IS, backlight, nighttime, and the like. As for the curvature of the road, an error is likely to occur in recognizing the shape of the road even in a scene approaching a branch. Specifically, at the intersection IS, there are many road markings such as arrows, and white lines are often blurred. In addition, since there are no white lines inside the intersection IS, camera errors and non-recognition tend to occur. As for branching, if the white line on the branching side is recognized by the camera, the display will be erroneous. As for the backlight, the repair marks of coal tar and the shiny road surface are likely to cause camera errors and unrecognized images. At night, the amount of exposure of the camera increases, and it becomes difficult to see the black patterns on the road surface due to blurring and noise, and it is easy to make errors in estimating the road gradient and road curvature.

Therefore, as shown in the map of (A) in FIG. 11, the closer the distance to the intersection IS based on the image recognition result of the intersection position, the lower the reliability. Further, as shown in the map of FIG. 11B, the closer the distance to the intersection IS or the branch on the map information, the lower the reliability. Also, as shown in the map of (C) in FIG. 11, the reliability decreases as the amount of overexposed areas in the upper half of the camera image increases. Also, as shown in the map of (D) in FIG. 11, the reliability decreases as the exposure amount of the camera increases. Confidence in this embodiment can also be obtained using at least one of these confidence maps. The processing for calculating the integrated reliability P using a plurality of reliability maps can be performed in the same manner as in each of the above-described embodiments.

(Eighth embodiment: Reliability setting by recognition instability determination)
This embodiment shows an example in which display output is suppressed when the recognized road shape is unstable, such as a snowy road, an unmaintained road, or a road surface reflection scene in rainy weather. That is, in the present embodiment, the amount of variation in the recognition results, such as the recognized horizontal position of the white line and the estimated curve radius, is used as a feature quantity to stochastically calculate the "unstable road shape", which means whether the road is unstable. and suppress display based on this. 12 to 14 show an overview of reliability calculation and display state control in this embodiment. In FIG. 12, "likelihood" indicates the degree of instability of road shape recognition, and is also called "instability". This "instability" is the opposite concept of the reliability in each of the above embodiments, and corresponds to a value obtained by subtracting the reliability from 100%. The vertical axis P' of each map in FIG. 14 indicates the degree of instability.

Specifically, as shown in FIG. 12, the reliability calculation unit 125 calculates the amount of change in the instantaneous likelihood, which is the instantaneous value of the likelihood, for each feature quantity, and calculates a plurality of instantaneous likelihood values. The integrated likelihood is calculated by integrating the degrees. The method of integrating multiple likelihoods is the same as the method of integrating reliability in each of the embodiments described above. In addition, the reliability calculation unit 125 improves the calculation accuracy of the integrated likelihood by linear prediction using a so-called “weighted regression” method or the like. That is, the display information generation unit 124 generates display information corresponding to the route display image based on the road shape estimation result by weighted regression using the reliability of the road shape estimation result.

The display state setting unit 126 sets the display state of the display image IM according to the integrated likelihood calculated by the reliability calculation unit 125, that is, the change state of the target image recognition result. Specifically, as shown in FIG. 13, the display state setting unit 126 suppresses display (for example, non-display) when the degree of instability, that is, the integrated likelihood, is equal to or higher than the upper threshold THH (for example, 90%). after that, when the degree of instability becomes less than the lower threshold value THL (for example, 50%), the display suppression is turned off and displayed again. After time t1 when the integrated likelihood once becomes equal to or greater than the upper threshold THH and display suppression is turned on, even if the integrated likelihood becomes less than the upper threshold THH, display suppression is turned on until time t2 when it becomes less than the lower threshold THL. state is maintained. Thus, the display state setting unit 126 hysteresis-controls the display state using the upper threshold THH and the lower threshold THL.

The method of determining the degree of instability is as follows. For white line recognition, the white line detection flag fluctuation amount, the white line estimated R fluctuation amount, and the white line lateral position fluctuation amount are used as feature amount information. As for the white line detection flag fluctuation amount, as shown in the map of (A) in FIG. 14, recognition becomes unstable when detection/non-detection is repeated. As shown in the map of FIG. 14B, the estimated white line R variation amount is unstable when the amount of variation in the direction of the recognized white line is large. As shown in the map of (C) in FIG. 14, the white line lateral position variation amount is unstable when the recognized white line position variation amount is large. As for the road gradient, the amount of change in the estimation success flag of the road gradient and the amount of change in the estimated value of the road gradient are used as feature quantity information. Regarding the road gradient estimation success flag fluctuation amount, as shown in the map of (D) in FIG. 14, the estimation becomes unstable when the estimation success/failure is repeated. As shown in the maps of (E) and (F) in FIG. 14, the amount of variation in road gradient is assumed to be unstable when the estimated variation in gradient is large, and the estimation is assumed to be unstable. Divide the map with Alternatively, for example, if it is possible to determine whether recognition is possible by camera output without using the recognition result information (for example, the entire screen continues to be pure white, the temperature of the camera is abnormally high, the rain is too strong and visibility is poor , the vehicle speed is too fast and the operation is not guaranteed, etc.), this may be used to acquire the instability.

(Modification)
The present invention is not limited to the above embodiments. Therefore, the above embodiment can be modified as appropriate. A representative modified example will be described below. In the following description of the modified example, differences from the above embodiment will be mainly described. Moreover, in the above-described embodiment and modifications, the same or equivalent portions are denoted by the same reference numerals. Therefore, in the description of the modification below, the description in the above embodiment can be used as appropriate for components having the same reference numerals as those in the above embodiment, unless there is a technical contradiction or special additional description.

The present invention is by no means limited to the specific device configurations shown in the above embodiments. That is, for example, the vehicle 1 on which the in-vehicle system 10 is mounted is not limited to an ordinary automobile. Specifically, such a vehicle 1 may be a large automobile such as a freight truck. The number of wheels is not particularly limited, and it may be a three-wheeled vehicle, or a six-wheeled or eight-wheeled vehicle such as a freight truck. The type of vehicle 1 may be a conventional vehicle equipped with only an internal combustion engine, an electric vehicle or fuel cell vehicle without an internal combustion engine, or a so-called hybrid vehicle. The shape and structure of the vehicle body 2 of the vehicle 1 are also not limited to a box shape, that is, a substantially rectangular shape in plan view. There are no particular restrictions on the use of the vehicle 1, the position of the steering wheel 8, the number of passengers, and the like. In other words, the driver D may be an occupant of the own vehicle who is in charge of or executes the dynamic driving task. In other words, the seating position of the driver D is not particularly limited as long as the driver D can execute the driving operation. Also, any operating device such as a joystick may be used in place of or in conjunction with the steering wheel 8 .

As a communication standard that configures the in-vehicle system 10, standards other than CAN (internationally registered trademark), such as FlexRay (internationally registered trademark), may be employed. In addition, the communication standard configuring the in-vehicle system 10 is not limited to one type. For example, the in-vehicle system 10 may have a subnetwork line conforming to a communication standard such as LIN. LIN is an abbreviation for Local Interconnect Network.

The display control device 12 is not limited to a configuration as a so-called in-vehicle computer including a processor and memory. Specifically, for example, all or part of the display control device 12 may be configured with a digital circuit, such as an ASIC or FPGA, configured to enable the above operations. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

A program according to the present invention that enables execution of various operations, procedures, or processes described in the above embodiments is stored in a computer-readable non-transitional tangible storage medium as instructions executed by a computer. can be That is, each of the functional configurations and methods described above can be expressed as a computer program including procedures for realizing it, or as a non-transitional physical storage medium storing the program. Such programs can be downloaded or upgraded via V2X communication. V2X is an abbreviation for Vehicle to X. Alternatively, such a program can be downloaded or upgraded via terminal equipment provided at the vehicle 1 manufacturing plant, maintenance plant, dealer, or the like. Such programs may be stored in a memory card, an optical disk, a magnetic disk, or the like.

Programs according to the present invention that enable execution of various operations, procedures, or processes described in the above embodiments can be downloaded or upgraded via V2X communication. V2X is an abbreviation for Vehicle to X. Alternatively, such a program can be downloaded or upgraded via terminal equipment provided at the vehicle's manufacturing plant, maintenance plant, dealer, or the like. Such programs may be stored in a memory card, an optical disk, a magnetic disk, or the like.

The head-up display device 11 is not limited to the type that projects onto the front windshield 4 . That is, for example, the head-up display device 11 may be of a so-called combiner type. The combiner-type head-up display device 11 may be of a so-called retrofit type.

The imaging device 13 as a camera is not limited to a type fixedly attached to the vehicle body 2 or the vehicle interior 3 . That is, for example, a camera mounted on a so-called retrofitted drive recorder may be used as the imaging device 13 . Alternatively, for example, as the image capturing device 13, a camera (for example, a so-called out-camera equipped in a smart phone) installed in a mobile terminal brought into the vehicle interior 3 by the passenger may be used. In this case, instead of a driver status monitor (not shown) for detecting the viewpoint of driver D or a camera for photographing the interior of the vehicle, another camera (for example, an in-camera equipped in a so-called smartphone) equipped in the mobile terminal is used. can be used. Alternatively, a LiDAR may be used as the imaging device 13 instead of the camera.

The recognition unit 14 and the recognition result acquisition unit 121 can be integrated. That is, the recognition result acquisition unit 121 may perform image recognition of the target included in the foreground based on the foreground image captured by the imaging device 13 . In other words, the present invention can be applied favorably to a vehicle 1 that does not have an automatic driving system or an ADAS system that uses the recognition result of the foreground imaged by the imaging device 13 for driving control or the like.

The navigation device 16 is not limited to a type fixedly attached to the dashboard 5 . That is, for example, a portable terminal brought into the vehicle interior 3 by the passenger may be used as the navigation device 16 . In this case, as described above, a camera mounted on the mobile terminal may be used as a sensor for detecting the image capturing device 13 and/or the viewpoint of the driver D.

The present invention is not limited to the specific operational modes or operational examples shown in the above embodiments. For example, confidence or combined confidence may be low-pass filtered. Further, the shape of the display image IM itself may be corrected in place of or along with the adjustment of the display form. Specifically, for example, the currently calculated value of the road gradient is calculated as road gradient = estimated value by image recognition x reliability, and road gradient = α × current calculated value + (1-α) × past value like Low-pass filtering may be performed. The same applies to road curvature and the like. Further, even when the route display by the display image IM on the head-up display device 11 is suppressed, the route display on the navigation device 16 does not have to be suppressed.

The present invention is not limited to the specific processing modes shown in the above embodiments. In other words, for example, the terms "equal to or greater than the threshold" and "exceeding the threshold" can be appropriately interchanged within a technically consistent range. Similarly, "below the threshold value" and "less than the threshold value" can be interchanged as appropriate within a technically consistent range.

So-called "weighted regression" is applicable to road shape estimation in each of the above embodiments. By estimating the road shape using more reliable recognition points, the accuracy of road shape estimation is improved, and superimposed display with less display deviation and discomfort becomes possible. Details of the contents of the "weighted regression" itself are already known or well known at the time of filing of the present application, and thus description thereof will be omitted in this specification.

Mutually similar expressions such as "obtain", "estimate", "detect", "detect", "calculate", "extract", and "generate" can be replaced as appropriate within the scope of technical compatibility. is.

Needless to say, the elements constituting the above-described embodiments are not necessarily essential, unless explicitly stated as essential or clearly considered essential in principle. In addition, when numerical values such as the number, amount, range, etc. of a constituent element are mentioned, unless it is explicitly stated that it is particularly essential, or when it is clearly limited to a specific numerical value in principle, the specific numerical value The present invention is not limited to Similarly, when the shape, direction, positional relationship, etc. of the constituent elements, etc. are mentioned, unless it is explicitly stated that it is particularly essential, or when it is limited to a specific shape, direction, positional relationship, etc. in principle , the shape, direction, positional relationship, etc., of which the present invention is not limited.

Modifications are also not limited to the above examples. Also, multiple variants can be combined with each other. In addition, the multiple embodiments described above may be overlapped with each other. Also, all or part of the above embodiments and all or part of the modifications may be combined with each other.

1 own vehicle 11 head-up display device (display device)
REFERENCE SIGNS LIST 12 display control device 13 imaging device 121 recognition result acquisition unit 122 host vehicle position acquisition unit 123 route information acquisition unit 124 display information generation unit 125 center position estimation unit TD traveling direction display

Claims (9)

A display control device (12) configured to control the display operation of a display device (11) that superimposes and displays a display image (IM) on the foreground of the vehicle (1) by being mounted on the vehicle (1) There is
a recognition result acquisition unit (121) for acquiring recognition result information corresponding to a target image recognition result based on the foreground captured image by an imaging device (13); a reliability calculation unit (125) that calculates based on feature amount information, which is intermediate information in the process of acquiring information or the recognition result information;
a display state setting unit (126) for setting a display state in the display image based on the reliability calculated by the reliability calculation unit;
A display controller with The reliability calculation unit
First correspondence information that defines a correspondence relationship between the first feature amount information and the first reliability, prepared corresponding to the first feature amount information that is one type of the plurality of types of the feature amount information. to obtain the first reliability using
A second correspondence that defines a correspondence relationship between the second feature amount information and the second reliability, prepared corresponding to the second feature amount information that is another type of the plurality of types of the feature amount information. using the relationship information to obtain the second reliability;
calculating the reliability by integrating the obtained first reliability and the second reliability;
The display control device according to claim 1. The display state setting unit sets the display state such that the visibility of the display image increases as the reliability increases.
The display control device according to claim 1 or 2. When the image recognition result includes a driving situation of the own vehicle that causes the reliability to be low, the reliability calculation unit corrects the reliability in the direction of decreasing the reliability.
The display control device according to any one of claims 1 to 3. The display state setting unit sets the display state according to a change state of the image recognition result of the target.
The display control device according to any one of claims 1 to 4. further comprising a display information generation unit (124) that generates display information for displaying the display image on the display device based on the recognition result information,
The reliability calculation unit calculates the reliability of an estimation result of a road shape, which is a road gradient or a road curvature, on a road ahead of the vehicle,
The display information generation unit generates the display information corresponding to the route display image, which is the display image for displaying the traveling route of the vehicle, taking into consideration the estimation result of the road shape,
The display state setting unit sets the display state in the route display image based on the reliability of the estimation result of the road shape.
The display control device according to any one of claims 1 to 5. The display information generation unit generates the display information corresponding to the route display image based on the estimation result of the road geometry by weighted regression using the reliability of the estimation result of the road geometry.
The display control device according to claim 6. The reliability calculation unit calculates the reliability of an estimation result of lateral position information corresponding to a position in the road width direction of the vehicle on which the vehicle is traveling,
The display state setting unit sets the display state of the display image for displaying guidance information about features around the own vehicle based on the reliability of the estimation result of the lateral position information.
The display control device according to any one of claims 1 to 7. Executed by a display control device (12) configured to control the display operation of a display device (11) mounted on a vehicle (1) to superimpose a display image (IM) on the foreground of the vehicle (1) a display control program, comprising:
The processing executed by the display control device includes:
a recognition result acquisition process for acquiring recognition result information corresponding to a target image recognition result based on the foreground captured image by the imaging device (13);
Reliability calculation processing for calculating the reliability of the recognition result information based on feature amount information, which is intermediate information in the process of acquiring the recognition result information or the recognition result information;
display state setting processing for setting a display state in the display image based on the reliability calculated by the reliability calculation processing;
including,
Display control program.

Images