JP7230960B2 - display controller - Google Patents

Description

The disclosure according to this specification relates to a display control device for controlling a display superimposed on the foreground.

Conventionally, for example, Patent Literature 1 discloses a display device for a vehicle that superimposes a guidance display for guiding a route to a destination on the driver's forward field of view through the windshield. This vehicular display device superimposes an arrow indicating the direction of the right or left turn on the road as guidance display based on the route information when the vehicle approaches an intersection that guides the vehicle to turn left or right by a predetermined distance.

WO2015/118859

As in Patent Document 1, when route guidance to a destination is performed, a scene may occur in which a plurality of guidance points that require guidance display are continuously present. In the case where a plurality of guidance points are consecutive in this way, and the vehicle is to turn right or left at the first point, the second point may be turned at an inappropriate timing such as when the vehicle is turning. Point guidance display could be initiated. In this case, the guide display for the second point may be superimposed on an erroneous object or the like in the forward field of view of the driver, which may induce erroneous recognition by the driver.

An object of the present disclosure is to provide a display control device capable of reducing the possibility of erroneous recognition of superimposed display for route guidance.

In order to achieve the above object, one aspect disclosed is a display control device for controlling display by a head-up display (20), which provides route guidance from a navigation device (50) for route guidance to a destination. A route information acquisition unit (71) for acquiring information, and a display control unit (78) for superimposing and displaying route guidance content from the route guidance information on the road surface. When it is determined that the route guidance information includes continuous information indicating that the points are consecutive on the route at intervals , after displaying the first guidance content ( CNT1 ) for performing the first route guidance a display control device for displaying a second guidance content ( CNT2 ) for performing a second route guidance at a timing when superimposed display of the second guidance content becomes possible after the first guidance content is hidden; be done .

Another disclosed aspect is a display control device for controlling display on a head-up display (20), which is route information for acquiring route guidance information from a navigation device (50) that provides route guidance to a destination. An acquisition unit (71) and a display control unit (78) for superimposing and displaying route guidance content from route guidance information on a road surface. If it is determined that the route guidance information includes continuation information indicating that the route guidance information is continuous, the first guidance content (CNT1) for performing the first route guidance is displayed, and the second route guidance target is displayed. Display control for starting the display of the second guidance content (CNT2) that performs the second route guidance based on the estimation that the traveling direction of the vehicle is directed along the road toward the second guidance point (P2). device.

According to these aspects, when the right/left turn point and the sequential guidance point are consecutive, the display of the guidance content for guiding the sequential guidance point is displayed after the right turn or left turn at the right/left turn point. is started on the condition that the vehicle is facing in a direction that can be displayed. According to the above, the display start timing of the guidance content that sequentially guides the guidance points can be appropriate. Therefore, it is possible to reduce the possibility of erroneous recognition due to the superimposed display for route guidance.

It should be noted that the reference numbers in parentheses above merely indicate an example of correspondence with specific configurations in the embodiments described later, and do not limit the technical scope in any way.

1 illustrates an overview of an in-vehicle network including an HCU according to an embodiment of the present disclosure; FIG. It is a figure which shows an example of the HUD apparatus mounted in a vehicle. It is a figure which shows an example of a schematic structure of HCU. FIG. 10 is a diagram for explaining an example of a scene in which guidance is provided continuously for right and left turn points and continuous points; FIG. 10 is a diagram showing one display example of right/left turn content that guides right/left turn points. FIG. 10 is a diagram showing one display example of guidance content that guides consecutive points; 4 is a flowchart showing details of guidance display processing performed by the HCU; FIG. 12 is a diagram showing a display as a comparative example when the start of display of guidance content is not appropriate; FIG. 10 is a diagram for explaining the effect of advancing the start of display of guidance content when there is no obstruction in the roadside area of a connecting road;

The functions of the display control device according to an embodiment of the present disclosure are realized by an HCU (Human Machine Interface Control Unit) 100 shown in FIGS. 1 and 2. FIG. The HCU 100 constitutes an HMI (Human Machine Interface) system 10 used in the vehicle A together with a head-up display (hereinafter referred to as "HUD") device 20, an operation device 26, and the like. The HMI system 10 has an input interface function for accepting operations by a passenger (eg, driver) of the vehicle A and an output interface function for presenting information to the driver.

The HMI system 10 is communicably connected to the communication bus 99 of the in-vehicle network 1 mounted on the vehicle A. HMI system 10 is one of a plurality of nodes provided in in-vehicle network 1 . A communication bus 99 of the in-vehicle network 1 is connected with, for example, a perimeter monitoring sensor 30, a locator 40, a navigation device 50, a vehicle control ECU (Electronic Control Unit) 54, a driving support ECU 57, etc. as respective nodes. These nodes connected to the communication bus 99 can communicate with each other.

The surroundings monitoring sensor 30 is an autonomous sensor that monitors the surrounding environment of the vehicle A. As shown in FIG. The surroundings monitoring sensor 30 detects moving objects such as pedestrians, cyclists, animals other than humans, and other vehicles from the detection range around the vehicle, as well as road markings such as fallen objects on the road, guardrails, curbs, and lane markings. and stationary objects such as roadside structures. The surroundings monitoring sensor 30 provides the detection information of objects detected around the vehicle A (especially in the front range) to the driving support ECU 57 and the HCU 100 through the communication bus 99 .

The surroundings monitoring sensor 30 has a front camera 31, a millimeter wave radar 32, a lidar 33, etc. as a detection configuration for object detection. The front camera 31 outputs, as detection information, at least one of imaging data obtained by imaging a range in front of the vehicle A and an analysis result of the imaging data. The millimeter wave radar 32 emits millimeter waves or quasi-millimeter waves toward a forward range, and generates detection information to be output to the outside by processing for receiving reflected waves reflected by moving and stationary objects. The lidar 33 generates detection information to be output to the outside by performing a process of irradiating a laser beam toward a forward range and receiving reflected light reflected by a moving object, a stationary object, or the like.

The locator 40 generates highly accurate position information and the like of the vehicle A by composite positioning that combines a plurality of acquired information. The locator 40 can specify, for example, the lane in which the vehicle A travels among multiple lanes. The locator 40 includes a GNSS (Global Navigation Satellite System) receiver 41, an inertial sensor 42, a high-precision map database (hereinafter referred to as "DB") 43, a locator ECU 44, and the like.

The GNSS receiver 41 receives positioning signals transmitted from a plurality of artificial satellites (positioning satellites). The GNSS receiver 41 can receive positioning signals from each positioning satellite of at least one satellite positioning system among satellite positioning systems such as GPS, GLONASS, Galileo, IRNSS, QZSS, and Beidou.

The inertial sensor 42 includes, for example, a gyro sensor and an acceleration sensor. The high-precision map DB 43 is mainly composed of non-volatile memory, and stores map data with higher precision than that used in the navigation device 50 (hereinafter referred to as "high-precision map data"). The high-precision map data includes information about lane markings and road markings, three-dimensional shape information of structures in the roadside area, information on the number of road lanes, three-dimensional shape information, and the like.

The locator ECU 44 is mainly composed of a microcomputer having a processor, a RAM, a storage unit, an input/output interface, and a bus connecting them. The locator ECU 44 combines the positioning signal received by the GNSS receiver 41, the measurement result of the inertial sensor 42, the vehicle speed information output to the communication bus 99, etc., and sequentially locates the vehicle position, traveling direction, and the like of the vehicle A. The locator ECU 44 provides the navigation device 50, the HCU 100, and the like via the communication bus 99 with position information and direction information of the vehicle A (own vehicle) based on the positioning result. In addition, the locator ECU 44 determines whether or not the required high-definition map data exists in the high-definition map DB 43 in response to a request from the HCU 100 or the like. If the requested high-definition map data exists in the high-definition map DB 43, the locator ECU 44 reads out the corresponding high-definition map data from the high-definition map DB 43 and provides it to the HCU 100 that made the request.

The navigation device 50 is an in-vehicle device that cooperates with the HMI system 10 and provides route guidance to a destination set by the passenger. The navigation device 50 includes a map database for navigation (hereinafter referred to as "navigation map DB") 51, a navigation ECU 52, and the like. The navigation map DB 51 is mainly composed of a non-volatile memory, and comprehensively stores map data in a wider range than the high-precision map DB 43 . The navigation map data stored in the navigation map DB 51 describes link data, node data, shape data, and the like for roads.

The navigation ECU 52 is mainly composed of a microcomputer having a processor, a RAM, a memory, an input/output interface, and a bus connecting them. The navigation ECU 52 acquires position information and direction information of the vehicle A (own vehicle) from the locator ECU 44 via the communication bus 99 .

The navigation ECU 52 acquires operation information input to the operation device 26 via the communication bus 99 and the HCU 100, and sets the destination based on the driver's operation. The navigation ECU 52 searches for multiple routes to the destination so as to satisfy conditions such as time priority and distance priority. When one of the searched multiple routes is selected, the navigation ECU 52 provides route information based on the set route along with related navigation map data to the HCU 100 via the communication bus 99 .

The navigation ECU 52 outputs a guidance implementation request to the HMI system 10 when approaching a branch point such as an intersection, the destination of the set route, or a warning point such as a temporary stop. In the HMI system 10, the HUD device 20 or the like provides route guidance to the driver based on the guidance execution request. In the following description, the points on the set route where such guidance to the driver is implemented are referred to as "guidance points".

The vehicle control ECU 54 is an electronic control unit that performs acceleration/deceleration control and steering control of the vehicle A. As shown in FIG. The vehicle control ECU 54 is connected to a group of sensors for detecting driving operations of the driver, such as a steering angle sensor 55, an accelerator position sensor, a brake stroke sensor, and the like. The vehicle control ECU 54 is connected to a group of actuators related to running of the vehicle A, such as an EPS (Electric Power Steering) motor, an electronically controlled throttle and injector, a brake actuator, and the like.

The vehicle control ECU 54 outputs measurement information (for example, steering angle information) from the sensor group to the communication bus 99 and acquires driving operation information, which will be described later, from the driving support ECU 57 via the communication bus 99 . The vehicle control ECU 54 controls each actuation of the actuator group, and thus the behavior of the vehicle A, based on the information measured by the sensor group or the driving operation information acquired from the driving support ECU 57 .

The driving assistance ECU 57 has at least one of a driving assistance function that assists the driving operation by the driver and an automatic driving function that can substitute the driving operation for the driver. The driving support function or automatic driving function includes an ACC (Adaptive Cruise Control) function that controls the traveling speed or the inter-vehicle distance, an LTC (Lane Trace Control) function that controls the steering angle according to the lane, and the like. The driving support function may further include an AEB (Autonomous Emergency Braking) function for forcibly decelerating the vehicle A, or the like.

The driving support ECU 57 recognizes the driving environment around the vehicle A based on the detection information acquired from the surroundings monitoring sensor 30 . The driving assistance ECU 57 generates driving operation information for driving assistance or automatic driving based on the recognition result of the driving environment. The driving support ECU 57 provides the generated driving operation information to the vehicle control ECU 54 to cause the vehicle control ECU 54 to perform the above-described acceleration/deceleration control and steering control.

The driving support ECU 57 can provide the HCU 100 with the analysis result of the detection information performed for recognizing the driving environment as the analyzed detection information. As an example, the driving support ECU 57 can provide the HCU 100 with information extracted from the imaged data of the front camera 31, specifically relative positional information of road markings (white lines) and road edges on which the vehicle is traveling. In addition, the driving support ECU 57 can also provide the HCU 100 with type information, three-dimensional shape information, and the like of structures and the like provided in roadside areas.

Next, details of the HUD device 20 and the HCU 100 will be described.

The HUD device 20 is mounted on the vehicle A as one of a plurality of in-vehicle display devices together with a multi-information display, a center information display, and the like. The HUD device 20 presents various information related to the vehicle A to the driver by augmented reality (hereinafter “AR”) display using the virtual image Vi. The AR-displayed virtual image Vi is visible to the driver and moves following the superimposition target as if it is relatively fixed to the superimposition target in the foreground.

The HUD device 20 is electrically connected to the HCU 100 and sequentially acquires video data generated by the HCU 100 . The HUD device 20 is housed in a housing space within the instrument panel 9 below the windshield WS. The HUD device 20 projects the light imaged as the virtual image Vi toward the projection range PA of the windshield WS. The light projected onto the windshield WS is reflected to the driver's side in the projection area PA and is perceived by the driver. The driver visually recognizes a display in which the virtual image Vi is superimposed on the superimposition target in the foreground that can be seen through the projection range PA.

The HUD device 20 includes a projector 21, an enlarging optical system 22, and the like. The projector 21 has an LCD (Liquid Crystal Display) panel and a backlight. The projector 21 is fixed to the housing of the HUD device 20 with the display surface of the LCD panel facing the magnifying optical system 22 . The projector 21 displays each frame image of the video data on the display surface of the LCD panel, illuminates the display surface with a backlight, and emits light formed as a virtual image Vi toward the magnifying optical system 22. do. The magnifying optical system 22 includes at least one concave mirror formed by vapor-depositing a metal such as aluminum on the surface of a substrate made of synthetic resin, glass, or the like. The enlarging optical system 22 spreads the light emitted from the projector 21 by reflection and projects it onto the upper projection range PA.

The HCU 100 is an electronic control unit that comprehensively controls display by a plurality of display devices including the HUD device 20 in the HMI system 10 . The HCU 100 is mainly composed of a computer including a processing unit 11, a RAM 12, a storage unit 13, an input/output interface 14, and a bus connecting them. The processing unit 11 is hardware for arithmetic processing coupled with the RAM 12 . The processing unit 11 includes at least one arithmetic core such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processing unit 11 may be configured to further include an IP core having an FPGA (Field-Programmable Gate Array) and other dedicated functions. The storage unit 13 is configured to include a nonvolatile storage medium. The storage unit 13 stores various programs (display control program, etc.) executed by the processing unit 11 .

The HCU 100 shown in FIGS. 1 to 3 executes a program stored in the storage unit 13 by the processing unit 11, and includes a plurality of functional units. Specifically, the HCU 100 includes a route information acquisition unit 71, a position information acquisition unit 72, a vehicle information acquisition unit 73, an external world information acquisition unit 74, a continuous guidance determination unit 75, a roadside object determination unit 76, and a vehicle attitude determination unit 77. and functional units such as the display generation unit 78 are constructed. The route information acquisition unit 71 , the position information acquisition unit 72 , the own vehicle information acquisition unit 73 and the external world information acquisition unit 74 are functional units that acquire information from the communication bus 99 .

When a destination is set in the navigation device 50 , the route information acquisition unit 71 acquires, from the navigation ECU 52 , route information regarding route guidance to the destination and navigation map data used for route guidance. In addition, the route information acquisition unit 71 acquires a guidance implementation request output by the navigation ECU 52 together with route information, navigation map data, etc., based on the approach to the guidance point.

The route information acquisition unit 71 can perform processing for acquiring high-definition map data from the locator 40 together with the navigation map data or in place of the navigation map data. Based on the route information acquired from the navigation ECU 52, the route information acquisition unit 71 requests the locator ECU 44 to provide related high-definition map data. When the requested high-precision map data is stored in the high-precision map DB 43 , the route information acquisition unit 71 acquires the high-precision map data returned from the locator ECU 44 .

The position information acquisition unit 72 acquires position information and direction information of the vehicle A from the locator ECU 44 . The own vehicle information acquisition unit 73 acquires the latest steering angle information measured by the steering angle sensor 55 as one of the own vehicle information indicating the running state of the vehicle A from the vehicle control ECU 54 .

The external world information acquisition unit 74 acquires detection information about the front range of the vehicle A from at least one of the surroundings monitoring sensor 30 and the driving support ECU 57 . The detection information may be imaging data of the front range captured by the front camera 31 or the like, or may be an analysis result obtained by the surrounding monitoring sensor 30 or the driving support ECU 57 recognizing the driving environment.

When the detection information is an analysis result, the analysis result includes information indicating the relative position of the lane marking or road edge in the forward range as described above. In addition, detection information as an analysis result includes type information and 3 It contains at least dimensional shape information. The shield BO is, for example, a building such as a building, a fence surrounding the site of the roadside area, or the like.

The continuous guidance determination unit 75 determines whether or not the route information acquired by the route information acquisition unit 71 includes continuity information indicating the continuity of a plurality of guidance points. The continuous information includes a guide point (hereinafter referred to as "right/left turn point P1") that guides vehicle A (own vehicle) to turn left or right, and a guidance point (hereinafter referred to as "continuous point P2”). If the route information transmitted together with the guidance execution request contains information on a plurality of guidance points, the continuous guidance determination unit 75 determines that these guidance points are continuous, that is, the route information contains continuous information. determined to be

Here, the right/left turn point P1 is limited to a guidance point such as an intersection where a right turn or left turn occurs. On the other hand, the continuous point P2 may be any of a branch point such as an intersection, a destination of the set route, and an alarm point for calling attention such as stopping and slowing down.

In addition, three or more guidance points may be contiguous on the set route at intervals of less than a predetermined distance (hereinafter, "threshold distance") from each other. In this case, the intermediate continuous point P2 sandwiched between the first right/left turn point P1 and the last continuous point P2 is limited to a branch point such as an intersection, like the right/left turn point P1. For example, when a plurality of (four) branches occur consecutively, a set of route information includes information on a plurality (four) of guidance points. The continuous guidance determination unit 75 sets the first guidance point as the right/left turn point P1, and sets the second to fourth guidance points as the continuous point P2 for convenience.

The roadside object determination unit 76 determines whether or not there is a specific object in the roadside area of the road on which the vehicle A travels, based on the detection information acquired by the external world information acquisition unit 74 . The specific object is an object for which conditions have been set in advance as being unsuitable for superimposing a virtual image Vi (hereinafter referred to as "guidance content CNT2", see FIG. 6) for guiding the continuous point P2. Specifically, the roadside object determination unit 76 identifies the shielding object BO (see FIG. 6) provided in the roadside area as described above as the specific object.

The roadside object determination unit 76 cooperates with the vehicle attitude determination unit 77 to grasp the progress of the vehicle A turning left or right at the right/left turn point P1. After the vehicle A starts to turn right or left at the right/left turn point P1 (branch point), the roadside object determination unit 76 detects a road (hereinafter referred to as "connecting road Rc" in FIG. 4), it is determined whether or not there is a shield BO in the roadside area. When there is no shield BO in the roadside area of the connecting road Rc, that is, when the roadside area is open, the roadside object determination unit 76 provides the vehicle posture determination unit 77 with the determination result indicating that there is no shield BO. do.

The vehicle posture determination unit 77 acquires the contents of the guidance content CNT2 (see FIG. 6) used for guidance of the continuous point P2 from the display generation unit 78 as content information. The vehicle posture determination unit 77 estimates the orientation of the vehicle A turning at the right/left turn point P1 with respect to the connecting road Rc. The vehicle posture determination unit 77 grasps the guidance content CNT2 used for guidance of the continuous point P2 based on the content information, and determines whether or not the orientation of the vehicle A is in a direction that enables the display of the grasped guidance content CNT2. (hereinafter referred to as "travel direction determination"). When the vehicle posture determination unit 77 determines that the vehicle A is facing in a direction in which the guidance content CNT2 can be displayed by executing the traveling direction determination, the vehicle posture determination unit 77 establishes the straight traveling determination and permits the display of the guidance content CNT2. Notify 78.

The vehicle posture determination unit 77 can determine the traveling direction using a plurality of pieces of acquired information that are different from each other. The first determination of the traveling direction is performed based on the detection information of the front camera 31 acquired by the external world information acquisition section 74 . This detection information may be imaged data of the front camera 31, or may be an analysis result of the imaged data as described above. Based on the detection information, the vehicle posture determination unit 77 grasps the relative position of the lane marking or the road edge on the connecting road Rc. The vehicle posture determination unit 77 establishes the straight travel determination at the timing when the lane markings or road edges located on both the left and right sides of the vehicle A are within the angle of view of the front camera 31 and can be photographed. . Note that the vehicle posture determination unit 77 may establish the straight travel determination at the timing when the lane marking or the road edge on one side enters the angle of view of the front camera 31 .

The second determination of the traveling direction is performed using the navigation map data or the high-precision map data acquired by the route information acquisition section 71 and the direction information acquired by the position information acquisition section 72 . For example, when neither the lane marking nor the road edge of the connecting road Rc can be detected from the imaging data, the driving direction is determined by combining the map data and the direction information.

The vehicle attitude determination unit 77 calculates an intersection angle θ1 at the right/left turn point P1 between the road leading to the right/left turn point P1 (hereinafter referred to as “first road R1”) and the connecting road Rc based on the map data. The vehicle posture determination unit 77 sets a threshold angle θth (θ1>θth) slightly smaller than the intersection angle θ1, using the intersection angle θ1 as a reference. The vehicle posture determination unit 77 calculates the posture change angle θc in the yaw direction of the vehicle A at the right/left turn point P1 based on the orientation information, with reference to the extending direction of the first road R1. The vehicle posture determination unit 77 establishes the straight travel determination at the timing when the posture change angle θc exceeds the threshold angle θth.

The third driving direction determination is performed using the steering angle information acquired by the own vehicle information acquisition section 73 . For example, when it is difficult to detect the lane markings and the road edge of the connecting road Rc and the reception environment of the positioning signal is not good, the driving direction is determined using the steering angle information.

The vehicle posture determination unit 77 monitors the transition of the steering operation at the right/left turn point P1 based on the steering angle information. The vehicle posture determination unit 77 establishes the straight travel determination when the angular velocity of the steering becomes less than a predetermined velocity, that is, at the timing when the change in the steering angle has stabilized. Note that the vehicle posture determination unit 77 may establish the straight travel determination at the timing when the steering angle becomes less than a predetermined threshold value in the process of returning the steering wheel based on the steering angle information. In this case, the predetermined threshold value for determining whether the vehicle is traveling straight is appropriately changed according to the intersection angle θ1, the shape of the connecting road Rc, and the like, based on the map data.

The vehicle posture determination unit 77 changes the determination criteria for establishing the straight running determination in the traveling direction determination according to the type of the guidance content CNT2. When the continuous point P2 is the destination and the guidance content CNT2 is superimposed on a building or the like in the roadside area serving as the goal point, the vehicle posture determination unit 77 relaxes the determination criteria for establishing the straight-ahead determination, and determines the guidance content CNT2. advance the timing for starting the display of .

In addition, the vehicle posture determination unit 77 changes the determination criteria for determining whether the vehicle is going straight according to the visibility of the roadside area of the connecting road Rc. When the roadside object determination unit 76 determines that there is no shield BO (see FIG. 6) in the roadside area of the connecting road Rc, the vehicle posture determination unit 77 determines that there is a shield BO in the roadside area. Also, the criteria for establishing the straight running determination are relaxed. In this case as well, adjustment is made to advance the start of display of the guidance content CNT2.

The display generator 78 generates video data that is sequentially output to the HUD device 20 . When the route information acquisition unit 71 acquires a guidance execution request output by the navigation ECU 52, the display generation unit 78 creates an original image of the content to be superimposed and displayed in the foreground at the guidance point on each frame image constituting the video data. draw. The display generation unit 78 has a function of selecting content to be used at a guidance point, a function of drawing content, a function of controlling the display period of content, and the like.

The display generation unit 78 selects content to be used for route guidance at each guidance point based on the route information and map data acquired by the route information acquisition unit 71 . The display generation unit 78 performs image processing such as combining the parts data stored in the storage unit 13 to prepare the drawing data of the selected content so that it can be drawn.

The display generation unit 78 uses either one of the guidance execution request notified by the navigation ECU 52 and the display permission notified by the vehicle attitude determination unit 77 as a trigger, and determines the respective timings for starting and ending display of the display content. Control. Based on the acquisition of the guidance execution request or the display permission notice, the display generation unit 78 converts the prepared drawing data into each frame image in front of each guidance point in a manner that matches the traveling of the vehicle A along the set route. Start the drawing process.

When the continuous guidance determination unit 75 determines that the right/left turn point P1 and the continuous point P2 are continuous, the display generation unit 78 generates the right/left turn content CNT1 (see FIG. 5) and the guidance content CNT2 (see FIG. 6). Draw in order on the frame image. The details of guidance display at such a continuous branch will be described based on FIGS. 4 to 6 and with reference to FIGS. 1 and 3. FIG.

When a route is set in the navigation device 50 to continuously turn right and left, the navigation ECU 52 provides guidance at a predetermined distance (for example, 300 m) before the first intersection (right/left turn point P1). Route information is output to HCU 100 together with the request. This route information includes distance information to the first intersection, distance information between two intersections (e.g. 150 m), direction of right and left turns to be made at each intersection, navigation map data indicating the shape of the intersection, and the number of lanes. and lane information indicating the direction of travel allowed for each lane.

Based on the acquisition of the guidance execution request by the route information acquisition unit 71, the display generation unit 78 starts drawing the right/left turn content CNT1 for guiding the right/left turn point P1 on each frame image. As shown in FIG. 5, the right/left turn content CNT1 is displayed superimposed on the road surface of the first road R1 to show the driver the planned travel locus of the vehicle A based on the route information. In the route guidance scene shown in FIG. 4, the right/left turn content CNT1 AR-displays the approach route to the right/left turn point P1 and the exit route from the right/left turn point P1. With such a display, the right/left turn content CNT1 prompts the driver to move to the right turn lane until reaching the right/left turn point P1 and to make a right turn at the right/left turn point P1.

When the vehicle A starts to turn right at the right/left turn point P1, the vehicle posture determination unit 77 determines the traveling direction. When the vehicle posture determination unit 77 determines that the vehicle is traveling straight based on the imaging data and the like, the vehicle posture determination unit 77 notifies the display generation unit 78 of permission to display the guidance content CNT2. The display generation unit 78 starts drawing the guidance content CNT2 for guiding the continuous point P2 on each frame image based on the acquisition of the display permission notification.

As shown in FIG. 6, the guidance content CNT2 is displayed superimposed on the road surface of the connecting road Rc to show the driver the planned travel locus of the vehicle A based on the route information. In the route guidance scene shown in FIG. 4, the guidance content CNT2 displays in AR the approach route to the continuous point P2 and the exit route from the continuous point P2. With such a display, the guidance content CNT2 shows the relative position of the continuous point P2 and prompts the driver to turn left at the continuous point P2.

The details of the guide display process performed by the HCU 100 will be described below based on the flowchart shown in FIG. 7 and with reference to FIGS. 3 and 4. FIG. The guidance display process shown in FIG. 7 is started based on a guidance execution request notified from the navigation ECU 52 to the HCU 100 .

In S101, the navigation-related information related to the guidance execution request that triggered the start of the current guidance display processing, specifically, the position information, direction information, route information, etc. of the vehicle A is acquired, and the process proceeds to S102.

In S102, the route information acquired in S101 is referred to, and it is determined whether continuous guidance is to be performed based on whether continuous information is included in the route information. When it is determined in S102 that continuous guidance is not to be performed, the process proceeds to S104. On the other hand, when it is determined in S102 that continuous guidance is to be performed, the process proceeds to S103. In S103, the number of continuous guidance points is set based on the route information, and the route information in a series of guidance sections is separated as information for each guidance point, and the process proceeds to S104.

In S104, the numerical value of the counter that counts the guidance points is set to "1", and the process proceeds to S105. In S105, based on the numerical value of the counter, content is generated for the current (Nth) guidance point. Specifically, the AR display of the content is started by drawing the content on the video data, and the process proceeds to S106.

In S106, it is determined whether or not there is the next (N+1th) guidance point. If there is no next guidance point, the display end timing of the current (Nth) display content is determined. In this case, for example, the display end timing is the timing when the vehicle has traveled a predetermined distance after passing the current guidance point. If it is determined in S106 that the display end timing has come, the process proceeds to S107.

On the other hand, in S106 when there is the next (N+1th) guidance point (continuous point P2), the traveling direction is determined. If it is determined in S106 that it is time to display the next guidance content CNT2 based on the determination that the vehicle A is going straight, the process proceeds to S107. In S107, the AR display of the current (Nth) guidance content CNT2 is terminated, and the process proceeds to S108.

In S108, it is determined whether or not there are remaining guidance points set in S103. If it is determined in S108 that there are no remaining guidance points, the series of guidance display processing based on this guidance execution request is terminated. On the other hand, when it is determined in S108 that there are remaining guidance points, the process proceeds to S109.

In S109, the value of the guidance point counter is incremented by one, and the process proceeds to S105. In S105 from the second time onward, based on the value of the counter incremented in S109, generation of guidance content CNT2 for guiding the next (N+1th) guidance point, ie, continuous point P2, and AR display are started. Then, in subsequent S106 to S109, it waits for switching or termination of the AR display. When the guidance at all the guidance points is completed in this way, the series of guidance display processing based on the current guidance execution request is terminated.

In the embodiment described so far, when the right/left turn point P1 and the continuous point P2 are continuous, the display of the guidance content CNT2 that guides the continuous point P2 is performed after the start of the right turn or left turn at the right/left turn point P1. It is started on the condition that the straight travel determination is established. According to the above, the display start timing of the guidance content CNT2 for guiding the continuous point P2 can be appropriate. Therefore, it is possible to reduce the possibility of erroneous recognition due to the superimposed display for route guidance.

Specifically, as in the comparative example shown in FIG. 8, when display of content CNTX for guiding the next guidance point (continuous point) during a turn at a right or left turn point is started, at least part of this content CNTX may be superimposed on the roadside shield BO. For example, when switching to the content CNTX is performed based on the result of map matching processing performed by the navigation ECU 52, display switching like this comparative example is likely to occur. Content CNTX, which is started to be displayed when the vehicle A is not facing the front of the road, may be misleading or annoying to the driver. Therefore, it becomes difficult for the information presented by the content CNTX to be correctly transmitted to the driver.

On the other hand, in the present embodiment shown in FIG. 6, the display start timing of the guidance content CNT2 is started in accordance with the determination that the vehicle A is going straight. Therefore, the guidance content CNT2 is superimposed on the correct superimposition target, and the information presentation function of guiding the continuous point P2 can be exhibited with high certainty.

Further, in the present embodiment, delays in the display start timing of the guidance content CNT2 are also reduced. According to the above, the time for recognizing the continuous point P2 is easily secured, so that the driver can smoothly recognize the continuous guidance points.

In addition, in this embodiment, after the start of the right or left turn at the right/left turn point P1, the vehicle A travels by using the information detected by the front camera 31 mounted on the vehicle A about the lane markings or the road edge of the connecting road Rc. A direction determination is performed. Therefore, the vehicle posture determination unit 77 can grasp the orientation of the vehicle A with respect to the connecting road Rc in real time and determine the display start timing of the guidance content CNT2. According to the above, it is possible to improve the accuracy of determining whether the vehicle A is traveling straight, and to further reduce the risk of erroneous recognition of route guidance.

Further, in this embodiment, after starting a right or left turn at the right/left turn point P1, the traveling direction of the vehicle A is determined using the map data and direction information based on the positioning signal. Therefore, even if the connecting road Rc is in a state where it is difficult to detect the lane markings and the road edge, the vehicle posture determination unit 77 can grasp the orientation of the vehicle A with respect to the connecting road Rc and determine the display start timing of the guidance content CNT2. . According to the above, since it is possible to carry out determination of going straight without depending on the road environment, the risk of erroneous recognition of route guidance can be reduced in many driving scenes.

Further, in the present embodiment, after the right turn or left turn is started at the right/left turn point P1, the traveling direction of the vehicle A is determined using the steering angle information. If the end of turning of the vehicle A can be grasped using the own vehicle information in this way, the vehicle posture determination unit 77 can determine the display start timing of the guidance content CNT2 without being affected by the external environment of the vehicle A. According to the above, it is possible to reduce the risk of erroneous recognition of route guidance in more driving scenes.

In addition, in the present embodiment, as shown in FIG. 9, when there is no shield BO (see FIG. 6) in the roadside area of the connecting road Rc, the display of the guidance content CNT2 is started earlier. In such a scene, even if the superimposed display of the guide content CNT2 is started early, superimposition on an erroneous superimposition target does not occur, and the risk of erroneous recognition can be substantially eliminated. Therefore, the process of accelerating the start of display of the guide content CNT2 based on the absence of the shield BO has the advantage of allowing the driver to quickly recognize the continuous point P2.

Further, in the present embodiment, when the superimposition target of the guidance content CNT2 guided at the continuous point P2 is in the roadside area of the connecting road Rc, the criteria for straight travel determination are relaxed compared to the case where the superimposition target is in the road surface area. be. According to this adjustment, in cases such as when the continuous point P2 is the destination, the guidance content CNT2 can be quickly superimposed on buildings, etc., in the roadside area serving as the goal point. Therefore, recognition of the continuous point P2 by the driver can also be implemented early.

In the above embodiment, the navigation map data and the high-precision map data correspond to the "map information", the front camera 31 corresponds to the "external sensor", the display generation unit 78 corresponds to the "display control unit", The shield BO corresponds to the "specific object". Further, the continuous point P2 corresponds to the "sequential guidance point", the connecting road Rc corresponds to the "road", and the HCU 100 corresponds to the "display control device".

(Other embodiments)
As described above, one embodiment of the present disclosure has been described, but the present disclosure is not interpreted as being limited to the above embodiments, and can be applied to various embodiments and combinations within the scope of the present disclosure. be able to.

In the modified example 1 of the above embodiment, the continuous guidance determining section 75 instead of the navigation ECU 52 performs the following determination as to whether or not a plurality of guidance points are continuous. Specifically, the continuous guidance determination unit 75 refers to the route information provided by the navigation ECU 52, and calculates the distance between the right/left turn point P1 and the continuous point P2 that are aligned on the set route indicated by the route information. If the distance between these two points on the set route is shorter than the threshold distance, the continuous guidance determination unit 75 determines that the right/left turn point P1 and the continuous point P2 are continuous, that is, there is continuous information. . The threshold distance used for continuous guidance determination by the continuous guidance determination unit 75 is set to, for example, several hundred to 1000 meters. The threshold distance may be adjustable by the user of vehicle A, such as the driver, or may be automatically adjusted according to the type of continuous points P2.

In modification examples 2 to 4 of the above embodiment, conditions for determining whether the vehicle is going straight are different from those in the above embodiment. The vehicle posture determination unit 77 of Modification 2 performs at least one of travel direction determination based on detection information, travel direction determination using map data and direction information, and travel direction determination using steering angle information, and guide content is determined. If it is determined that CNT2 can be displayed, the straight travel determination is established. Further, the vehicle posture determination unit 77 of Modification 3 establishes the straight running determination based on the majority decision of these three driving direction determinations. Furthermore, the vehicle attitude determination unit 77 of Modification 4 has only one or two determination functions out of the three driving direction determinations.

In the above embodiment, when there is no specific object such as a shield BO in the roadside area of the connecting road Rc, the switching timing to the guidance content CNT2 is advanced. A specific type of such a specific object is not limited to the shielding object BO described above, and may be changed as appropriate according to, for example, the contents of the guidance content CNT2. Further, the process of recognizing a specific object may be performed by the perimeter monitoring sensor 30 or the driving support ECU 57 , or may be performed by the external world information acquisition section 74 or the roadside object determination section 76 .

In the fifth modification of the above-described embodiment, the display generation unit 78 determines the display start timing of the guidance content CNT2 based on the presence or absence of the specific object in the roadside area. That is, the display generation unit 78 of Modification 5 acquires the determination result indicating that there is no shield BO from the roadside object determination unit 76, and advances the display start timing. Further, in Modification 6 of the above-described embodiment, the process of adjusting the display start timing of the guidance content CNT2 based on the presence or absence of the specific object in the roadside area is omitted. Furthermore, in Modification 7 of the above-described embodiment, the process of adjusting the display start timing of the guidance content CNT2 based on whether or not the superimposition target is in the roadside area is omitted.

The aspect of the content superimposed on the foreground at each guidance point may be changed as appropriate. In Modification 8 of the above-described embodiment, a non-AR display object is used for guiding the guide point. In this modification 8, the AR display object and the non-AR display object are appropriately used according to the type of the guide content CNT2. The non-AR display object is not superimposed on a specific superimposition target, and is displayed as if it is relatively fixed to the vehicle configuration such as the windshield WS.

In Modification 9 of the above embodiment, a user terminal such as a smart phone is connected to an in-vehicle network. A route to a destination is set in an application executed on a user terminal by a user's operation of a driver or the like. The user terminal can provide route information to the destination, related navigation map data, and the like to the route information acquisition unit 71 through an in-vehicle network or the like.

Further, the route information acquisition unit 71 of Modification 10 of the above-described embodiment can acquire route information, map data, and the like from a server on the cloud through the network outside the vehicle. As in the above modified examples 9 and 10, if the information necessary for route guidance can be acquired from a smartphone or a cloud server, the navigation device does not have to be installed in the vehicle.

In Modified Example 11 of the above-described embodiment, the HCU 100 and the HUD device 20 are configured integrally. That is, the processing functions of the HCU 100 are implemented in the control circuit of the HUD device 20 of the eleventh modification.

The specific configuration of the projector of the HUD device used for superimposed display may be changed as appropriate. For example, the HUD device of Modification 12 is provided with an EL (Electro Luminescence) panel instead of the LCD and the backlight. Furthermore, instead of the EL panel, a projector using a display device such as a plasma display panel, a cathode ray tube, and an LED can be adopted as the HUD device.

Further, the HUD device of Modification 13 is provided with a laser module (hereinafter "LSM") and a screen instead of the LCD and backlight. The LSM has a configuration including, for example, a laser light source and a MEMS (Micro Electro Mechanical Systems) scanner. The screen is for example a micromirror array or a microlens array. In the HUD device of Modification 14, a display image is drawn on the screen by scanning laser light emitted from the LSM. A HUD device projects a display image drawn on a screen onto a windshield using a magnifying optical element to display a virtual image in mid-air.

Further, the HUD device of Modification 15 is provided with a DLP (Digital Light Processing, registered trademark) projector. A DLP projector has a digital mirror device (hereinafter “DMD”) provided with a large number of micromirrors and a projection light source that projects light toward the DMD. A DLP projector renders a display image on a screen through coordinated control of the DMD and the projection light source. Furthermore, the HUD device of Modification 16 employs a projector using LCOS (Liquid Crystal On Silicon). Furthermore, in the HUD device of Modification 17, a holographic optical element is adopted as one of the optical systems for displaying a virtual image in mid-air.

Each function provided by the HCU in the above embodiments can be provided by software and hardware for executing it, only software, only hardware, or a complex combination thereof. Furthermore, if such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits, including numerous logic circuits, or analog circuits.

Also, the form of the storage medium storing the program and the like capable of implementing the display control method described above may be changed as appropriate. For example, the storage medium is not limited to being provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to the control circuit of the HCU. . Furthermore, the storage medium may be an optical disc, hard disk drive, or the like, which is the basis for copying the program to the HCU.

The vehicle equipped with the HMI system is not limited to a general private passenger car, and may be a rental car vehicle, a manned taxi vehicle, a ride-sharing vehicle, a freight vehicle, a bus, or the like. Furthermore, the HMI system and the HCU may be installed in vehicles dedicated to unmanned operation used for mobility services.

The controller and techniques described in this disclosure may be implemented by a special purpose computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and techniques described in this disclosure may be implemented by dedicated hardware logic circuitry. Alternatively, the apparatus and techniques described in this disclosure may be implemented by one or more special purpose computers configured in combination with a processor executing a computer program and one or more hardware logic circuits. The computer program may also be stored as computer-executable instructions on a computer-readable non-transitional tangible recording medium.
Technical ideas understood from the embodiments and modifications described so far will be described below.
(Appendix 1)
A display control device used in a vehicle (A) for controlling display superimposed on the foreground for route guidance,
The route information used for the route guidance includes continuation information indicating the continuation of the right/left turn point (P1) that guides the vehicle to turn left or right, and the sequential guidance point (P2) that provides guidance next to the right/left turn point. A continuous guidance determination unit (75) that determines whether or not
Estimate the orientation of the vehicle with respect to the road (Rc) on which the vehicle is traveling, and determine whether or not the orientation of the vehicle is such that the guidance content (CNT2) for guiding the sequential guidance points can be displayed. a part (77);
When it is determined that the continuous information is included in the route information, after displaying the right/left turn content (CNT1) for guiding the right/left turn point, the direction of the vehicle becomes a direction in which the guidance content can be displayed. a display control unit (78) for starting the display of the guidance content on the condition that
A display controller comprising:
(Appendix 2)
The vehicle orientation determination unit determines that the orientation of the vehicle has become an orientation in which the guidance content can be displayed, based on information detected by an external sensor (31) mounted on the vehicle on the division line of the road or the edge of the road. The display control device according to Supplementary Note 1, which determines that.
(Appendix 3)
The vehicle attitude determination unit can display the guidance content according to the orientation of the vehicle using map information about the road on which the vehicle is traveling and direction information of the vehicle based on a positioning signal received from a positioning satellite. 3. The display control device according to Supplementary Note 1 or 2, which determines that the display has been oriented.
(Appendix 4)
4. The vehicle attitude determination unit according to any one of appendices 1 to 3, using steering angle information about the steering angle of the vehicle to determine that the orientation of the vehicle has become an orientation in which the guidance content can be displayed. A display controller as described.
(Appendix 5)
Roadside object determination unit ( 76), further comprising
5. The method according to any one of appendices 1 to 4, wherein when it is determined that the specific object is not in the roadside area, the display of the guidance content is started earlier than when it is determined that the specific object is in the roadside area. display controller.
(Appendix 6)
The vehicle attitude determination unit determines whether the guidance content can be displayed when the guidance content is superimposed on the roadside area of the road, compared to when the guidance content is superimposed on the road surface area. 6. The display control device according to any one of appendices 1 to 5, which relaxes the judgment criteria for judging.
(Appendix 7)
A display control program used in a vehicle (A) for controlling a display superimposed on the foreground for route guidance,
in at least one processing unit (11),
The route information used for the route guidance includes continuation information indicating the continuation of the right/left turn point (P1) that guides the vehicle to turn left or right, and the sequential guidance point (P2) that provides guidance next to the right/left turn point. is included (S102),
The orientation of the vehicle with respect to the road (Rc) on which the vehicle is traveling is estimated, and it is determined whether or not the orientation of the vehicle is such that the guidance content (CNT2) for guiding the sequential guidance points can be displayed (S106). ,
When it is determined that the continuous information is included in the route information, after displaying the right/left turn content (CNT1) for guiding the right/left turn point, the direction of the vehicle becomes the direction in which the guidance content can be displayed. on the condition that the display of the guidance content is started (S105),
A display control program that executes processing including:

A vehicle, BO shield (specific object), CNT1 right/left turn content, CNT2 guidance content, P1 right/left turn point, P2 continuous point (sequential guidance point), Rc connecting road (road), 11 processing unit, 31 front camera (outside world sensor), 75 continuous guidance determination unit, 76
Roadside object determination unit 77 Vehicle posture determination unit 78 Display generation unit (display control unit) 100 HCU (display control unit)

Claims (2)

A display control device for controlling display by a head-up display (20),
a route information acquisition unit (71) for acquiring route guidance information from a navigation device (50) that provides route guidance to a destination;
A display control unit (78) for superimposing and displaying route guidance content on the road surface from the route guidance information,
The display control unit, when determining that the route guidance information includes continuous information indicating that a plurality of guidance points for which route guidance is to be provided are continuous on the route with an interval therebetween, performs the first route guidance. After the first guidance content ( CNT1 ) is displayed, the second guidance content ( CNT2 ) for performing the second route guidance is hidden after the first guidance content is hidden. A display control device that displays at the timing when the superimposed display of is possible. A display control device for controlling display by a head-up display (20),
a route information acquisition unit (71) for acquiring route guidance information from a navigation device (50) that provides route guidance to a destination;
A display control unit (78) for superimposing and displaying route guidance content on the road surface from the route guidance information,
The display control unit, when determining that the route guidance information includes continuous information indicating that a plurality of guidance points for which route guidance is to be provided are continuous on the route with an interval therebetween, performs the first route guidance. Based on the fact that the first guidance content (CNT1) is displayed and the traveling direction of the vehicle is estimated to be in the direction along the road toward the second guidance point (P2), which is the target of the second route guidance, 2. A display control device for starting the display of the second guidance content (CNT2) for the first route guidance .

Images