JP2021117220A - Display device, mobile body, method for display, and program - Google Patents

Abstract

To allow route guidance information to be presented even when a projection angle of view is small.SOLUTION: The display device according to an embodiment of the present invention is a display device which displays a projection target image. The display device includes: an image generation unit for generating a presentation image which presents route guidance information; and a projection unit provided in a mobile body, the projection unit projecting the projection target image including the presentation image, the presentation image presenting the direction in which the mobile body is to move.SELECTED DRAWING: Figure 11

Description

The present invention relates to a display device, a moving body, a display method, and a program.

Conventionally, a display device such as a head-up display has been known in which various information is superimposed on the foreground of a vehicle by projecting a projected image including various information onto the front windshield of the vehicle, and the driver of the vehicle can visually recognize the various information. ing.

Further, as one of the display information in such a display device, as the route guidance information for guiding the route of the vehicle, an image presenting the planned travel route (scheduled movement route) of the vehicle is displayed, and the remaining information to the place to be notified is displayed. A technique for changing this display form based on a distance is disclosed (see, for example, Patent Document 1).

However, in the technique of Patent Document 1, a wide-angle projection angle of view is required to display an image, and the size and cost of the display device may increase.

An object of the present invention is to make it possible to present route guidance information even with a narrow projection angle of view.

The display device according to one aspect of the present invention is a display device that displays a projected image, includes an image generation unit that generates a presentation image that presents route guidance information, and a moving body that includes the presentation image. A projection unit for projecting the projected image is provided, and the presented image presents a planned movement direction of the moving body.

According to the present invention, route guidance information can be presented even with a narrow projection angle of view.

It is a figure which shows the configuration example in which the display device which concerns on embodiment is mounted on a vehicle. It is a figure which shows the structural example of the display part. It is a block diagram which shows the configuration example of the in-vehicle system in which the display device is mounted on a vehicle. It is a block diagram which shows the configuration example of a detection device. It is a block diagram which shows the functional configuration example of a recognition device. It is a figure which shows the relative position example of an obstacle with respect to a vehicle or a display component of a vehicle. It is a block diagram of the hardware configuration example of the control part which concerns on embodiment. It is a block diagram of the functional structure example of the control part which concerns on embodiment. It is a flowchart which shows the processing example by the control part which concerns on embodiment. It is a figure which shows the display example in the foreground when the remaining distance is not less than a presentation threshold value. It is a figure which shows the display example in the foreground when the remaining distance is less than a presentation threshold value. It is a figure which shows the relationship between the size of the projected image and the presented image, (a) is a figure which concerns on a comparative example, and (b) is a figure which concerns on embodiment. It is a figure which shows the relationship example of the size of the projected image, and the projected angle of view. It is a figure which shows the relationship example of the display interval of an arrow figure, and the superimposition range. It is a figure explaining the display position example of an arrow figure.

Hereinafter, modes for carrying out the invention will be described with reference to the drawings. In each drawing, parts having the same configuration may be designated by the same reference numerals, and duplicate description may be omitted.

In the embodiment, a presented image that presents the planned movement direction of the moving body is generated as route guidance information, and a projected image including the presented image is projected and displayed by a projection unit provided on the moving body. By projecting a small image that presents the planned movement direction of the moving body without projecting a large image for presenting the planned movement route of the moving body, it is possible to present route guidance information even with a narrow projection angle of view. do.

<Terms in the embodiment>
(Mobile)
A movable object. For example, moving objects include land-moving bodies such as automobiles, vehicles, trains, trains or forklifts, and aerial moving bodies such as airplanes, balloons or drones that can move in the air, such as ships, ships and steamers. Alternatively, it includes a floating body such as a boat that can move on the water.

(foreground)
The scenery in front of the moving body that the driver of the moving body sees. For example, the scenery in front of the vehicle that the driver sees through the front windshield corresponds to the foreground.

(Planned movement route)
Of the routes in which the moving body is scheduled to move from the starting point to the destination, the route in which the moving body is scheduled to move most recently. In the embodiment, since the case where the moving body is a vehicle is taken as an example, the planned movement route will be described as the planned travel route of the vehicle.

(Planned movement direction)
The direction in which the moving body is scheduled to move in the planned movement route of the moving body. For example, when there is an intersection on the planned movement route, the direction in which the moving body should travel at the intersection corresponds to the planned movement direction. In the embodiment, since the case where the moving body is a vehicle is taken as an example, the planned moving direction will be referred to as the planned traveling direction of the vehicle.

(Route guidance information)
Information that guides the route that a moving body travels from the starting point to the destination. The route guidance information includes information on the planned travel route (planned travel route), the planned travel direction (planned travel direction), information on the point where the route guidance information is presented, and the like.

(Projected image)
An image projected by a projection unit and made visible to the driver of a moving object.

(Presentation image)
An image that presents route guidance information. By being projected by the projection unit, it is visually recognized by the driver of the moving body. The presented image according to the embodiment can present the planned movement direction of the moving body (the planned travel direction of the vehicle).

(driver)
A person who drives or steers a moving body.

Hereinafter, embodiments will be described using a head-up display (HUD) mounted on a vehicle as an example of a display device.

<Configuration example>
FIG. 1 is a diagram showing an example of a configuration in which the display device 1 according to the embodiment is mounted on the own vehicle 8.

The display device 1 is embedded inside a dashboard provided in the own vehicle 8, and projects an image to be projected from the injection window 3 provided on the upper surface of the display device 1 toward the front windshield 71. The projected image is displayed as a virtual image I in the positive Y direction from the front windshield 71, and is visually recognized by the driver V of the own vehicle 8. By visually recognizing the virtual image I, the driver V can acquire useful information for driving with a small movement of the line of sight while keeping the line of sight of another vehicle or the road surface in the positive Y direction of the own vehicle 8. ..

The display device 1 may be installed on the ceiling, sun visor, or the like in addition to the dashboard, as long as the projected image can be projected on the front windshield 71.

As shown in FIG. 1, the display device 1 includes a projection unit 10 and a control unit 20 as main components. The projection unit 10 projects the projected image toward the front windshield 71. As a projection method of the projected image by the projection unit 10, a laser scanning method and a panel method are known. The laser scanning method is a method in which a laser beam emitted from a laser light source is scanned by a two-dimensional scanning device to form an intermediate image (a real image projected on a screen described later). The panel method is a method of forming an intermediate image with an imaging device such as a liquid crystal panel, a DMD (Digital Micro-mirror Device) panel, or a vacuum fluorescent display (VFD).

The laser scanning method is different from the panel method in which an image is formed by emitting light from the entire screen and partially blocking light, and since it is possible to assign light emission / non-light emission to each pixel, a high-contrast image is generally obtained. It is suitable in that it can be formed. Since the high contrast improves the visibility, it has been clarified that the occupants of the vehicle can visually recognize the information with less attention resources than the panel type HUD.

In particular, in the panel method, light that cannot be completely blocked is projected even in an area where there is no information, and the display frame is projected in a range where the HUD can display an image (this phenomenon is called a postcard). The laser scanning method does not have such a phenomenon and can project only the content. In particular, in AR (Augmented Reality), the reality when the generated image is superimposed and displayed on the actual landscape is improved.

Note that AR is translated as "augmented reality" and refers to a technology that virtually expands the world in front of you by superimposing images of non-existent objects on an existing landscape. However, even if it is a panel-type HUD, it may be a HUD that can display information in a manner that can be visually recognized (less tired) with less attention resources.

FIG. 2 is a diagram showing an example of the configuration of the projection unit 10. The projection unit 10 mainly includes a light source unit 101, a light deflector 102, a mirror 103, a screen 104, and a concave mirror 105. Note that FIG. 2 only shows the main components, and may include components other than those shown in the figure, or may not include some of the components shown in the figure.

The light source unit 101 includes three laser light sources (hereinafter referred to as LD: laser diode) corresponding to Red (R), Green (G), and Blue (B), a coupling lens, an aperture, a synthetic element, a lens, and the like. The laser beams emitted from the three LDs are combined and guided toward the reflecting surface of the light deflector 102. The laser beam guided to the reflecting surface of the light deflector 102 is two-dimensionally deflected by the light deflector 102.

As the optical deflector 102, one minute mirror that swings with respect to two orthogonal axes, two minute mirrors that swing or rotate about one axis, and the like can be used. The optical deflector 102 can be a MEMS (Micro Electro Mechanical Systems) mirror manufactured by a semiconductor process or the like. The optical deflector 102 can be driven by an actuator whose driving force is the deformation force of the piezoelectric element. However, as the light deflector 102, a galvano mirror, a polygon mirror, or the like may be used.

The laser beam two-dimensionally deflected by the light deflector 102 is incident on the mirror 103, folded back by the mirror 103, and draws a two-dimensional image (intermediate image) on the surface (scanned surface) of the screen 104. As the mirror 103, a concave mirror or the like can be used, but a convex mirror or a plane mirror may be used. By deflecting the direction of the laser beam with the light deflector 102 and the mirror 103, the size of the projection unit 10 can be reduced or the arrangement of the components can be flexibly changed.

As the screen 104, it is preferable to use a microlens array or a micromirror array having a function of diverging the laser beam at a desired divergence angle, but a diffuser plate for diffusing the laser beam, a transmission plate having a smooth surface, or a reflector. Etc. may be used. Generally, the light source unit 101 to the screen 104 are called a HUD device. However, other parts may be included.

The laser beam emitted from the screen 104 is reflected by the concave mirror 105 and projected onto the front windshield 71. The concave mirror 105 has a function similar to that of a lens, and has a function of forming an image at a predetermined focal length. Therefore, assuming that the concave mirror 105 is a lens, the image on the screen 104 corresponding to the object is formed at the position of the _{distance R 2 determined by the focal length of the concave mirror 105.} Therefore, when viewed from the occupant such as the driver V of the own vehicle 8, the virtual image I is displayed at the position of the _{distance R 1} + R _{2 from the front windshield 71.} Assuming that the distance from the driver V of the own vehicle 8 to the front windshield 71 is R ₃ , in FIG. 2, a virtual image is formed at the position of the _{distance R (= R 1} + R ₂ + R ₃ ) from the viewpoint E of the driver V of the own vehicle 8. I is displayed (imaging).

At least a part of the light flux to the front windshield 71 is reflected toward the viewpoint E of the driver V of the own vehicle 8. As a result, the driver V of the own vehicle 8 can visually recognize the virtual image I in which the intermediate image of the screen 104 is enlarged through the front windshield 71. In other words, when viewed from the driver V of the own vehicle 8, a virtual image I in which the intermediate image is enlarged and displayed is displayed through the front windshield 71.

In general, the front windshield 71 is not flat but slightly curved. Therefore, the image formation position of the virtual image I is determined not only by the focal length of the concave mirror 105 but also by the curved surface of the front windshield 71, but the distance R is substantially determined by the _{distance R 1} + R _{2 as described above.} In order to form the virtual image I in the distance so that the movement of the line of sight is small, the distance R ₁ or R ₂ is lengthened. Distance R ₁ as a method of long there is a way to fold the optical path by a mirror, the distance R2 as a method for long, there is a method of adjusting the focal distance of the concave mirror 105.

Since the front windshield 71 causes optical distortion in which the horizontal line of the intermediate image becomes convex upward or downward, at least one of the mirror 103 and the concave mirror 105 is designed and arranged so as to correct the distortion. Is preferable. Alternatively, it is preferable that the projected image is corrected in consideration of distortion.

Further, a combiner may be arranged as a transmission reflection member on the viewpoint E side of the front windshield 71. Even if the combiner is irradiated with the light from the concave mirror 105, the information can be presented as a virtual image I as in the case where the front windshield 71 is irradiated with the light from the concave mirror 105.

<Configuration example of in-vehicle system 2 on which the display device 1 is mounted>
Next, an in-vehicle system 2 in which the display device 1 is mounted on the own vehicle 8 will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the configuration of the in-vehicle system 2 in which the display device 1 is mounted on the own vehicle 8.

The in-vehicle system 2 includes a car navigation system 11 that communicates via an in-vehicle network NW such as a CAN (Controller Area Network) bus, an engine ECU (Electronic Control Unit) 12, a display device 1, a brake ECU 13, and a steering ECU 14. It includes a recognition device 15 and a detection device 16.

The car navigation system 11 includes a GNSS (Global Navigation Satellite System) represented by GPS (Global Positioning System), detects the current location of the own vehicle 8, and displays the position of the own vehicle 8 on an electronic map.

In addition, it accepts the input of the departure point and the destination, searches for the route guidance information from the departure point to the destination and displays it on the electronic map, or voices the direction of travel to the occupants of the own vehicle 8 before changing the course. Guidance is provided by characters (displayed on the display) or animation. The car navigation system 11 may communicate with the server via a mobile phone network or the like. In this case, the server can transmit the electronic map to the own vehicle 8 and search the route.

The engine ECU 12 determines an ideal fuel injection amount, controls the advance / retard of the ignition timing, the valve operating mechanism, and the like according to the information from each sensor and the situation of the own vehicle 8. In addition, the necessity of shifting is determined by referring to a map in which the switching line of the shifting stage is defined for the relationship between the current vehicle speed and the accelerator opening. The engine ECU 12 combines these to perform acceleration and deceleration control during follow-up traveling to the preceding vehicle. An electric motor may be used as a power source together with the engine or without using the engine as a power source.

The brake ECU 13 has ABS (Antilock Braking System) control, braking control when following a preceding vehicle, automatic braking based on TTC (Time To Collision) with an object, stop state maintenance control when starting on a slope, and the like. Even if the driver V of the vehicle 8 does not operate the brake pedal, the braking force is controlled for each wheel of the own vehicle 8.

The steering ECU 14 detects the steering direction and steering amount of the steering wheel by the driver V of the own vehicle 8 and performs power steering control to add steering torque to the steering direction. Further, steering is performed in the direction of avoiding deviation from the traveling lane, in the direction of maintaining the traveling in the center of the traveling lane, or in the direction of avoiding approaching an obstacle, even if the driver V of the own vehicle 8 does not operate the steering wheel. do.

The detection device 16 includes various sensors, and detects an object around the own vehicle 8 and detects operation information of the own vehicle 8 by the driver V.

The recognition device 15 determines what the object detected by the detection device 16 is, the relative position (direction and distance) of the object with respect to the own vehicle 8 and the relative position (direction and distance) of the object with respect to the display component indicating the own vehicle 8. recognize. Information such as vehicle speed, identification result of the object, and relative position are input to the display device 1.

<Configuration example of detection device 16>
Next, the configuration of the detection device 16 will be described with reference to FIG. FIG. 4 is a block diagram showing an example of the configuration of the detection device 16. The detection device 16 relates to a vehicle speed sensor 161 that detects the vehicle speed displayed by the display device 1, a vehicle information sensor 162 that acquires vehicle information displayed by the display device 1, a radar sensor 163 and a surrounding camera 164 that detect an object, and an occupant. An occupant status information sensor 165 that acquires occupant status information, which is information, a VICS receiver 166 (registered trademark. VICS: Vehicle Information and Communication System Center) that receives congestion information from the outside, and an external communication device that connects to the Internet, etc. It is equipped with 167 and the like.

It should be noted that the sensors included in the detection device 16 do not have to be collectively present in the detection device 16, and may be mounted on a vehicle.

The vehicle speed sensor 161 detects a magnet that rotates with the rotation of the shaft of the drive train system by a sensor unit fixed to the vehicle body, and generates a pulse wave proportional to the rotation speed. The vehicle speed can be detected by the number of pulses per unit time.

The vehicle information sensor 162 includes one or more sensors that detect vehicle information other than the vehicle speed sensor 161. For example, a fuel gauge sensor, a shift lever position sensor, an odometer, a trip meter, a winker sensor, a water temperature sensor, and the like. It suffices if each vehicle information can be acquired in these with a general configuration. The fuel gauge sensor detects the current residual fuel.

The shift lever position sensor detects the shift lever position operated by the driver V of the own vehicle 8. The odometer accumulates the mileage of the own vehicle 8 to provide the total mileage. The trip meter provides the section mileage from the initialization by the driver V of the own vehicle 8 to the present.

The winker sensor detects the direction of the winker operated by the driver V of the own vehicle 8. The water temperature sensor detects the temperature of the engine cooling water. These are only examples of information that can be acquired from the vehicle, and in addition, the information that can be acquired from the own vehicle 8 can be vehicle information. For example, in an electric vehicle or a hybrid vehicle, the remaining battery level, the amount of regenerative power, the amount of power consumption, and the like can be obtained.

The peripheral camera 164 is an imaging device that images the surroundings of the vehicle. The mounting locations are preferably installed in a plurality of locations so that the rear can be imaged from the side for the vehicle. For example, it is installed in the left rear corner, the right rear corner, the rear part, etc. of the vehicle on the roof or bumper of the own vehicle 8. The image pickup device installed at the rear is called a back monitor, but the peripheral camera 164 at the rear is not limited to the back monitor. In addition, it may be arranged on a side mirror, a pillar, a side portion of the roof, a door, or the like. The peripheral camera 164 may include an imaging device that images the front. For example, it can be installed on or near the back of a rear-view mirror.

The peripheral camera 164 may be a monocular camera or a stereo camera. In the case of a monocular camera or a stereo camera capable of acquiring distance information, the radar sensor 163 may not be provided. However, in addition to the peripheral camera 164 that can acquire the distance information, the radar sensor 163 fuses (integrates) the distance information of the peripheral camera 164 and the distance information of the radar sensor 163, and complements each other's disadvantages. Highly accurate distance information can be acquired. In addition to the radar sensor 163 and the surrounding camera 164, a sonic sensor (ultrasonic sensor) or the like may be provided.

The radar sensor 163 transmits radar to the surroundings of the own vehicle 8, such as the front, side, and rear of the own vehicle 8, and receives the radar reflected by an object and returned. The placement location may be any location where obstacles around the own vehicle 8 can be detected. The radar sensor 163 has a TOF method (Time Of Fright) in which the distance to an object is detected by the time from transmission to reception and the direction of the object is detected by the irradiation direction of the radar.

LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) is known as a TOF type radar sensor. In addition, the FMCW method (Frequency Modulation Continuous Wave) that generates a mixed wave of the received wave and the transmitted wave while continuously increasing the frequency of the transmitted wave and converts the humming frequency of the mixed wave generated by the slightly different frequencies into a distance. ). In the FMCW method, the direction of an object is estimated by detecting the phase shift of the received wave with a plurality of receiving antennas.

The occupant state information sensor 165 is a sensor that detects occupant state information directly or indirectly detected from the occupants of the own vehicle 8. A typical one is a face camera. The face camera captures the face of the occupant of the own vehicle 8 and performs face recognition to identify or identify the occupant of the own vehicle 8. In addition, it becomes possible to detect the direction of the face and the direction of the line of sight from the face image.

In addition, the occupant state information sensor 165 may include, for example, an electrocardiographic sensor, a heart rate sensor, a blood pressure sensor, a body temperature sensor, a pulse sensor, a respiratory sensor, a sweating sensor, a blinking sensor, a pupil sensor, a brain wave sensor, a myoelectric potential sensor, or the like. It may be. As the occupant state information sensor 165, for example, there is a form in which a vehicle occupant wears a wristwatch-type wearable terminal (smart watch).

The VICS receiver 166 receives radio waves distributed by VICS. VICS is a system that transmits road traffic information such as traffic congestion and traffic regulation to an in-vehicle device in real time using FM multiplex broadcasting or a beacon. The external communication device 167 connects to the Internet or the like via a network such as 3G, 4G, 5G, LTE and wireless LAN, and receives various information. For example, it can receive weather information such as rain, snow, and fog.

You may also receive news, music, videos and the like. Further, the external communication device 167 can acquire, for example, the state information of the traffic light and the time until the signal changes. In this way, the VICS receiving device 166 and the external communication device 167 may perform road-to-vehicle communication. In addition, the external communication device 167 may acquire the information detected by the other vehicle 6 in the inter-vehicle communication.

In addition, the advanced driver assistance system (ADAS) not only displays information and warns, but also controls the own vehicle 8. In this case, the ECU cooperates with the engine ECU 12, the brake ECU 13, and the steering ECU 14 to provide various driving support based on the distance information of the object detected by at least one of the radar sensor 163 and the surrounding camera 164. For example, acceleration / deceleration control, automatic braking, avoidance of deviation from the traveling lane, lane keeping traveling, avoidance steering of an object, and the like during follow-up traveling to a preceding vehicle are performed. For such control, the ECU recognizes road paint such as white lines from the image captured by the surrounding camera 164.

In acceleration / deceleration control during follow-up travel, the ECU controls power and braking force so as to maintain a target distance according to the vehicle speed. In automatic braking, a warning, a display prompting the brake pedal to be pressed, a seatbelt winding when there is a high possibility of a collision, collision avoidance braking, and the like are performed according to the TTC. In avoiding deviation from the traveling lane, the driving support ECU 36 recognizes a white line (traveling division line) from the captured image and applies steering torque in a direction opposite to the direction deviating from the traveling lane.

Further, in lane-keeping driving, the center of the traveling lane is set as the target traveling line, and steering torque proportional to the deviation from the target traveling line is added in the direction opposite to the deviation. In avoidance steering of an object, when it is determined that a collision cannot be avoided by braking, a traveling line for avoiding the object is determined, and steering torque traveling on this traveling line is added.

Further, for example, when the radar sensor 163 or the surrounding camera 164 detects a vehicle traveling in parallel in an area (blind spot area) that is not reflected in the door mirror of the adjacent lane when changing lanes, the occupant is warned. Such support is called a blind spot monitor.

<Example of functional configuration of recognition device 15>
Next, the functional configuration of the recognition device 15 will be described with reference to FIG. FIG. 5 is a block diagram showing an example of the functional configuration included in the recognition device 15. The recognition device 15 includes a target determination unit 151 and a relative position determination unit 152.

The object determination unit 151 mainly analyzes the ambient image data obtained by the surrounding camera 164 to determine the classification of the captured object. In the embodiment, for example, another vehicle, a pedestrian, a two-wheeled vehicle, or the like is determined.

When the peripheral camera 164 is a stereo camera, each pixel or pixel block of the peripheral image data includes distance information. Such surrounding image data is called a distance image. Further, the target determination unit 151 can determine an object not only from the surrounding image data but also from the radar ranging information. For example, when the density of the point cloud of LIDAR is sufficiently high, the shape of the object can be obtained, so that the shape can be analyzed to determine the classification of the object.

As one of the methods for the object determination unit 151 to determine an object, there is an image recognition method using machine learning. Machine learning is a technology for making a computer acquire learning ability like a human being, and the computer autonomously generates an algorithm necessary for judgment such as data identification from learning data taken in advance, and newly It is a technology that applies this to various data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, intensive learning, and deep learning, and may be a learning method that combines these learning methods. Machine learning It doesn't matter how you learn for. Machine learning methods include perceptrons, deep learning, support vector machines, logistic regression, naive bays, decision trees, and random forests.

As shown in FIG. 6, the relative position determining unit 152 determines the relative position (direction and distance) of the object with respect to the own vehicle 8 or the display component 62 indicating the own vehicle 8. FIG. 6 is a diagram illustrating an example of the relative position of the object with respect to the own vehicle 8 or the display component 62 indicating the own vehicle 8. As shown in FIG. 6, it is assumed that the display component 62 (center of the virtual image I) indicating the own vehicle 8 is displayed at the position Q in the vehicle width direction and P in the vehicle length direction from the center of the own vehicle 8.

It is assumed that the other vehicle 6 is detected to have the coordinates (A, B) with the center of the own vehicle 8 as the origin by the surrounding image data or the radar distance measurement information. Further, the distance from the center of the own vehicle 8 to the other vehicle 6 is represented _{by L 1} , and the direction thereof is _{represented by θ 1.} If the surrounding image data is a distance image, (A, B) can be obtained directly, but if it is radar distance measurement information, the distance and direction may be decomposed into (A, B). The relative position-fixing unit 152 converts the coordinates (A, B) into the coordinates (C, D) based on the display component 62 indicating the own vehicle 8.

Since the display component 62 indicating the own vehicle 8 is presented as a virtual image I, it exists at a fixed position in front of the own vehicle 8. As described above, assuming that the distance in the vehicle width direction from the center of the vehicle to the display component 62 indicating the own vehicle 8 is Q and the distance in the vehicle length direction is P, the coordinates (C, D) are determined as follows.
(C, D) = (A + Q, B + P)

The relative position determining unit 152 performs the same processing for each radar sensor or surrounding camera arranged in the vehicle, and determines the relative position of the object with respect to the display component 62 indicating the own vehicle 8. The distance and direction can also be obtained by determining the relative position. The distance from the display component 62 indicating the own vehicle 8 to the other vehicle 6 is represented _{by L 2} , and the direction thereof is _{represented by θ 2.} In the example of FIG. 6, the distance L ₂ = √ (C ² + D ² ) and the direction θ ₂ = arctan (D / C). The reference for the directions θ ₁ and θ ₂ (which direction is 0 degree) may be set as appropriate. In FIG. 6, the 9 o'clock direction is set to 0 degree.

The recognition device 15 includes classification information indicating the classification of the object determined by the target determination unit 151, distance information from the own vehicle 8 to the object acquired by the relative position determination unit 152, and relative to the object 8 with respect to the own vehicle 8. The position information indicating the position is output to the control unit 20.

<Hardware configuration example of control unit 20>
Next, the hardware configuration of the control unit 20 will be described with reference to FIG. 7. FIG. 7 is a block diagram showing an example of the hardware configuration of the control unit 20. As shown in FIG. 7, the control unit 20 includes an FPGA (Field-Programmable Gate Array) 201, a CPU (Central Processing Unit) 202, a ROM (Read Only Memory) 203, a RAM (Random Access Memory) 204, and the like. It includes an I / F (Interface) 205, a bus line 206, an LD driver 207, and a MEMS controller 208. The FPGA 201, CPU 202, ROM 203, RAM 204, and I / F 205 are connected to each other via the bus line 206.

The CPU 202 controls each function of the control unit 20. The ROM 203 stores a program 203p executed by the CPU 202 to control each function of the control unit 20. The program 203p is expanded in the RAM 204, and is used as a work area for the CPU 202 to execute the program 203p. Further, the RAM 204 includes an image memory 209.

The image memory 209 is used to generate an image projected as a virtual image I. The I / F 205 is an interface for communicating with the recognition device 15 and the detection device 16, and is connected to the CAN (Controller Area Network) bus of the vehicle 9, Ethernet (registered trademark), or the like.

The FPGA 201 controls the LD driver 207 based on the image created by the CPU 202. The LD driver 207 controls the light emission of the LD according to the image by driving the LD of the light source unit 101 of the projection unit 10. The FPGA 201 operates the light deflector 102 of the projection unit 10 via the MEMS controller 208 so that the laser beam is deflected in the direction corresponding to the pixel position of the image.

<Example of functional configuration of control unit 20>
Next, the functional configuration of the control unit 20 will be described with reference to FIG. FIG. 8 is a block diagram showing an example of the functional configuration of the control unit 20.

As shown in FIG. 8, the control unit 20 includes a remaining distance acquisition unit 21, a determination unit 22, a direction determination unit 23, an image generation unit 24, and an output unit 25. Of these, the functions of the remaining distance acquisition unit 21, the determination unit 22, the direction determination unit 23, and the image generation unit 24 are realized by the CPU 202 of FIG. 7 executing a predetermined program or the like. Further, the function of the output unit 25 is realized by the I / F 205 or the like shown in FIG.

Based on the route guidance information input from the car navigation system 11, the remaining distance acquisition unit 21 acquires the remaining distance information until the own vehicle 8 reaches the point where the presented image is presented, and outputs the acquisition result to the determination unit 22. Output.

The determination unit 22 determines whether or not the remaining distance is equal to or less than a predetermined presentation threshold value, and outputs the determination result to the direction determination unit 23.

When the remaining distance becomes equal to or less than the presentation threshold value, the direction determination unit 23 determines the planned travel direction on the planned travel route of the own vehicle 8 based on the route guidance information input from the car navigation system 11, and images the determination result. Output to the generation unit 24. For example, when there is an intersection on the planned travel route of the own vehicle 8, the direction in which the own vehicle 8 should travel is determined at the intersection.

The image generation unit 24 generates a presentation image for presenting the planned travel direction on the planned travel route of the own vehicle 8 according to the planned travel direction determined by the direction determination unit 23. Then, the image data of the generated presentation image is output to the projection unit 10 via the output unit 25.

The projection unit 10 (see FIG. 1) projects a projected image including the presented image on the front windshield 71 based on the input presented image data, and makes the driver V of the own vehicle 8 visually recognize the virtual image of the presented image.

The image generation unit 24 displays not only the presented image but also the remaining distance image that displays the remaining distance information acquired by the remaining distance acquisition unit 21 and the vehicle speed information of the own vehicle 8 detected by the vehicle speed sensor 161 of FIG. It is also possible to generate other images such as a vehicle speed image. Then, the projected image data in which the presented image, the remaining distance image, the vehicle speed image, and the like are arranged in one image can be output to the projection unit 10 via the output unit 25.

<Processing example by control unit 20>
Next, the flow of processing by the control unit 20 will be described with reference to FIG. FIG. 9 is a flowchart showing an example of processing by the control unit 20.

First, in step S91, the remaining distance acquisition unit 21 and the direction determination unit 23 input route guidance information from the car navigation system 11.

Subsequently, in step S92, the remaining distance acquisition unit 21 acquires the remaining distance information until the own vehicle 8 reaches the point where the presented image is presented based on the route guidance information, and the acquisition result is transmitted to the determination unit 22. Output.

Subsequently, in step S93, the determination unit 22 determines whether or not the remaining distance is equal to or less than the presentation threshold value.

If it is determined in step S93 that the remaining distance is not equal to or less than the presentation threshold value (steps S93, No), the processes after step S91 are repeated again.

On the other hand, when it is determined in step S93 that the remaining distance is equal to or less than the presentation threshold value (step S93, Yes), in step S94, the direction determining unit 23 determines the planned traveling direction of the own vehicle 8 based on the route guidance information. To determine.

Subsequently, in step S95, the image generation unit 24 generates a presentation image according to the planned traveling direction determined by the direction determination unit 23.

Subsequently, in step S96, the output unit 25 outputs the presented image data to the projection unit 10.

In this way, the control unit 20 can generate the presentation image and output it to the projection unit 10. During steps S91 to S94, the image generation unit 24 generates the above-mentioned residual distance image, vehicle speed image, and the like, and outputs the projected image data including these to the projection unit 10 via the output unit 25. You can also do it. In this case, the projected image including the remaining distance image, the vehicle speed image, and the like without including the presented image is superimposed and displayed on the foreground of the own vehicle 8.

(Various display examples)
Next, various display examples by the display device 1 will be described. In the following, a case where an image is superimposed and displayed on the foreground of the own vehicle 8 in a state where the destination is set using the car navigation system 11 of FIG. 3 will be described as an example. Further, in a general car navigation system, the planned travel route of a vehicle is guided at each intersection or a branch point where a road branches. Therefore, a case where the presented image is displayed at each intersection or branch point corresponding to this and the planned travel route is guided will be described as an example.

First, FIG. 10 is a diagram showing an example of display on the foreground 300 of the own vehicle 8 when the remaining distance is not equal to or less than the presentation threshold value.

The foreground 300 includes a preceding vehicle 301, a road 302, and the like. Further, in the foreground 300, the vehicle speed image 303 and the remaining distance image 304 are superimposed and displayed.

The vehicle speed image 303 is an image displaying the current vehicle speed of the own vehicle 8. In the example of FIG. 10, 70 (km / h) is displayed as the vehicle speed.

The remaining distance image 304 is an image that displays the remaining distance until the own vehicle 8 reaches the point where the presented image is presented. In the example of FIG. 10, "Tsuki I.C." is displayed as a point for presenting the presented image, and 200 mm, which is the remaining distance to "Tsuki I.C.", is displayed.

The remaining distance image 304 is displayed when the remaining distance becomes equal to or less than a predetermined display threshold value (for example, 500 m), and is displayed until the remaining distance becomes equal to or less than the presentation threshold value (for example, 100 m). In FIG. 10, since the remaining distance is not equal to or less than the presentation threshold value, the presentation image is not yet displayed.

Next, FIG. 11 is a diagram showing an example of display of the own vehicle 8 on the foreground 300a when the remaining distance is equal to or less than the presentation threshold value.

In the foreground 300a, a crossing road 305 that intersects the road 302 appears. Further, since the remaining distance is equal to or less than the presentation threshold value, the remaining distance image 304 in FIG. 10 is hidden and the presented image 306 is newly displayed in the foreground 300a.

The presented image 306 is an image that presents the planned traveling direction of the own vehicle 8 so as to guide (guide) the intersection road 305. As shown in FIG. 11, the presented image 306 is configured to include two arrow figures 306a and 306b.

The arrow figures 306a and 306b are oriented to indicate the crossroads 305. The directions indicated by the arrow figures 306a and 306b correspond to the planned traveling directions of the own vehicle 8 (the planned moving directions of the moving body).

Further, each of the arrow figures 306a and 306b is a figure having an image size smaller than the size corresponding to the projection angle of view of the projection unit 10 in the Z direction, and is displayed intermittently from the positive Z direction to the negative Z direction. ing.

Here, the Z direction indicated by the arrow in FIG. 11 corresponds to the direction along the vertical direction in the foreground 300a, and the X direction corresponds to the direction along the horizontal direction in the foreground 300a. This point is the same in the X direction and the Z direction shown in FIGS. 12 to 15 below. The relationship between the size of the projected image and the projected angle of view will be described later with reference to FIG.

The driver V of the own vehicle 8 can easily recognize the planned travel direction on the planned travel route by visually recognizing the presentation image 306 superimposed and displayed on the foreground 300a.

The arrow figures 306a and 306b are examples of "plurality of figures". In the example of FIG. 11, two arrows are shown, but the present invention is not limited to this, and the number of arrows may be further increased. Further, the figure is not limited to the arrow, and may be a figure of any shape as long as it indicates a direction.

Further, in FIG. 11, the area 307 shown by the broken line indicates the area where the projected image is displayed. The projected image is displayed in the area of the area 307 in the foreground 300a. Since the presented image 306 is an image included in the projected image, it is displayed in this area 307. The area 307 is shown for explaining the area where the projected image is displayed, and is not displayed in the foreground 300a of the own vehicle 8.

Next, FIG. 12 is a diagram for explaining the relationship between the size of the projected image and the presented image. (A) is a diagram showing the relationship between the size of the projected image according to the comparative example and the presented image, and (b) is a diagram showing the relationship between the size of the projected image and the presented image according to the embodiment. Further, FIG. 12 corresponds to a view of the superimposed display of the projected image on the foreground in FIGS. 10 and 11 as viewed from above the own vehicle 8 (direction looking down from the sky).

In FIG. 12A, a presentation image 306'of a thick arrow extending along the route is displayed in order to present the planned travel route of the own vehicle 8 traveling on the road 302. In this case, since it is necessary to extend the thick arrow along the planned travel route, the projected image including the presentation image 306'is required to have a size capable of presenting the superposition range 308'. Here, the superimposition range is a range determined by the projection angle of view and the depression angle (looking down angle) of the projection unit 10, and refers to a range in which the foreground and the projected image are superposed.

On the other hand, as shown in FIG. 12B, since the presentation image 306 according to the embodiment presents the planned traveling direction, it is not necessary to present the presentation image 306 so as to show the entire route. Therefore, it is possible to generate a projected image with a size capable of presenting a superimposition range 308, which is smaller than the superimposition range 308'in FIG. 12A.

Next, FIG. 13 is a diagram illustrating an example of the relationship between the size of the projected image and the projected angle of view by the projection unit 10. In FIG. 13, virtual image I'shows a virtual image according to a comparative example. The virtual image I'is a virtual image of the projected image that is visually recognized by the driver V by projecting the projected image at a projection angle of view φ'. On the other hand, the virtual image I shows a virtual image according to the embodiment. The virtual image I is a virtual image of the projected image that is visually recognized by the driver V by projecting the projected image at a projection angle of view φ.

The size h of the virtual image I of the projected image including the presented image 306 according to the embodiment in the Z direction is compared with the size h'of the virtual image I'including the presented image 306'according to the comparative example in the Z direction. Is small. Therefore, it is possible to present the presentation image 306 at a projection angle of view φ that is narrower than the projection angle of view φ'and to present the route guidance information.

Here, the projection angle of view φ corresponds to the maximum projection angle of view of the projection unit 10 in the Z direction along the vertical direction.

Next, FIG. 14 is a diagram illustrating an example of the relationship between the display intervals of the arrow figures 306a and 306b in the presented image 306 and the superimposition range. In FIG. 14, the display interval d indicates the display interval of the arrow figures 306a and 306b, and the superimposition distance e is the length (distance) of the superimposition range in the traveling direction of the own vehicle 8 (direction orthogonal to both the X direction and the Y direction). ) Is shown. As shown in FIG. 14, the display interval d is preferably smaller than the superposition distance e. Further, when the display interval d is set to be equal to or less than the maximum projection angle of view of the projection unit 10 in the Z direction, a plurality of figures such as arrow figures 306a and 306b can be displayed in the foreground, and the planned traveling direction can be presented more clearly with the plurality of figures. Therefore, it is suitable.

<Action and effect of display device 1>
As described above, in the embodiment, a presented image presenting the planned traveling direction of the own vehicle 8 is generated as route guidance information, and a projected image including the presented image is projected by the projection unit 10 provided in the own vehicle 8. To display. The presentation image 306 with a narrow projection angle of view φ (see FIG. 13) in order to project a small image showing the planned movement direction of the own vehicle 8 without projecting a large image for presenting the planned travel route of the own vehicle 8. Can be presented to present route guidance information. As a result, it is possible to avoid an increase in the size and cost of the display device 1.

Further, the presented image 306 according to the embodiment includes arrow figures 306a and 306b, which are a plurality of figures having an image size smaller than the size corresponding to the projection angle of view of the projection unit 10. Further, the arrow figures 306a and 306b are displayed intermittently from the positive Z direction to the negative Z direction. By doing so, the area of the presented image 306 superimposed on the road in the foreground can be reduced as compared with the case where the presented image of the thick arrow extending along the planned travel route is superimposed on the road in the foreground. .. As a result, the visual annoyance (complexity) in the foreground can be eliminated.

Here, the planned traveling direction can also be presented by the display positions of the arrow figures 306a and 306b in the presented image 306. FIG. 15 is a diagram illustrating an example of display positions of arrow figures 306a and 306b in the presented image 306.

It is simplest to display the arrow figures 306a and 306b on the central axis 309 of the driver V, but the reference position 310 of the intersection is determined and the arrow figures 306a and 306b are linearly connected to the reference position 310 and the driver V. It may be displayed so that 306b is arranged.

Such a display state will be described by replacing it with the case where the foreground 300a of the own vehicle 8 is visually recognized. The virtual image of the presented image 306 is a plurality of figures arranged in a direction along the vertical direction in the foreground 300a, and presents the planned traveling direction. As for the plurality of figures, the figure that presents the planned travel direction at a position closer to the intersection in the vertical direction in the foreground presents the planned travel direction at a position closer to the intersection in the horizontal direction in the foreground.

More specifically with reference to FIG. 11, among the arrow figures 306a and 306b constituting the presented image 306, the vehicle is scheduled to travel at a position close to the crossing road 305 in the direction along the vertical direction (Z direction) in the foreground 300a. The arrow figure 306a presenting the direction presents the planned traveling direction at a position closer to the crossing road 305 in the direction along the horizontal direction (X direction) in the foreground 300a as compared with the arrow figure 306b.

By doing so, the driver V can recognize the planned traveling direction not only by the directions of the arrow figures 306a and 306b but also by the display positions of the arrow figures 306a and 306b.

The remaining distance image 304 in the above-described embodiment is presented to present route guidance information in order to present information on a point at which the presented image is presented and information on the remaining distance until the own vehicle 8 reaches this point. It can also be called an image. However, since the remaining distance image 304 is character information and does not require a wide projection angle of view for display, the influence on the projection angle of view is small.

<Other preferred examples>
Although the best mode for carrying out the present invention has been described above with reference to examples, the present invention is not limited to these examples, and various modifications and modifications and variations and within a range not deviating from the gist of the present invention are made. Substitutions can be added.

In the above-described embodiment, the control unit 20 may have a part or all of the functions provided by the recognition device 15. Further, a part of the functions provided by the control unit 20 may be provided by other components such as the recognition device 15, the car navigation system 11, and the detection device 16.

The embodiment also includes a display method. For example, the display method is a display method using a display device that displays a projected image, and includes a step of generating a presentation image that presents route guidance information and a projection unit provided on the moving body that includes the presentation image. The step of projecting the image to be projected is performed, and the presented image presents the planned movement direction of the moving body. By such a display method, the same effect as that of the display device described above can be obtained.

The embodiment also includes a program. For example, the program is a program that operates on a display device that displays a projected image, generates a presented image that presents route guidance information, and is projected by a projection unit provided on the moving body that includes the presented image. A computer is made to execute a process of projecting an image, and the presented image presents a planned movement direction of the moving body. With such a program, the same effect as that of the display device described above can be obtained.

Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array) and conventional circuit modules.

1 Display device 2 In-vehicle system 8 Own vehicle (an example of a moving body)
10 Projection unit 11 Car navigation system 15 Recognition device 16 Detection device 164 Surrounding camera 20 Control unit 21 Remaining distance acquisition unit 22 Judgment unit 23 Direction determination unit 24 Image generation unit 25 Output unit 300, 300a Foreground 301 Leading vehicle 302 Road 303 Vehicle speed image 304 Remaining distance image 305 Crossing road 306 Presented images 306a, 306b Arrow figures (example of multiple figures)
307 Area 308 Superimposition range d Display interval e Superimposition distance h Virtual image size in Z direction I Virtual image V Driver X Direction along the horizontal direction in the foreground Z Direction along the vertical direction in the foreground φ Projection angle of view

JP-A-2019-207155

Claims (10)

A display device that displays the projected image.
An image generator that generates a presentation image that presents route guidance information,
A projection unit provided on the moving body and projecting the projected image including the presented image is provided.
The presented image is a display device that presents the planned movement direction of the moving body. The display device according to claim 1, wherein the presented image presents the planned movement direction with a plurality of figures. The display device according to claim 2, wherein the figure is an image of an arrow figure having a directionality indicating the planned movement direction. The display device according to any one of claims 1 to 3, wherein a virtual image of the presented image is superimposed and displayed on the foreground of the moving body. The display device according to claim 4, wherein the virtual image of the presented image presents the planned movement direction at an intersection in the foreground. The virtual image of the presented image presents the planned movement direction by a plurality of figures arranged in a direction along the vertical direction in the foreground.
The display device according to claim 5, wherein the distance between the plurality of figures is equal to or less than the maximum projection angle of view of the projection unit in a direction along the vertical direction. The virtual image of the presented image presents the planned movement direction by a plurality of figures arranged in a direction along the vertical direction in the foreground.
The plurality of figures present the planned movement direction at a position closer to the intersection in the direction along the vertical direction in the foreground, and the figure is closer to the intersection in the direction along the horizontal direction in the foreground. The display device according to claim 5 or 6, which presents the planned movement direction. A mobile body having the display device according to any one of claims 1 to 7. It is a display method using a display device that displays the projected image.
The process of generating a presentation image that presents route guidance information,
A step of projecting the projected image including the presented image by a projection unit provided on the moving body is performed.
The presented image is a display method for presenting a planned movement direction of the moving body. A program that runs on a display device that displays the projected image.
Generate a presentation image that presents route guidance information,
A computer is made to execute a process of projecting the projected image including the presented image by a projection unit provided on the moving body.
The presented image is a program that presents the planned movement direction of the moving body.

Images