JP2019172243A - Control device, display device, movable body, control method and program - Google Patents

Abstract

To provide a behavior of own vehicle so as to be visible at certain future timing.SOLUTION: A control device generates an image to be displayed so as to be visible while being superposed on a surrounding environment when being viewed from an occupant of a movable body. The control device generates image data including a symbol that indicates a location of the movable body at certain future timing based on at least any one of internal information and external information of the movable body.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a control device, a display device, a moving body, a control method, and a program.

  Development of a head-up display (HUD) mounted on a moving body such as a vehicle, a ship, an aircraft, or an industrial robot is being promoted. The HUD projects information directly on the human visual field and provides various information to the passengers of the moving body. In the HUD, the generated light image is diffracted in the direction of the occupant by a windshield, a combiner, or the like, and displayed as if an image exists at the virtual image position in front of the sight line of the occupant. The image is displayed superimposed on the surrounding environment at the virtual image position.

  2. Description of the Related Art A vehicle display device that displays a virtual preceding vehicle that does not actually exist in front of an occupant is known (see, for example, Patent Document 1). In this document, when a preceding vehicle is not present, a virtual preceding vehicle is displayed in front of the occupant, and the occupant can automatically change the traveling state of the virtual preceding vehicle based on the traveling state of the host vehicle and the road condition. It is possible to grasp the state of the vehicle and the information ahead in a sense of incongruity.

  In a known vehicle display device, a position that is separated by a reference inter-vehicle distance Lro on the traveling lane ahead of the host vehicle is set as the position of the virtual preceding vehicle. The reference inter-vehicle distance Lro is defined as a function of the speed of the host vehicle, and the behavior of the host vehicle after the current time is not taken into consideration. If the behavior of the host vehicle at a certain time in the future can be presented to the occupant so as to be visible, the occupant should be able to drive the host vehicle with a sense of security.

  An object of this invention is to provide the control technique which can show to the passenger | crew of a mobile body the behavior of the mobile body in the future at a certain time point so that visual recognition is possible.

In order to achieve the above object, a control device that generates data of an image that is displayed so as to be superimposed on the surrounding environment when viewed from a passenger of a moving object,
An image data generating unit that generates image data including a symbol indicating a position of the moving body at a predetermined future time based on at least one of internal information and external information of the moving body;
Have

  With the above configuration, the occupant can visually grasp the behavior of the moving body at a certain time in the future, and can operate the moving body with a sense of security.

It is a schematic diagram which shows the motor vehicle carrying HUD as an example of the mobile body carrying a display apparatus. It is a figure explaining rotation of the automobile around a predetermined axis. It is a figure which shows the example of arrangement | positioning of a projection area. It is a figure which shows the structural example of the display system by which a display apparatus is mounted. It is a hardware block diagram of the display apparatus of embodiment. It is a schematic diagram which shows the connection relation of a display apparatus and the other electronic device mounted in a moving body. It is a functional block diagram of the image control part of the display apparatus of embodiment. It is a figure which shows the example of the superimposed display of the symbol of the future own vehicle. It is a figure which shows the example of the superimposed display of the symbol of the future own vehicle. It is a figure explaining calculation of the display timing according to the amount of change of the state of the own vehicle. It is a figure explaining calculation of the display timing according to the amount of change of the state of the own vehicle. It is a figure explaining calculation of the display timing according to the amount of change of the state of the own vehicle. It is a figure which shows the example of acquisition of the variation | change_quantity of the state of the own vehicle. It is a flowchart of the display control method of an embodiment.

  In the embodiment, with a display device mounted on a moving body such as a vehicle, a traveling image of a future moving body (own vehicle) after the current time is displayed superimposed on a real environment such as a road ahead.

  FIG. 1 schematically shows an automobile 300 as an example of a moving body on which the display device 1 is mounted. In this example, the display device 1 is an in-vehicle head-up display (hereinafter abbreviated as “HUD”). The moving body on which the display device 1 is mounted is not limited to the automobile 300, and the display device 1 can be mounted on a moving body such as a vehicle, a ship, an aircraft, and an industrial robot. The automobile 300 has a function of adaptive cruise control (ACC: semi-automatic running) and is assumed to be able to run by switching between semi-automatic driving and manual driving. However, the present invention has an ACC function. It is also applicable to vehicles that do not.

  The display device 1 is installed, for example, on a dashboard of the automobile 300 or in a dashboard, and a predetermined projection area 311 of a windshield 310 in front of a passenger or an operator (hereinafter referred to as “occupant P”) A light image is projected.

  The display device 1 includes an optical device 10 and a control device 20. The control device 20 mainly controls generation and display of an image projected on the windshield 310. The optical device 10 projects the generated image onto the projection area 311 of the windshield 310. Since the configuration of the optical device 10 is not directly related to the present invention, a detailed configuration is not shown. For example, as will be described later, the laser light output from the laser light source is applied to a screen provided between the projection area 311 and the light source. Two-dimensional scanning is performed to form an intermediate image, and this intermediate image is projected onto the projection area 311. The screen can be composed of a microlens array, a micromirror array, or the like.

  The projection area 311 of the windshield 310 is formed of a transmission / reflection member that reflects a part of the light component and transmits the other part. The intermediate image formed by the optical device 10 is reflected by the projection area 311 and travels toward the passenger P. When the reflected light enters the occupant P's pupil through an optical path indicated by a broken line, the occupant P visually recognizes the image projected on the projection area 311 of the windshield 310. At this time, the occupant P feels as if the optical image is incident on the pupil from the virtual image position I through the dotted optical path. The displayed image is recognized as if it exists at the virtual image position I.

  The virtual image at the virtual image position I is displayed superimposed on the real environment in front of the automobile 300, for example, on the traveling road. In this sense, an image formed at the virtual image position I may be referred to as an AR (Augmented Reality) image.

  FIG. 1B shows the pitching angle, yaw angle, and roll angle of the automobile 300. Rotating (or tilting) an object with a fixed front / rear, left / right, and up / down movement, such as a moving object, with respect to the front / rear axis (Z-axis in the figure) and rotating with respect to the left / right axis (X-axis in the figure) (Or tilting) is called pitching, and rotating (or tilting) with respect to the vertical axis (Y-axis in the figure) is called yawing. Each rotation amount or inclination amount is referred to as a roll angle, a pitching angle, or a yaw angle.

  FIG. 2 is a diagram illustrating an arrangement example of the projection area 311. The projection area 311 is a relatively small area that is disposed slightly below the front position of the windshield 310 when viewed from the driver's seat, for example. A line segment connecting the viewpoint of the occupant P and the virtual image position I is included in the range of the projection area 311, and an image enlarged at the virtual image position I is visually recognized.

  The projection area 311 is not the same as a display area on which an image to be described later is superimposed and displayed. The projection area 311 is a surface on which an optical image formed by laser light is projected, whereas the display area is outside the projection area 311 and within the field of view of the occupant P. A certain area including the virtual image position I to be imaged. The display area is set, for example, at a position about a dozen meters away from the viewpoint of the passenger P.

  The automobile 300 may be equipped with a camera 5 that acquires information about the environment around the automobile 300 and a detector 5 such as LiDAR (Light Detection and Raging). The detector 5 captures an image of the external environment such as the front or side of the automobile 300, for example. The detector 5 is an example of a sensor for acquiring external information, and an ultrasonic radar, a laser radar, or the like may be used instead of the camera or in combination with the camera.

  FIG. 3A shows a configuration example of a display system in which the display device 1 is mounted. The display system 150 includes a vehicle navigation device 400, a steering angle sensor 152, a display device 1, and a vehicle speed sensor 154 that are connected to each other via an in-vehicle network NW such as a CAN (Controller Area Network) bus.

  The vehicle navigation apparatus 400 has a GNSS (Global Navigation Satellite System) represented by GPS, detects the current location of the vehicle, and displays the position of the vehicle on an electronic map. It also accepts input of departure and destination, searches for the route from the departure point to the destination and displays the route on an electronic map, or voices and displays the direction of travel to the driver before changing the route (display Or displayed by animation or the like. Note that the vehicle navigation apparatus 400 may communicate with a server via a mobile phone network or the like. In this case, the server can transmit an electronic map to the automobile 300, perform a route search, and the like.

  The steering angle sensor 152 is a sensor that detects the steering angle of the steering wheel by the driver. It mainly detects the direction and amount of steering. For example, there is one that counts ON / OFF of light passing through a slit disk that rotates in conjunction with a steering wheel.

  The vehicle speed sensor 154 detects, for example, the rotation of the wheel with a hall element and outputs a pulse wave corresponding to the rotation speed. The vehicle speed is detected from the rotation amount (number of pulses) per unit time and the outer diameter of the tire.

  The display device 1 can acquire information from each sensor mounted on the vehicle. The display device 1 may acquire information from an external network instead of the in-vehicle network. For example, car navigation information, steering angle, or vehicle speed can be acquired. With regard to the steering angle and vehicle speed, it is considered that in-vehicle devices can be controlled by observing the position and orientation of the traveling vehicle and the vehicle speed by ITS (Intelligent Transport Systems) when automatic driving is put into practical use in the future.

  FIG. 3B is a hardware configuration example of the display device 1 according to the embodiment. The optical device 10 of the display device 1 includes a laser diode (LD) 101 as a light source and a MEMS (Micro Electro Mechanical System) 102 as an optical scanning device. LD 101 includes, for example, a laser element that outputs red (R), green (G), and blue (B) light. The MEMS 102 draws an optical image by two-dimensionally scanning the laser beam output from the LD 101 on a screen disposed between the LD 101 and the projection area 311. As the optical scanning device, a polygon mirror, a carbano mirror, or the like may be used in addition to MEMS.

  The control device 20 includes an FPGA (Field-Programmable Gate Array) 201, a CPU (Central Processing Unit) 202, a ROM (Read Only Memory) 203, a RAM (Random Access Memory) 204, and an interface (hereinafter referred to as “I / F”). 205, a bus line 206, an LD driver 207, a MEMS controller 208, and an SSD (Solid State Drive) 209 as an auxiliary storage device. Further, the recording medium 211 may be detachably arranged.

  The FPGA 201 controls the operation of the LD driver 207 and the MEMS controller 208. The LD driver 207 generates and outputs a drive signal for driving the LD 101 under the control of the FPGA 201. The light emission timing of each laser element that emits R, G, B light is controlled by the drive signal. The MEMS controller 208 generates and outputs a MEMS control signal under the control of the FPGA 201, and controls the scanning angle and scanning timing of the MEMS 102. Instead of the FPGA 201, other logic devices such as PLG (Programmable Logic Device) may be used.

  The CPU 202 controls the overall image data processing of the display device 1. The ROM 203 stores various programs including a program executed by the CPU 202 to control each function of the display device 1. The ROM 203 may store various image objects used for the superimposed display of route images. The RAM 204 is used as a work area for the CPU 202.

  The I / F 205 is an interface for communicating with an external controller or the like, and is connected to, for example, the detector 5, the vehicle navigation device, various sensor devices, and the like via the CAN bus of the automobile 300.

  The display device 1 can read and write to the recording medium 211 via the I / F 205. An image processing program for realizing processing in the display device 1 may be provided by the recording medium 211. In this case, the image processing program is installed from the recording medium 211 to the SSD 209 via the I / F 205. Installation of the image processing program is not necessarily performed by the recording medium 211, and may be downloaded from another computer via a network. The SSD 209 stores the installed image processing program and also stores necessary files and data.

  Examples of the recording medium 211 include portable recording media such as a flexible disk, a CD-ROM, a DVD disk, an SD memory card, and a USB (Universal Serial Bus) memory. Further, as an auxiliary storage device, an HDD (Hard Disk Drive), a flash memory, or the like may be used instead of the SSD 209. The auxiliary storage device such as the SDD 209 and the recording medium 211 are both computer-readable recording media.

  FIG. 4 is a schematic diagram illustrating a connection relationship between the display device 1 according to the embodiment and other electronic devices mounted on the automobile 300. The display device 1 includes an optical unit 230 and an image control unit 250. The optical unit 230 roughly corresponds to the optical device 10, but the FPGA 201, the LD driver 207, and the MEMS controller 208 may be included in the optical unit 230. The image control unit 250 is realized by at least a part of the control device 20.

  The display device 1 is connected to electronic devices such as an ECU (Electronic Control Unit) 600, a vehicle navigation device 400, and a sensor group 500 via an I / F 205 and a CAN. The sensor group 500 includes the steering angle sensor 152 and the vehicle speed sensor 154 of the system of FIG. 3A. When the detector 5 is installed in the automobile 300, the detector 5 may also be connected to the display device 1 via the I / F 205.

  The display device 1 acquires external information from the vehicle navigation device 400, the sensor group 500, the detector 5, and the like, and detects the presence of an intersection, a curve, an obstacle, etc. ahead of the route on which the host vehicle travels.

  The sensor group 500 includes an accelerator amount sensor, a brake amount sensor, a steering wheel angle sensor, a tire angle sensor, an acceleration sensor, a gyro sensor (or a yaw rate sensor), a vehicle speed sensor, a laser radar device, a brightness sensor, and raindrops. It includes sensors and the like, and detects the behavior of the automobile 300, the surrounding environment, the distance to the traveling vehicle ahead, and the like.

  The vehicle navigation apparatus 400 has navigation information including a road map, GPS information, traffic regulation information, construction information for each road, and the like.

  Information acquired by the vehicle navigation device 400, the sensor group 500, and the detector 5 is supplied to the image control unit 250, and at least a part of the acquired information is used to generate image data including a symbol of a future host vehicle. Used.

  FIG. 5 is a functional block diagram of the image control unit 250. The image control unit 250 includes an information input unit 800, an information analysis unit 810, a display timing acquisition unit 820, an image data generation unit 830, and an image drawing unit 840.

  The information input unit 800 is realized by, for example, the I / F 205, and inputs information from the vehicle navigation device 400, the sensor group 500, the ECU 600, the detector 5, and the like. The information input unit 800 includes an internal information input unit 8001 and an external information input unit 8002. The internal information is information representing the state of the automobile 300 itself. The internal information input unit 8001 acquires the current position, speed, angular velocity information (yaw, roll, pitch) of the automobile 300 from the sensor group 500 and the ECU 600 via CAN or the like. The yaw indicates the left and right rotation of the automobile, and may be calculated from the steering angle or may be obtained from a three-axis sensor. The roll indicates the left / right inclination of the automobile 300, and the pitch indicates the front / rear inclination of the automobile 300.

  The external information is information indicating an external situation of the automobile 300 other than the internal information. The external information input unit 8002 acquires navigation information, map information, and the like from the vehicle navigation device 400. Moreover, you may acquire imaging information from the detector 5. FIG.

  The information analysis unit 810 includes a road condition detection unit 8110 and a vehicle change amount calculation unit 8120. The road condition detection unit 8110 detects road conditions such as obstacles, intersections, and curves based on the acquired external information. The vehicle change amount calculation unit 8120 calculates a change amount of a state such as the position and orientation of the host vehicle based on the acquired internal information. The road condition (or running condition) and the amount of change in the condition of the host vehicle may be combined and called “movement situation”. The information analysis unit 810 analyzes the movement status of the host vehicle based on the external information and internal information acquired by the information input unit 800.

  The display timing acquisition unit 820 is based on the internal information and external information of the own vehicle acquired by the information input unit 800 and the analysis information obtained by the information analysis unit 810, and the display timing acquisition unit 820 Timing information indicating whether to display an image in the future is acquired. The calculation of the timing of how far the future image of the host vehicle is to be generated may be performed by the display timing acquisition unit 820 or by a computer outside the image control unit 250.

  The image data generation unit 830 generates image data including the symbol of the host vehicle at a certain future time (or position) based on the movement situation obtained by the information analysis unit 810. The “movement situation” is at least one of a road situation detected by the road situation detection unit 8110 and a change amount of the state of the host vehicle obtained by the vehicle change amount calculation unit 8120. The road conditions include the presence of obstacles on the planned route, right / left turn roads, branches, presence / absence of intersections, and the like. The amount of change in the state of the host vehicle includes the amount of change in position and orientation, the amount of change in speed (acceleration / deceleration), and the like.

  The image data generation unit 830 reads out an object of the host vehicle stored in, for example, the ROM 204 as a symbol of the host vehicle, processes the object by a three-dimensional computer graphics technique, and generates a three-dimensional image of the host vehicle at a certain time in the future. Data may be generated. The image data generation unit may use an image stored in advance as a symbol of the host vehicle. In this case, images of a plurality of angles corresponding to the symbol of the host vehicle may be stored and read out and used. The symbol of the host vehicle may be generated from an image obtained by photographing an actual vehicle, or may be generated from a CAD image. Further, the symbol of the host vehicle may be displayed as an image such as an icon.

  The image drawing unit 840 includes a control unit 8410 and controls the image projection operation by the optical device 10 based on the image data generated by the image data generation unit 830. The image drawing unit 840 can be realized by the FPGA 201, the LD driver 207, and the MEMS controller 208. The image drawing unit 840 draws a future optical image of the host vehicle, and the optical image is projected onto the projection area 311 of the windshield 310. Thereby, the virtual image of the future own vehicle is superimposed and displayed on the display area including the virtual image position I.

<Example of superimposed display of future vehicle>
FIG. 6 shows an example of superimposed display of the symbol 11 of the future host vehicle. The symbol 11 of the future own vehicle is displayed as a virtual image so as to be superimposed on a real environment, for example, a road 12 scheduled to travel ahead in a predetermined display area 13. FIG. 6A is an image showing a future traveling state of the host vehicle when the host vehicle is traveling stably. FIG. 6B is an image showing the traveling state of the host vehicle at a future time when the traveling mode of the host vehicle changes greatly.

  In FIG. 6A, when the amount of change in the state of the host vehicle is small (eg, less than a predetermined threshold), or when no obstacle or right / left turn road is detected on the road 12, A symbol 11 of a future host vehicle traveling at a distant time or position is displayed in the display area 13 as an image obtained by projecting a 3D image in two dimensions, for example.

  In FIG. 6B, when the amount of change in the state of the host vehicle is large, or when an obstacle or a right / left turn road is detected on the road 12, the vehicle is traveling in the near future or a relatively close position. A symbol 11 of the future host vehicle is displayed in the display area 13 as an image obtained by projecting a 3D image in two dimensions, for example.

  The case where the amount of change of the host vehicle is large is when the change of the vehicle speed is large (when the acceleration or deceleration rate is large), or when the amount of any of yaw, pitch, or roll is large. For example, when approaching a curve, the yaw rate and rolling amount increase. The pitch increases when approaching downhill or uphill slopes. When trying to avoid an obstacle, the yaw rate increases but the speed decreases. When trying to change lanes, the yaw rate increases.

  The image displayed in the display area 13 is displayed in synchronism with the actual environment (in this example, the road 12 ahead), scale, and perspective when viewed from the viewpoint of the passenger. The virtual image of the symbol 11 of the host vehicle in the relatively far future in FIG. 6A is displayed so as to appear small in the distance, and the virtual image of the symbol 11 of the host vehicle in the near future in FIG. , It is displayed so that it looks larger on the front side.

  Thereby, the passenger | crew can grasp | ascertain intuitively the change of a driving | running | working condition, for example, can grasp | ascertain correctly the timing which switches from semiautomatic driving mode to manual driving | operation. Even in the case of the automobile 300 that does not have the ACC function, since the occupant can grasp in advance the change in the driving situation in the host vehicle, it is possible to avoid delays in the steering wheel operation and the accelerator / brake operation.

  When the symbol 11 of the host vehicle in the near future occupies a wide part of the display area 13, the transparency of the symbol 11 of the host vehicle in the future may be changed so as not to disturb the sight of the passenger.

  FIG. 7 shows another example of the superimposed display of the symbol 11 of the future host vehicle. Since the symbol 11R of the host vehicle in the relatively far future of FIG. 7A occupies a small area in the display area 13, it is displayed in a dark color or low transparency. Since the virtual image of the symbol 11N of the host vehicle in the near future in FIG. 7B occupies a large proportion in the display area 13, the symbol 11N of the host vehicle in the future has a light color or high transparency ( Displayed in a translucent state. By increasing the transparency of the driving image of the symbol of the vehicle in the near future, the occupant can visually recognize the actual background (road condition).

  Further, the color of the symbol 11 of the future host vehicle in FIG. 7 may be changed based on the difference between the current speed of the host vehicle and the speed of the host vehicle at a certain time in the future. For example, when the speed of the future own vehicle displayed is slower than the current speed of the own vehicle, the symbol 11R of the future own vehicle is displayed in blue in FIG. When the speed of the own vehicle is high, the symbol 11R of the future own vehicle is displayed in red. In this case, the occupant can intuitively grasp whether the brake operation will be performed on the host vehicle or the accelerator operation will be performed.

  Further, the symbol 11R or 11N of the future host vehicle displayed in the display area 13 may be changed to be lighter or smaller at a predetermined timing. For example, when other contents (for example, messages such as “Foot traffic ahead, attention!”, “Far ahead, there is an accident vehicle!”, Etc.) are displayed in the display area 13 in an emergency, the symbol 11R or 11N of the future own vehicle is not conspicuous. By changing the display like this, it is possible to call attention to the emergency message.

  FIG. 8A is a diagram illustrating a calculation example of display timing (how many seconds later the own vehicle is displayed) according to the change amount of the own vehicle. In FIG. 8A, the horizontal axis represents time, and the vertical axis represents the change amount D of the host vehicle. Time t = 0 represents the present time, and time t1 is a future time when the amount of change D exceeds the threshold Th1.

The amount of change D is, for example,
D = α × S (t) + β × V (t) (1)
It is represented by Here, α and β are weighting coefficients (α + β = 1), S is a change value based on the steering angle of the steering at a certain time point, and V is a change value based on the vehicle speed at a certain time point. S and V are both estimated values at a certain time. This is an estimated value predicted based on the current values of the steering angle and vehicle speed of the steering wheel and the history up to now.

  The change value for obtaining the change amount D is not limited to the steering angle of the steering and the vehicle speed. Further, the function is not limited to a function in which the steering angle and the vehicle speed are integrated, and a plurality of functions using these parameters independently may be used.

  FIG. 8A is a graph in which the amount of change D calculated based on Equation (1) is plotted for each fixed time t after the current time. The predicted amount of change D gradually increases from the current time, exceeds a threshold Th1 at a certain point, reaches a peak thereafter, gradually decreases after the peak, and becomes smaller than the threshold Th. In this example, the symbol of the host vehicle at the time point t1 when the predicted change amount D first exceeds the threshold Th1 is generated.

  FIG. 8B is a diagram illustrating a case where the timing exceeding the threshold Th1 is close to the current time point. If the amount of change has already changed drastically and the timing at which the threshold value is exceeded substantially coincides with the current time, the vehicle's symbol is generated and displayed at time t1 ′ after Δt from the current time to alert the passenger. be able to.

On the other hand, if the threshold Th1 is not exceeded at any point on the time axis, the symbol 11 of the future host vehicle at the time t _lim (see FIG. 8A) corresponding to the limit point of the predetermined display timing is displayed in the display area 13. May be.

  FIG. 9 is a diagram showing another example of calculation of display timing (how many seconds later the own vehicle is displayed) according to the change amount of the own vehicle. FIG. 9 is a graph in which the change amount D calculated based on the equation (1) is plotted for each predetermined time t after the current time. The predicted amount of change gradually increases from the current time, reaches a peak at a certain point, and then gradually decreases.

Area of the hatched region which is calculated as the integral value S _in the amount of change D up to a certain time from the current time exceeds the threshold value Th2 of the (S _in> Th2) time t2 time is calculated as a display timing. If the integral value S _in does not exceed the threshold Th2 at any point on the time axis (that is, when the change amount D is small), the symbol 11 of the future _host vehicle at the time t _lim corresponding to the limit point of the predetermined display timing. Is displayed in the display area 13.

  In both FIG. 8 and FIG. 9, when an obstacle such as a front vehicle interruption or a pedestrian is detected, the change amount D as a function of time is recalculated in real time each time, so that the display area 13 The display of the symbol 11 of the future vehicle displayed on the screen also changes.

  FIG. 10 is a diagram illustrating an example of acquiring the amount of change in the state of the host vehicle. 8 and 9, the change amount D1 between the current time point t0 and a future time point t3 is acquired. However, as shown in FIG. 10, the change amount D2 between different future time points t3 and t4 is acquired. May be. Alternatively, both the change amount D1 from the current time t0 to the first future time point t3 and the change amount D3 from the current time t0 to the second future time point t4 may be used. By considering a plurality of future timings, the estimated accuracy of the predicted traveling state of the host vehicle is improved.

  FIG. 11 is a flowchart of display control performed in the display device 1. This control flow is performed by the image control unit 250 of the display device 1.

  The image control unit 250 acquires at least one of internal information and external information of the host vehicle (S11). The internal information includes speed information acquired from the sensor group 500 and the ECU 600, steering steering angle information (yaw, roll, pitch), tire angle information, position information estimated by the host vehicle, and the like. The external information includes vehicle navigation device 400, detector 5, sensor group 500 (laser radar, etc.), map information acquired from GPS, imaging information, surrounding environment information, distance measurement information, and the like.

  The image control unit 250 calculates the future timing (or position) of the host vehicle to be displayed in the current display area 13 based on the acquired information (S12). As shown in FIG. 8, the future timing (time point) is calculated as shown in FIG. 9 at a time point after a predetermined time Δt from the time point when the amount of change in the state of the vehicle exceeds a predetermined threshold value Th1. As described above, it is the time when the integrated value of the amount of change exceeds a predetermined threshold Th2.

  The display device 1 also determines from the acquired external information whether an obstacle has been detected on the route planned for traveling of the host vehicle (S13). If an obstacle such as a pedestrian, an interrupting vehicle, or road construction is detected (YES in S13), the process returns to step S11 to recalculate the timing or position of the future host vehicle (S12). If no obstacle is detected on the planned travel route (NO in S13), image data including a symbol of the future host vehicle is generated (S14).

  The generated image data is output, a laser image is scanned by the optical device 10 to draw an optical image, and a virtual image of the future host vehicle is displayed in the display rear (S15). The drawing of the light image is not limited to the laser scanning method, and any projection means capable of forming a light image, such as a panel method as described later, may be used.

  Steps S11 to S15 are repeated until the display control ends (NO in S22). When the vehicle finishes running (when the engine is turned off) or when an instruction to turn off the display control is input, the display control is finished (YES in S18), and the treatment is finished.

When this display control is executed by a program, the display control program may be stored in the ROM 203 or the SSD 209, and the program may be read and executed by the CPU 202. In this case, when the CPU 202 generates image data of an image that is displayed so as to be superimposed on the surrounding environment when viewed from the occupant of the moving body, at least,
(a) a procedure for generating image data including a symbol indicating a position of the moving body at a predetermined future time point based on at least one of internal information and external information of the automobile 300;
Execute.

  By using the configuration and method described above, an image of a future host vehicle after the current time point is displayed superimposed on the actual environment, so that the passenger can intuitively grasp the operation of the host vehicle. Even when the operation of the vehicle changes greatly, it can be predicted in advance.

  The present invention is not limited to the embodiment described above. For example, instead of obtaining the amount of change of the host vehicle with a polynomial of the steering angle S and the vehicle speed V, the (S, X, Y) polynomial of the steering angle S, the accelerator amount X, and the brake amount Y of the steering is calculated. May be. Further, in FIG. 8 and FIG. 9, instead of calculating the change amount D as a function of time, the change amount D is calculated as a function of the position of the future host vehicle, and the position of the future host vehicle virtually displayed. May be determined.

  As the optical device 10, a panel method may be adopted instead of the laser scanning method. As the panel method, an imaging device such as a liquid crystal panel DMD (Digital Mirror Device) panel or a dot fluorescent display tube (VFD) may be used.

  A combiner formed of a half mirror, a hologram, or the like may be provided in the projection area 311 of the windshield 310. A light transmissive / reflective reflective film may be deposited on the surface of the windshield 310 or between the layers.

  You may implement | achieve at least one part of each function of the display apparatus 1 by the cloud computing comprised by one or more computers.

DESCRIPTION OF SYMBOLS 1 Display apparatus 5 Detector 10 Optical apparatus 11 Symbol 13 Display area 20 Control apparatus 250 Image control part (control apparatus)
300 automobile (mobile)
310 windshield 311 projection area 400 vehicle navigation device 500 sensor group 600 ECU
800 Information input unit 810 Information analysis unit 820 Display timing acquisition unit 830 Image data generation unit 840 Image drawing unit

JP 2002-144913

Claims (12)

A control device that generates image data of an image that is displayed so as to be superimposed on the surrounding environment when viewed from a passenger of a moving body,
An image data generating unit that generates image data including a symbol indicating a position of the moving body at a predetermined future time point based on at least one of internal information and external information of the moving body;
Control device.   The control apparatus according to claim 1, wherein the image data generation unit determines the future predetermined time based on a movement state of the moving body indicated by the internal information and the external information.   The control apparatus according to claim 2, wherein the image data generation unit determines the future predetermined time based on a change amount of a state of the moving body.   The control apparatus according to claim 3, wherein the image data generation unit determines a time when the amount of change exceeds a predetermined threshold as the predetermined future time.   The image data generation unit, when a time when the amount of change exceeds the predetermined threshold is close to a current time, determines a certain time after the current time as the predetermined time in the future. 4. The control device according to 4.   The control apparatus according to claim 3, wherein the image data generation unit determines a time point at which a time integral value of the change amount exceeds a predetermined threshold as the future time point.   The control apparatus according to claim 3, wherein the image data generation unit obtains the change amount between a current time point and a future one time point, or the change amount between a plurality of different future time points.   The image data generation unit changes at least one of a color, a density, a transparency, and a size of the symbol of the moving body to be generated according to a value indicating the predetermined future time point. The control device according to any one of 1 to 7. A control device according to any one of claims 1 to 8,
An optical unit that projects light based on the image data onto a predetermined projection area of the moving body;
A display device comprising:   A moving body equipped with the control device according to claim 1. A control method for generating an image that is displayed so as to be superimposed on a surrounding environment when viewed from a passenger of a moving object,
Generating image data including a symbol indicating a position of the mobile body at a predetermined future time based on at least one of internal information and external information of the mobile body;
Control method. A program for generating an image that is displayed so as to be superimposed on the surrounding environment when viewed from a passenger of a moving object,
A procedure for generating image data including a symbol indicating a position of the mobile body at a predetermined future time based on at least one of internal information and external information of the mobile body;
A program that causes a computer to execute.