JP2021072455A - Projection system - Google Patents

Abstract

To provide a projection system capable of accurately suppressing the misalignment of virtual images.SOLUTION: A projection system (100) displays virtual images in front of a windshield of a vehicle. The projection system (100) includes: a projection unit (10) that projects light representing a virtual image onto the windshield; an information acquisition unit (20) that acquires at least one of the location of a vehicle and information outside the vehicle; a display processing unit (30) that controls the display of the virtual images based on the information acquired by the information acquisition unit; a detection unit (40) that detects posture variation of the vehicle; and a correction processing unit (50) that sets a correction amount of the display position of the virtual image based on the posture variation of the vehicle. The virtual images are displayed based on a reference position and the correction amount and the display position of the virtual images is returned to the reference position based on predetermined conditions during the control of the display of the virtual images.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a projection system that displays a virtual image in front of the windshield of a vehicle.

Patent Document 1 discloses a vehicle information projection system that performs augmented reality (AR) display using a head-up display (HUD) device. The HUD device projects a light representing a virtual image onto the windshield of the vehicle so that a viewer who is a occupant of the vehicle can see the virtual image together with the actual view of the outside world of the vehicle. For example, a virtual image representing a vehicle guidance route is displayed in association with a display target (for example, a road) in the actual scene. As a result, the occupant can confirm the guide route while visually recognizing the actual scene. The vehicle information projection system of Patent Document 1 includes a vehicle speed sensor and corrects a virtual image display position according to acceleration. As a result, it is possible to prevent the virtual image from being displaced during sudden deceleration and sudden acceleration of the vehicle.

Japanese Unexamined Patent Publication No. 2015-101311

The present disclosure provides a projection system that accurately suppresses the misalignment of a virtual image.

The projection system of the present disclosure is a projection system that displays a virtual image in front of the front glass of a vehicle, and a projection device that projects a light representing the virtual image onto the front glass, and at least one of information on the position of the vehicle and the outside of the vehicle. An information acquisition device to be acquired, a display processing device that controls the display of a virtual image based on the information acquired by the information acquisition device, a detection device that detects a change in the attitude of the vehicle, and a display of a virtual image based on the change in the attitude of the vehicle. It is equipped with a correction processing device that sets the correction amount of the position, displays the virtual image based on the reference position and the correction amount, and returns the display position of the virtual image to the reference position based on a predetermined condition while controlling the display of the virtual image. ..

These general and specific aspects may be realized by systems, methods, and computer programs, and combinations thereof.

According to the projection system of the present disclosure, the display position of the virtual image is returned to the reference position when the amount of fluctuation related to the posture change of the vehicle detected by the detection device is larger than the threshold value. As a result, the misalignment of the virtual image can be corrected with high accuracy. Specifically, the detection error of the vehicle posture can be suppressed, and the vehicle posture can be detected with high accuracy.

A diagram for explaining a head-up display (HUD). A block diagram showing a configuration of a projection system according to the first, second, and third embodiments. The figure which shows the example when the vehicle is not tilted Diagram showing an example of a real scene seen from the windshield The figure which shows the example which the virtual image is displayed in the reference position. Diagram showing an example of augmented reality (AR) display Diagram showing the forward leaning posture of the vehicle The figure for demonstrating the example that the position shift of a virtual image occurs when a vehicle is leaning forward. The figure which shows the display example of the virtual image after correction Diagram to explain the misalignment of the virtual image due to sensor noise Flowchart showing display processing in the first embodiment Flow chart showing correction processing in the first embodiment Flowchart showing display processing in the second embodiment Flow chart showing correction processing in the second embodiment Flow chart showing correction processing in the third embodiment Flow chart showing correction processing in the fourth embodiment

(Knowledge on which this disclosure was based)
When the display position of the virtual image is corrected according to the state of the vehicle (for example, the posture) detected based on the output of the sensor, a correction error due to the noise of the sensor occurs. Therefore, if correction is performed based on the output of the sensor at all times or for a long time, the correction error is accumulated and the display position of the virtual image is greatly deviated from a predetermined display target (for example, a road) in the actual scene. In some cases.

For example, it is conceivable to use a gyro sensor in order to detect the vibration of the vehicle due to the shape such as the unevenness of the road surface with high accuracy. The angles (roll angle, pitch angle, and yaw angle) of the vehicle in the three axial directions are obtained by integrating the angular velocity detected by the gyro sensor. However, due to the characteristics of the device, the angular velocity of the output of the gyro sensor does not become zero even in the stationary state. So-called drift occurs. Therefore, in the integral calculation of the angular velocity, the error due to the drift is accumulated, and an error occurs in the obtained angle. In this case, if the display position of the virtual image is continuously corrected based on the output of the gyro sensor, the correction error becomes large. Therefore, the viewer feels uncomfortable with the display of the virtual image.

The projection system of the present disclosure returns the display position of the virtual image to the reference position when the fluctuation amount related to the posture change of the vehicle detected by the detection device such as the gyro sensor is larger than the threshold value. This reduces the correction error caused by the noise of the sensor that detects the attitude change of the vehicle. For example, the correction error caused by the drift of the gyro sensor is reduced.

(First Embodiment)
Hereinafter, the first embodiment will be described with reference to the drawings.

1. 1. Configuration of Projection System FIG. 1 is a diagram for explaining a head-up display (HUD). In FIG. 1, the roll axis of the vehicle 200 is the X axis, the pitch axis of the vehicle 200 is the Y axis, and the yaw axis of the vehicle 200 is the Z axis. That is, the X-axis is an axis orthogonal to the Y-axis and the Z-axis and along the line-of-sight direction of the occupant D who visually recognizes the virtual image Iv. The Y-axis is an axis along the left-right direction when viewed from the occupant D who visually recognizes the virtual image Iv. The Z-axis is an axis along the height direction of the vehicle 200.

The projection system 100 of the present embodiment is a HUD system that performs so-called augmented reality (AR) display in which a virtual image Iv is superimposed on an actual view in front of the windshield 210 of the vehicle 200. The virtual image Iv indicates predetermined information. For example, the virtual image Iv is a figure and a character indicating a route for guiding to a destination, an estimated time of arrival at the destination, a traveling direction, a speed, various warnings, and the like. The projection system 100 is installed in the vehicle 200 and projects the display light Lc representing the virtual image Iv into the display area 220 of the windshield 210 of the vehicle 200. In the present embodiment, the display area 220 is a part of the windshield 210. The display area 220 may be the entire area of the windshield 210. The indicator light Lc is reflected in the vehicle interior direction by the windshield 210. As a result, the occupant (viewer) D in the vehicle 200 visually recognizes the reflected display light Lc as a virtual image Iv in front of the vehicle 200.

The projection system 100 includes a projection device 10, an information acquisition device 20, a display processing device 30, a posture detection device 40, and a correction processing device 50.

The projection device 10 projects the display light Lc representing the virtual image Iv into the display area 220. The projection device 10 is, for example, a liquid crystal display element for displaying an image of a virtual image Iv, a light source such as an LED for illuminating the liquid crystal display element, a mirror and a lens for reflecting the display light Lc of the image displayed by the liquid crystal display element on the display area 220. And so on. The projection device 10 is installed, for example, in the dashboard of the vehicle 200.

The information acquisition device 20 acquires information on the position of the vehicle and the outside of the vehicle. Specifically, the information acquisition device 20 measures the position of the vehicle 200 and generates position information indicating the position. The information acquisition device 20 further acquires information outside the vehicle indicating the object and the distance to the object. Objects are people, signs, roads, and so on. The information acquisition device 20 outputs vehicle-related information including at least one of the position information and the outside information of the vehicle 200.

The display processing device 30 controls the display of the virtual image Iv based on the vehicle-related information obtained from the information acquisition device 20, and outputs the image data of the virtual image Iv to the projection device 10.

The attitude detection device 40 detects the attitude change of the vehicle 200.

The correction processing device 50 calculates the correction amount of the display position of the virtual image Iv based on the posture change of the vehicle 200 detected by the posture detection device 40.

FIG. 2 is a block diagram showing an internal configuration of the projection system 100.

In the present embodiment, the information acquisition device 20 includes a GPS (Global Positioning System) module 21 that detects a position indicating the current location of the vehicle 200 in the geographic coordinate system. Specifically, the GPS module 21 receives radio waves from GPS satellites and positions the latitude and longitude of the received points. The GPS module 21 generates position information indicating the measured latitude and longitude. The information acquisition device 20 further includes a camera 22 that captures an outside scene and generates imaging data. The information acquisition device 20 identifies an object from, for example, imaging data, and measures the distance to the object. The information acquisition device 20 generates information indicating the object and the distance to the object as outside information. The information acquisition device 20 outputs vehicle-related information including position information and outside vehicle information to the display processing device 30. The imaging data generated by the camera 22 may be output to the display processing device 30.

The display processing device 30 includes a communication unit 31, a display control unit 32, and a storage unit 33.

The communication unit 31 is a predetermined communication standard (for example, LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark), CAN (controller area network), SPI (Serial Peripheral Interface)). Includes a circuit that communicates with external devices in accordance with.

The display control unit 32 can be realized by a semiconductor element or the like. The display control unit 32 can be composed of, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, and an ASIC. The function of the display control unit 32 may be configured only by hardware, or may be realized by combining hardware and software. The display control unit 32 realizes a predetermined function by reading data or a program stored in the storage unit 33 and performing various arithmetic processes.

The storage unit 33 is a storage medium that stores programs and data necessary for realizing the functions of the display processing device 30. The storage unit 33 can be realized by, for example, a hard disk (HDD), SSD, RAM, DRAM, ferroelectric memory, flash memory, magnetic disk, or a combination thereof.

A plurality of image data 33i representing the virtual image Iv are stored in the storage unit 33. The display control unit 32 determines the virtual image Iv to be displayed based on the vehicle-related information obtained from the information acquisition device 20. The display control unit 32 reads the image data 33i of the determined virtual image Iv from the storage unit 33 and outputs the image data 33i to the projection device 10. Further, the display control unit 32 sets the display position of the virtual image Iv. The display control unit 32 outputs display information indicating whether or not the virtual image Iv is displayed or whether or not the virtual image Iv is being displayed to the correction processing device 50.

In the present embodiment, the attitude detection device 40 includes a gyro sensor 41 that detects an angular velocity. The gyro sensor 41 outputs the detected angular velocity to the correction processing device 50 as posture change information indicating the posture change of the vehicle 200.

The correction processing device 50 includes a communication unit 51 and a correction control unit 52.

The communication unit 51 conforms to a predetermined communication standard (for example, LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark), CAN (controller area network), SPI (Serial Peripheral Interface)). Includes circuits that comply with and communicate with external devices.

The correction control unit 52 can be realized by a semiconductor element or the like. The correction control unit 52 can be composed of, for example, a microcomputer, a CPU, an MPU, a GPU, a DSP, an FPGA, and an ASIC. The function of the display control unit 32 may be configured only by hardware, or may be realized by combining hardware and software. The correction control unit 52 realizes a predetermined function by reading data or a program stored in a storage unit (not shown) in the correction processing device 50 and performing various arithmetic processes.

The correction control unit 52 includes a deviation amount calculation unit 52a, a correction amount calculation unit 52b, and a determination unit 52c as functional configurations.

The deviation amount calculation unit 52a calculates the attitude (angle deviation amount) of the vehicle 200 based on the attitude fluctuation information output by the attitude detection device 40. For example, the deviation amount calculation unit 52a calculates the angles (roll angle, pitch angle, and yaw angle) of the vehicle 200 in the three axial directions by integrating the angular velocities detected by the gyro sensor 41. Thereby, the deviation amount (angle) of the vehicle 200 in the X-axis (roll axis), Y-axis (pitch axis), and Z-axis (yaw axis) directions shown in FIG. 1 can be calculated. In the present embodiment, all angles in the triaxial direction are calculated, but angles in the uniaxial or biaxial direction may be calculated. For example, only the angles in the Y-axis and Z-axis directions may be calculated.

The correction amount calculation unit 52b calculates the correction amount of the display position of the virtual image Iv according to the posture (angle deviation amount) of the vehicle 200. Specifically, the correction amount calculation unit 52b converts the deviation amount of the angle (pitch angle and yaw angle) calculated by the deviation amount calculation unit 52a into the number of pixels, and the number of pixels corresponding to the deviation (hereinafter, "" The amount of correction that restores the "number of displaced pixels") is determined. The roll angle is output as it is. For example, the deviation amount calculation unit 52a determines a correction amount for restoring the deviation amount of the roll angle. In the present embodiment, the correction amount is indicated by the number of pixels in the Y-axis direction and the Z-axis direction. The correction amount calculation unit 52b outputs the calculated correction amount to the display processing device 30.

The determination unit 52c calculates and processes the output of the posture detection device 40 or the output of the posture detection device 40 (hereinafter referred to as the fluctuation amount V) in the internal comparison unit 52d, and the threshold value a (first threshold value). It is determined whether or not to return the display position of the virtual image to the reference position by comparing the magnitude relations, and the determination result is output to the deviation amount calculation unit 52a. The calculation processing of the output of the posture detection device 40 may be performed by the deviation amount calculation unit 52a or the determination unit 52c of the posture detection device 40 or the correction processing device 50, or another configuration. The fluctuation amount V is, for example, the angular velocity detected by the gyro sensor 41 or the angle obtained by integrating the angular velocity (absolute value of the angle and the amount of change in the angle). The determination unit 52c, for example, binarizes the data as a Boolean type and outputs the data. If the fluctuation amount V is equal to or less than the threshold value a, it is TRUE, and if the fluctuation amount V is larger than the threshold value a, it is FALSE. The determination unit 52c does not limit the data to the Boolean type, and may output the data as an integer type or any other type.

The display processing device 30 and the correction processing device 50 communicate with each other in both directions by the communication units 31 and 51. The display processing device 30 outputs display information to the correction processing device 50. The correction processing device 50 outputs a correction amount to the display processing device 30.

The AR display will be described with reference to FIGS. 3A to 3D. FIG. 3A shows an example when the vehicle 200 is not tilted. FIG. 3B shows an example of an actual view seen from the windshield 210 of the vehicle 200 shown in FIG. 3A. FIG. 3C shows an example of the virtual image Iv seen from the display area 220. FIG. 3D shows an example in which the virtual image Iv shown in FIG. 3C is superposed on the actual scene shown in FIG. 3B. The projection system 100 superimposes the virtual image Iv shown in FIG. 3C on the actual scene shown in FIG. 3B. The reference position (initial position) P0 of the virtual image Iv is a position determined based on the type of the virtual image Iv, the state (position and posture) of the vehicle 200, the map data, and the like. For example, when the display target 230 is a traveling lane and the virtual image Iv is an arrow indicating the traveling direction, the reference position P0 is the center of the traveling lane. The reference position P0 is set by the values of the Y coordinate and the Z coordinate in the display area 220, for example, in FIG. 3C. The reference position P0 is acquired from an external device. Since the reference position P0 output from the external device may change based on the number of occupants, the fluctuation of the load, and the fluctuation of the posture due to the decrease of gasoline, for example, the reference position (initial position) acquired first May be different. The display processing device 30 may set the reference position P0 based on the vehicle-related information, the map data, and the like. The display processing device 30 may set the size of the virtual image Iv based on the vehicle-related information.

FIG. 4A shows an example of a state in which the vehicle 200 is in a forward leaning posture. FIG. 4B illustrates a case where the display position of the virtual image Iv deviates from the display target 230 according to the posture change of the vehicle 200. FIG. 4C shows the display position of the corrected virtual image Iv.

The vehicle 200 may tilt due to unevenness of the road surface, sudden acceleration or deceleration of the vehicle 200, or the like. For example, when the vehicle 200 suddenly decelerates, the vehicle 200 takes a forward leaning posture as shown in FIG. 4A. In this case, as shown in FIG. 4B, the position of the display target 230 seen from the windshield 210 changes according to the inclination of the vehicle 200. Therefore, when the virtual image Iv is displayed at the reference position P0, the virtual image Iv deviates from the display target 230. For example, as shown in FIG. 4B, the tip of the arrow is in the oncoming lane 231. Therefore, the projection system 100 adjusts the display position of the virtual image Iv in the direction of returning the deviation according to the posture of the vehicle 200. Specifically, as shown in FIG. 4C, the correction amount C is calculated so that the correction processing device 50 is at the position P1 where the display position does not deviate due to the angle of the vehicle 200. That is, the display processing device 30 sets the display position of the virtual image Iv to "reference position P0 + correction amount C". As a result, the projection device 10 can display the virtual image Iv at the position P1 corresponding to the display target 230. In this way, even when the vehicle 200 is tilted, by changing the display position of the virtual image Iv from the reference position P0 based on the correction amount C, the virtual image Iv is moved to the position P1 corresponding to the display target 230 in the actual scene. Can be displayed.

FIG. 5 illustrates a case where the display position of the virtual image Iv deviates from the display target 230 due to noise of a sensor such as the gyro sensor 41. As described above, for example, the angular velocity detected by the gyro sensor 41 includes an error due to drift. Therefore, if the calculation of the correction amount based on the integral calculation of the angular velocity is continued for a long period of time, the error included in the correction amount becomes large. .. In this case, for example, even when the vehicle 200 is not actually tilted, it is detected that the vehicle 200 is tilted, and the correction amount C does not become zero. Therefore, as the duration of the virtual image Iv correction process becomes longer, the display position of the virtual image Iv (= reference position P0 + correction amount C) deviates significantly from the display target 230. For example, the position P1 (= reference position P0 + correction amount C) that is actually displayed does not become the position P2 that is desired to be displayed with respect to the display target 230. In this embodiment, in order to reduce the misalignment E caused by the noise of this sensor, the correction amount C is set to zero based on the output of the determination unit 52c, that is, the amount of fluctuation related to the posture change of the vehicle, as will be described later. Reset to. Specifically, when the fluctuation amount V is larger than the threshold value a, the correction amount C is reset to zero. As a result, when the fluctuation amount V is larger than the threshold value a, the display position of the virtual image Iv is returned to the desired position P2.

2. Operation of the Display Processing Device FIG. 6 shows the display processing performed by the display control unit 32 of the display processing device 30. The display process shown in FIG. 6 is started, for example, when the engine of the vehicle 200 is started, or when a button for instructing the display start of the virtual image Iv is operated.

The display control unit 32 acquires vehicle-related information from the information acquisition device 20 (S101). The display control unit 32 determines the virtual image Iv to be displayed based on the vehicle-related information (S102). The display control unit 32 acquires the reference position P0 of the virtual image Iv from an external device (S103). The display control unit 32 acquires the correction amount C of the display position output from the correction processing device 50 (S104).

The display control unit 32 causes the projection device 10 to display the virtual image Iv based on the reference position P0 and the correction amount C (S105). For example, the display control unit 32 reads the image data 33i of the virtual image Iv corresponding to the display target from the storage unit 33, sets the display position of the virtual image Iv to "reference position P0 + correction amount C", and outputs the image data to the projection device 10. To do.

The display control unit 32 determines whether or not to continue the display process (S106). For example, when the engine of the vehicle 200 is stopped, or when the button for instructing the end of the display of the virtual image Iv is operated, the display control unit 32 ends the display process. If the display process is to be continued, the process returns to step S101.

3. 3. Operation of the Correction Processing Device FIG. 7 shows the correction processing performed by the correction control unit 52 of the correction processing device 50. The correction process shown in FIG. 7 is started, for example, when the engine of the vehicle 200 is started, or when a button for instructing the start of displaying the virtual image Iv is operated. The correction process of FIG. 7 is started together with the display process of FIG. 6, for example. The correction process shown in FIG. 7 may be started when the button for instructing the start of the position correction of the virtual image Iv is operated.

The deviation amount calculation unit 52a acquires attitude fluctuation information indicating the angular velocity output from the gyro sensor 41 (S201). The deviation amount calculation unit 52a calculates the attitude of the vehicle 200, that is, the deviation amount which is an angle with respect to the three axial directions, based on the acquired attitude change information (S202). Specifically, the deviation amount calculation unit 52a calculates the angle of the vehicle 200 by integrating the angular velocity. The correction amount calculation unit 52b calculates the correction amount C of the display position of the virtual image Iv based on the deviation amount with respect to the three axial directions (S203). Specifically, the correction amount calculation unit 52b converts the deviation amount, which is the angle of the vehicle 200 with respect to the pitch angle and the yaw angle, into the number of pixels, and corrects the correction amount C so as to cancel the deviation amount indicated by the number of pixels. decide. Regarding the roll angle, the correction amount C that cancels the deviation amount is determined while keeping the angle.

In the present embodiment, the correction amount C is defined as "correction amount C =-(current deviation amount) + (deviation amount at the time of zero reset)". Hereinafter, the amount of deviation at the time of zero reset is also referred to as an offset value. The offset value is set in step S208 described later. The initial value of the offset value is, for example, zero. In the calculation of the correction amount in step S203, the deviation amount calculation unit 52a calculates "-current posture (angle) + offset value (angle)" in angle units and outputs the correction amount to the correction amount calculation unit 52b. The calculation unit 52b may convert the input value into the number of pixels. Further, the deviation amount calculation unit 52a outputs the current posture (angle) to the correction amount calculation unit 52b, and after the correction amount calculation unit 52b converts the posture (angle) into the number of pixels, "-current deviation". Amount (number of pixels) + offset value (number of pixels) "may be calculated.

The comparison unit 52d acquires the fluctuation amount V (calculation processing result based on the output of the attitude detection device 40 or the output of the attitude detection device 40) (S204). Then, the comparison unit 52d compares the fluctuation amount V with the threshold value a (S205).

If the fluctuation amount V is equal to or less than the threshold value a (Yes in S205), the correction amount calculation unit 52b outputs the calculated correction amount C to the display processing device 30 (S206). As a result, the virtual image Iv is displayed at the position indicated by "reference position P0 + correction amount C".

When the fluctuation amount V is larger than the threshold value a (No in S205), the correction control unit 52 resets the correction amount C to zero (S208). Specifically, for example, the deviation amount calculation unit 52a sets the offset value (angle) to "offset value (angle) = posture (angle)". As a result, the deviation amount calculation unit 52a outputs the correction amount calculation unit 52b to the angle indicated by "-posture (angle) + offset value (angle)", that is, 0 degree. Alternatively, the correction amount calculation unit 52b converts the posture (angle) calculated by the deviation amount calculation unit 52a into the number of pixels (number of deviation pixels), and converts the offset value (number of pixels) into "offset value (number of pixels) = deviation pixel". Set to "Number". As a result, the correction amount C calculated by "-shift amount (number of pixels) + offset value (number of pixels)" becomes zero. In this way, when the fluctuation amount V related to the posture change of the vehicle is larger than the threshold value a, the display position is returned to the reference position P0.

The correction control unit 52 determines whether or not to continue the correction process (S207). For example, when the engine of the vehicle 200 is stopped, or when the button for instructing the end of the display of the virtual image Iv is operated, the correction control unit 52 ends the correction process. If the correction process is to be continued, the process returns to step S201. After returning to step S201, the offset value set in the previous step S208 is used in the calculation of the correction amount in the next S203.

As described above, in the present embodiment, when the fluctuation amount V is larger than the threshold value a, the correction amount C is set to zero by setting “offset value = deviation amount”. In other words, when the fluctuation amount V is larger than the threshold value a, the display position is reset to the reference position P0. Since "correction amount C = -shift amount + offset value", the "reference position P0 + correction amount C", which is the display position when the virtual image Iv is displayed next (step S105 in FIG. 6), is "reference position P0 +". It corresponds to "offset value-shift amount". By setting this offset value to the amount of deviation when the fluctuation amount V is larger than the threshold value a, the correction error caused by the noise of the sensor output from the correction processing device 50 to the display processing device 30 is reset. ..

4. Effects and supplements The projection system 100 of the present invention displays a virtual image Iv in front of the windshield 210 of the vehicle 200. The projection system 100 includes a projection device 10 that projects a display light Lc representing a virtual image Iv onto a windshield 210, an information acquisition device 20 that acquires at least one of the position of the vehicle 200 and information outside the vehicle, and an information acquisition device 20. A display processing device 30 that controls the display of the imaginary image Iv based on the acquired information, an attitude detection device 40 that detects the attitude change of the vehicle 200, and a correction amount C of the display position of the imaginary image Iv based on the attitude change of the vehicle 200. The correction processing device 50 for setting the above is provided. The display processing device returns the display position of the virtual image to the reference position based on the fluctuation amount V related to the posture change of the vehicle detected by the detection device.

As a result, the detection error of the vehicle attitude caused by the noise of the attitude detection device 40 can be suppressed, and the accurate detection of the vehicle attitude becomes possible.

When the fluctuation amount V is equal to or less than the threshold value a, the projection system 100 displays the virtual image Iv based on the reference position P0 and the correction amount C, and when the fluctuation amount V is larger than the threshold value a, the display position of the virtual image Iv is used as a reference. Return to position P0. Specifically, the correction processing device 50 resets the correction amount C to zero when the fluctuation amount V is larger than the threshold value a.

By resetting the correction amount C to zero, it is possible to reduce the amount of deviation of the display position due to the accumulation of noise of the sensor used for detecting the vehicle posture. Further, since the reset is performed when the fluctuation amount V is larger than the threshold value a, that is, when the posture change of the vehicle 200 is large and the change in the external view seen from the occupant D is large, the display position of the virtual image Iv fluctuates due to the reset. It is possible to prevent a person from feeling uncomfortable. Further, since the correction amount is reset every time the fluctuation amount V becomes larger than the threshold value a, the chance of resetting the correction amount increases. Therefore, the detection error of the vehicle posture caused by the noise of the sensor can be suppressed, and the vehicle posture can be detected with high accuracy. When the fluctuation amount V is equal to or less than the threshold value a, the display position is corrected by the correction amount C based on the vehicle posture. Therefore, it is possible to suppress the deviation of the display position due to the vehicle posture.

In the present embodiment, the correction amount C when the fluctuation amount V is equal to or less than the threshold value a is the amount of deviation of the display position based on the posture change of the vehicle 200 when the fluctuation amount V is equal to or less than the threshold value a and the fluctuation amount V is the threshold value a. It is set based on the deviation amount (offset value) of the display position based on the attitude change of the vehicle 200 before becoming the following.

The display processing device 30 determines whether or not to display the virtual image Iv based on the position of the vehicle 200, the information outside the vehicle, the display timing, and the like. As a result, the virtual image Iv can be displayed in association with the display target in the actual scene. Therefore, the information indicated by the virtual image Iv can be visually recognized without discomfort to the viewer.

In step S208, the method of resetting the correction amount C to zero is arbitrary. In the present embodiment, "correction amount C =-shift amount + offset value" is set, but the correction amount C may be set as "correction amount C = -shift amount". In this case, the correction amount C is reset to zero by setting the deviation amount itself to zero. Specifically, when the vehicle attitude is calculated based on the output of the gyro sensor 41, the integrated amount of the angular velocity calculated by the deviation amount calculation unit 52a is reset to zero.

(Second Embodiment)
In the first embodiment, the correction amount C is changed according to the offset value. Specifically, when displaying the virtual image Iv, the display position is set to "reference position P0 + correction amount C", and the correction amount C at this time is "correction amount C = -deviation amount + offset value (at the time of zero reset). The amount of deviation) ”was set. In the present embodiment, the reference position is changed by the offset value as "correction amount C =-shift amount" and "reference position P0'= reference position P0 + offset value (shift amount at the time of zero reset)". The initial value of the offset value is, for example, zero.

FIG. 8 shows the display processing performed by the display control unit 32 of the display processing device 30. Steps S301 to S303 and S306 of FIG. 8 of the second embodiment are the same as steps S101 to S103 and S106 of FIG. 6 of the first embodiment. In the present embodiment, the display control unit 32 acquires an offset value (number of pixels) from the correction processing device 50 together with the correction amount when displaying the virtual image (S304). The display control unit 32 sets the display position of the virtual image Iv to "reference position P0'(= reference position P0 + offset value) + correction amount C (=-shift amount)", and causes the projection device 10 to display the virtual image Iv. (S305).

FIG. 9 shows the correction processing performed by the correction control unit 52 of the correction processing device 50. Steps S401 to S405 and S407 of FIG. 9 of the second embodiment are the same as steps S201 to S205 and S207 of FIG. 7 of the first embodiment. In the present embodiment, when the fluctuation amount V is equal to or less than the threshold value a, the correction control unit 52 outputs an offset value (number of pixels) together with the correction amount to the display processing device 30 (S406). This offset value is set when the fluctuation amount V is larger than the threshold value a (S408). Specifically, the offset value is set to "offset value = deviation amount at zero reset". As a result, in step S305 of FIG. 8, the reference position P0'of the virtual image Iv indicated by "reference position P0'(= reference position P0 + offset value)" is shifted from the reference position P0 by the offset value.

(Third Embodiment)
In the first embodiment, the correction control unit 52 resets the correction amount C to zero when the fluctuation amount V is larger than the threshold value a. In the present embodiment, the correction control unit 52 reduces the size of the correction amount C by a predetermined amount so as not to become zero. Then, the correction amount C is gradually reduced each time the fluctuation amount V becomes larger than the threshold value a.

FIG. 10 shows the correction process in the third embodiment. Steps S501 to S507 of FIG. 10 of the third embodiment are the same as steps S201 to S207 of FIG. 7 of the first embodiment.

In the present embodiment, when the fluctuation amount V is larger than the threshold value a (No in S505), the correction amount calculation unit 52b determines whether or not the correction amount C calculated in step S503 is zero (S508). If the correction amount C is not zero, the size of the correction amount C is reduced by a predetermined amount (a value smaller than the correction amount C) so that the size of the correction amount C does not become zero (S509). Specifically, for example, the correction amount calculation unit 52b sets "offset value = predetermined amount" in "correction amount C =-(displacement amount-offset value)". The predetermined amount may be set according to the display position in the display area 220 of the virtual image Iv. If the correction amount C is zero, the process proceeds to step S507.

In this way, the correction processing device 50 reduces the correction amount C by a predetermined amount each time the fluctuation amount V becomes larger than the threshold value a, so that the position of the virtual image Iv gradually returns to the reference position P0. Since the position of the virtual image Iv does not suddenly change significantly, it is possible to prevent the occupant D from feeling uncomfortable with the change in the display position of the virtual image Iv. That is, it is possible to suppress a feeling of strangeness due to the shift of the display position.

Instead of reducing the correction amount C by a predetermined amount, the offset value shown in the second embodiment may be set to a value that is a certain amount smaller than the deviation amount when the fluctuation amount V is larger than the threshold value a.

(Fourth Embodiment)
When the fluctuation amount V is larger than the threshold value a, the correction amount C is reset to zero in the first embodiment, and the size of the correction amount C is set to zero in the third embodiment so that the size of the correction amount C does not become zero. The predetermined amount was reduced. In the present embodiment, the amount of change in the correction amount is changed according to the magnitude of the correction amount C. Specifically, when the correction amount C is equal to or greater than the threshold value b (second threshold value), the correction amount C is reduced by a predetermined amount so that the size of the correction amount C does not become zero, and the correction amount C is less than the threshold value b. When, the correction amount is reset to zero.

FIG. 11 shows the correction process in the fourth embodiment. Steps S601 to S607 of FIG. 11 of the fourth embodiment are the same as steps S201 to S207 of FIG. 7 of the first embodiment and steps S501 to S507 of FIG. 10 of the third embodiment.

In the present embodiment, when the fluctuation amount V is larger than the threshold value a (No in S605), the correction amount calculation unit 52b determines whether or not the correction amount C calculated in step S603 is equal to or greater than the threshold value b (S608). If the correction amount C is equal to or greater than the threshold value b, the correction amount C is reduced by a predetermined amount (a value smaller than the correction amount C) so that the magnitude of the correction amount C does not become zero (S609). Specifically, for example, the correction amount calculation unit 52b sets "offset value = predetermined amount" in "correction amount C =-(displacement amount-offset value)". If the correction amount C is smaller than the threshold value b, the correction amount C is reset to zero. Specifically, it is set to "offset value = deviation amount".

As described above, each time the fluctuation amount V becomes larger than the threshold value a, the correction processing device 50 reduces the correction amount by a predetermined amount when the correction amount C is equal to or more than the threshold value b, and the correction amount C is smaller than the threshold value b. If so, reset the correction amount to zero. As a result, the display position can be corrected according to the inclination of the vehicle 200 without any discomfort in appearance.

(Other embodiments)
As described above, the above-described embodiment has been described as an example of the technology disclosed in the present application. However, the technique in the present disclosure is not limited to this, and can be applied to embodiments in which changes, replacements, additions, omissions, etc. are appropriately made. Therefore, other embodiments will be illustrated below.

In the above embodiment, the case where the projection device 10, the information acquisition device 20, the display processing device 30, the posture detection device 40, and the correction processing device 50 are separate devices has been illustrated. However, a plurality of devices may be integrally formed as one device. For example, the display processing device 30 and the correction processing device 50 may be integrally formed as one device. The information acquisition device 20 and the display processing device 30 may be integrally formed as one device. The posture detection device 40 and the correction processing device 50 may be integrally formed as one device. The separately formed devices are communicatively connected to each other by wire or wirelessly. The projection device 10, the information acquisition device 20, the display processing device 30, the posture detection device 40, and the correction processing device 50 may all be formed as one device. In this case, the communication units 31 and 51 may not be provided.

In the above embodiment, an example in which the information acquisition device 20 includes the GPS module 21 and the camera 22 has been described. However, the information acquisition device 20 may include a distance sensor that measures the distance and direction from the vehicle 200 to the surrounding object, and may output distance information indicating the measured distance and direction to the display processing device 30. .. The information acquisition device 20 may include a vehicle speed sensor that detects the speed of the vehicle 200, or may include a navigation system. The information acquisition device 20 may include one or more of the GPS module 21, the distance sensor, the camera 22, and the like. In this case, the GPS module 21, the distance sensor, the camera 22, and the like having a function as the information acquisition device 20 may be built in one device, or may be individually attached to the vehicle 200.

In the above embodiment, an example in which the posture detection device 40 includes the gyro sensor 41 has been described. However, the posture detection device 40 may include an acceleration sensor that detects the acceleration of the vehicle 200, and may output the detected acceleration as posture fluctuation information. The posture detection device 40 may include a vehicle height sensor that detects the height from the road surface, and may output the detected height as posture fluctuation information. The attitude detection device 40 may include other known sensors. The attitude detection device 40 may include one or more of a gyro sensor 41, an acceleration sensor, a vehicle speed sensor, and the like. In this case, the gyro sensor 41, the acceleration sensor, the vehicle height sensor, and the like having a function as the attitude detection device 40 may be built in one device, or may be individually attached to the vehicle 200.

In the above embodiment, the display processing device 30 displays the virtual image based on the reference position and the correction amount set by the correction processing device 50, and the display position of the virtual image is used as a reference based on a predetermined condition while controlling the display of the virtual image. Return to position. This predetermined condition is regarded as the amount of fluctuation related to the posture change of the vehicle detected by the detection device. Then, when the fluctuation amount is larger than the threshold value a, the correction value is reset to zero, or the correction amount is reduced by a predetermined amount so as not to become zero. When the fluctuation amount is larger than the threshold value a and the correction amount is equal to or more than the threshold value b, the correction amount is reduced by a predetermined amount so as not to become zero, and when the correction amount is smaller than the threshold value b, the correction amount is reset to zero. did.

This predetermined condition is assumed to be a state in which the virtual image is displayed / hidden while the display processing device 30 is controlling the display of the virtual image, and the correction amount is set to zero when the virtual image is not displayed (while the virtual image is not displayed). It may be reset. Further, this predetermined condition may be the relationship between the eye point of the occupant D and the eye box of the projection system, and the correction amount may be reset to zero when the eye point deviates from the eye box due to the movement of the occupant D. .. Further, this predetermined condition may be assumed to be the line of sight of the occupant D, and the correction amount may be reset to zero when the line of sight of the occupant D departs from the virtual image displayed by the display processing device 30. Further, this predetermined condition may be the moving speed of the vehicle, and the correction value may be reset to zero when the moving speed is larger than the threshold value c. Further, it is assumed that the outside view is from the vehicle acquired by the information acquisition device 20, and the correction value may be reset to zero when the change in the outside view is large.

(Outline of Embodiment)
(1) The projection system of the present disclosure is a projection system that displays a virtual image in front of the front glass of a vehicle, and is a projection device that projects a light representing the virtual image onto the front glass, and at least information on the position of the vehicle and the outside of the vehicle. An information acquisition device that acquires one, a display processing device that controls the display of a virtual image based on the information acquired by the information acquisition device, an attitude detection device that detects a vehicle attitude change, and a vehicle attitude change. It is equipped with a correction processing device that sets the correction amount of the display position of the virtual image, displays the virtual image based on the reference position and the correction amount, and displays the display position of the virtual image based on a predetermined condition while controlling the display of the virtual image. To return to the reference position.

As a result, the misalignment of the virtual image can be corrected with high accuracy.

(2) In the projection system of (1), the predetermined condition may be a fluctuation amount related to the posture change of the vehicle detected by the detection device.

(3) In the projection system of (2), the correction processing device may reset the correction amount to zero when the fluctuation amount V is larger than the threshold value a.

(4) In the projection system of (2), when the fluctuation amount V is larger than the threshold value a, the correction processing device may reduce the correction amount by a predetermined amount so as not to become zero.

(5) In the projection system of (2), when the fluctuation amount V is larger than the threshold value a and the correction amount is equal to or more than the threshold value b, the correction processing device reduces the correction amount by a predetermined amount so as not to become zero, and corrects the correction amount. If the amount is smaller than the threshold value b, the correction amount may be reset to zero.

(6) In the projection system of (2), the correction amount when the fluctuation amount V is the threshold value a or less is the attitude change of the vehicle when the fluctuation amount V is the threshold value a or less and the fluctuation amount V is the threshold value a or less. It may be set based on the attitude change of the vehicle in front.

(7) In the projection system of (2) to (6), the amount of fluctuation may be the angular velocity detected by the attitude detection device.

(8) In the projection system of (2) to (6), the amount of fluctuation may be the amount of change in the angle calculated from the angular velocity detected by the attitude detection device.

(9) In the projection system of (2) to (6), the attitude detection device may include at least one of a gyro sensor, an acceleration sensor, and a vehicle height sensor.

(10) Another projection system of the present disclosure is a projection system that displays a virtual image in front of the front glass of the vehicle, and is a projection device that projects light representing the virtual image onto the front glass, and information on the position of the vehicle and the outside of the vehicle. An information acquisition device that acquires at least one of the above, a display processing device that controls the display of a virtual image based on the information acquired by the information acquisition device, an attitude detection device that detects a vehicle attitude change, and a vehicle attitude change. A correction processing device for calculating the amount of deviation of the display position of the virtual image according to the above is provided, and the reference position is set based on a predetermined condition during the control of the display of the virtual image, and the virtual image is set based on the reference position and the amount of deviation. Set the display position.

(11) In the projection system of (10), as a predetermined condition, when the fluctuation amount related to the posture fluctuation of the vehicle detected by the output device is larger than the first threshold value, the posture fluctuation of the vehicle when the fluctuation amount is larger than the first threshold value. The reference position is set based on the above, and when the fluctuation amount is equal to or less than the first threshold value, the posture of the vehicle detected by the detection device that sets the display position of the imaginary image based on the reference position and the deviation amount calculated by the correction processing device. It may be a fluctuation amount related to fluctuation.

The projection system described in all claims of the present disclosure is realized by cooperation with hardware resources such as a processor, memory, and a program.

The present disclosure is applicable to a projection system in which light representing a virtual image is projected onto the windshield of a vehicle so that the virtual image is visually recognized in front of the windshield.

10 Projection device 20 Information acquisition device 21 GPS module 22 Camera 30 Display processing device 31 Communication unit 32 Display control unit 33 Storage unit 40 Attitude detection device 41 Gyro sensor 50 Correction processing device 51 Communication unit 52 Correction control unit 52a Misalignment calculation unit 52b Correction amount calculation unit 52c Judgment unit 52d Comparison unit 100 Projection system

Claims (13)

A projection system that displays a virtual image in front of the windshield of a vehicle.
A projection device that projects light representing the virtual image onto the windshield,
An information acquisition device that acquires at least one of the vehicle position and information outside the vehicle, and
A display processing device that controls the display of the virtual image based on the information acquired by the information acquisition device, and
A detection device that detects the posture change of the vehicle and
A correction processing device that sets a correction amount for the display position of the virtual image based on the posture change of the vehicle, and
With
The virtual image is displayed based on the reference position and the correction amount, and the display position of the virtual image is returned to the reference position based on a predetermined condition while controlling the display of the virtual image.
Projection system. The predetermined condition is a fluctuation amount related to the posture change of the vehicle detected by the detection device.
The projection system according to claim 1. The correction processing device resets the correction amount to zero when the fluctuation amount is larger than the first threshold value.
The projection system according to claim 2. When the fluctuation amount is larger than the first threshold value, the correction processing device reduces the correction amount by a predetermined amount so as not to become zero.
The projection system according to claim 2. When the fluctuation amount is larger than the first threshold value, the correction processing device reduces the correction amount by a predetermined amount so as not to become zero when the correction amount is equal to or more than the second threshold value, and the correction amount is the second threshold value. If it is smaller than the threshold value, the correction amount is reset to zero.
The projection system according to claim 2. The correction amount when the fluctuation amount is equal to or less than the first threshold value is the posture change of the vehicle when the fluctuation amount is equal to or less than the first threshold value and the vehicle before the fluctuation amount becomes equal to or less than the first threshold value. Set based on the posture fluctuation of
The projection system according to claim 2. The amount of fluctuation is the angular velocity detected by the detection device.
The projection system according to any one of claims 2 to 6. The fluctuation amount is an angle change amount calculated from the angular velocity detected by the detection device.
The projection system according to any one of claims 2 to 6. The detection device includes at least one of a gyro sensor, an acceleration sensor, and a vehicle height sensor.
The projection system according to any one of claims 2 to 6. A projection system that displays a virtual image in front of the windshield of a vehicle.
A projection device that projects light representing the virtual image onto the windshield,
An information acquisition device that acquires at least one of the vehicle position and information outside the vehicle, and
A display processing device that controls the display of the virtual image based on the information acquired by the information acquisition device, and
A detection device that detects the posture change of the vehicle and
A correction processing device that calculates the amount of deviation of the display position of the virtual image due to the posture change of the vehicle, and
With
While controlling the display of the virtual image, a reference position is set based on a predetermined condition, and the display position of the virtual image is set based on the reference position and the deviation amount.
Projection system. As the predetermined condition
When the fluctuation amount related to the posture fluctuation of the vehicle detected by the detection device is larger than the first threshold value, the reference position is set based on the posture fluctuation of the vehicle when the fluctuation amount is larger than the first threshold value.
When the fluctuation amount is equal to or less than the first threshold value, the display position of the virtual image is set based on the reference position and the deviation amount calculated by the correction processing device.
The projection system according to claim 10. The amount of fluctuation is the angular velocity detected by the detection device.
The projection system according to claim 11. The fluctuation amount is an angle change amount calculated from the angular velocity detected by the detection device.
The projection system according to claim 11.

Images