WO2020083318A1

WO2020083318A1 - Head-up display system and display method, and automobile

Info

Publication number: WO2020083318A1
Application number: PCT/CN2019/112816
Authority: WO
Inventors: 张永亮
Original assignee: 中兴通讯股份有限公司
Priority date: 2018-10-23
Filing date: 2019-10-23
Publication date: 2020-04-30
Also published as: CN111086451A; CN111086451B

Abstract

Disclosed is a head-up display system (10), comprising: an image collection apparatus (11) for capturing a real-time image of a pre-set image collection region; a processing apparatus (12) for identifying an identification object in the real-time image and/or relative motion parameter information of the identification object and a carrier of the head-up display system by using a pre-set image identification rule, for generating, according to the identification object and/or the relative motion parameter information, prompt information by using a pre-set information processing rule, and for combining the prompt information and the real-time image into an output image; and an image projection apparatus (13) for projecting the output image to an assigned region of a pre-set projection surface. Further provided are a head-up display method and an automobile.

Description

Head-up display system, display method and automobile

Technical field

The present disclosure relates to the technical field of vehicle-mounted equipment, and particularly to a head-up display system, a display method, and an automobile.

Background technique

Most of the rearview mirrors on both sides of the car have blind spots. The traffic situation in the blind zone has no way for drivers to find out from the rearview mirrors on both sides. In order to expand the observation field of view of the side mirrors on both sides, a common method is to add a convex mirror on the side mirrors on both sides or make part of the side mirrors into convex mirrors. In addition, the rear-view mirror mainly relies on the driver's active observation and lacks active prompt information.

Summary of the invention

An embodiment of the present disclosure provides a head-up display system, including: an image acquisition device, a processing device, and an image projection device. The image acquisition device is used to capture a real-time image of a preset image acquisition area. The processing device is used to recognize the identification object in the real-time image and / or the relative motion parameter information of the identification object and the carrier of the head-up display system using a preset image recognition rule, and used to / Or the relative motion parameter information, using preset information processing rules to generate prompt information, and merging the prompt information and the real-time image into an output image. The image projection device is used to project the output image to a designated area of a preset projection surface.

An embodiment of the present disclosure also provides a head-up display method, which includes: capturing a real-time image of a preset image collection area; using a preset image recognition rule to identify the identification object in the real-time image and / or the identification object and head-up Display relative motion parameter information of the carrier of the system, and generate prompt information according to the recognition object and / or the relative motion parameter information, using preset information processing rules, and merge the prompt information and the real-time image into an output An image; and projecting the output image to a designated area of a preset projection surface.

An embodiment of the present disclosure also provides an automobile, including a body and the head-up display system described above.

BRIEF DESCRIPTION

1 is a schematic structural diagram of a head-up display system according to an embodiment of the present disclosure;

2 is a schematic diagram of components of a head-up display system according to an embodiment of the present disclosure;

3 is a schematic diagram of a head-up display system according to an embodiment of the present disclosure being provided outside the vehicle;

4 is a schematic diagram of a head-up display system according to an embodiment of the present disclosure provided inside a vehicle;

5 is a schematic diagram of a working process of a head-up display system according to an embodiment of the present disclosure;

FIG. 6 is a schematic flowchart of a process according to the first embodiment of the present disclosure;

7 is a schematic diagram of a head-up display system projecting to a side window according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a processing flow in case 2 according to an embodiment of the present disclosure;

9 is a schematic diagram of a processing flow of a steering scenario according to an embodiment of the present disclosure;

10 is a schematic flowchart of a head-up display method according to an embodiment of the present disclosure.

detailed description

In the embodiment of the present disclosure, the image acquisition device captures the real-time image of the preset image acquisition area; the processing device identifies the identification object in the real-time image and the carrier of the identification object and the head-up display system according to the preset image recognition rules Relative motion parameter information, and merge the identification object and the relative motion parameter information into an output image according to a preset information processing rule; an image projection device projects the output image to a designated area of a preset projection surface.

The implementation, functional characteristics, and advantages of the technical solutions of the present disclosure will be described in further detail with reference to the accompanying drawings and embodiments below. As shown in FIG. 1, the head-up display system provided by an embodiment of the present disclosure includes an image acquisition device 11, a processing device 12 and an image projection device 13.

Here, the head-up display system 10 may be a single whole, or may be composed of different devices distributed at various positions of the vehicle body. The head-up display system 10 may be installed on a carrier such as an automobile. Similar to other in-vehicle systems, the head-up display system 10 may be powered by a carrier such as an automobile, or may be powered by the head-up display system 10 itself.

Augmented reality (AR, Augmented Reality) is a combination of real scenes and virtual scenes. On the basis of the pictures taken by the camera, through the computer processing power, the virtual data is superimposed on the real environment, and then the same picture is used to display and bring An interactive mode. The head-up display system 10 can provide driving information to the driver through the AR mode.

The image acquisition device 11 is used to capture a real-time image of a preset image acquisition area.

Here, the image acquisition device 11 may be an image or video capture device such as a camera. The preset image acquisition area may be an area that covers a blind spot in the rearview mirror on both sides.

In some embodiments, the image acquisition device 11 may include a first image acquisition device 111 and a second image acquisition device 112. The first image acquisition device 111 and the second image acquisition device 112 may capture real-time images of the preset image acquisition area from two different angles, respectively, for subsequent image processing.

Taking the head-up display system 10 as a whole as an example, the detailed internal structure of the head-up display system 10 may be as shown in FIG. 2, which includes an image acquisition device 11, a processing device 12 (for example, a main control circuit board), and an image projection device 13 . The image acquisition device 11 includes a first image acquisition device 111 and a second image acquisition device 112, for example, two cameras. The main control circuit board is also provided with a power management circuit for managing the power supply of the entire head-up display system 10, including managing external power supply and optional internal battery-powered power supply switching, charging and discharging, and various devices in the head-up display system 10 Power supply distribution etc. The main control circuit board may include a processor for data processing, and circuits and interfaces corresponding to various functions such as a power supply.

The processing device 12 is used for identifying the identification object in the real-time image and / or the relative motion parameter information of the identification object and the carrier of the head-up display system 10 by using preset image recognition rules, and The identification object and / or the relative motion parameter information uses preset information processing rules to generate prompt information, and merges the prompt information and the real-time image into an output image.

Here, the preset image recognition rule may be set according to the image acquisition device 11, and when the image acquisition device 11 is a camera, a real-time image captured by a single camera may be identified; the image acquisition device 11 is two Or when there are more than two cameras, the real-time images captured by the two cameras can be identified through binocular parallax ranging algorithm and other methods. Recognition of recognized objects can be achieved by methods such as model training and deep learning. The relative motion parameter may include a relative distance and a relative motion speed of the identification object and the carrier. The relative motion speed may be determined by dividing the relative distance between two time points by the time interval between the two time points. When the image acquisition device 11 is two or more cameras, a real-time image of one camera may be pre-selected and combined as an output image according to the preset information processing rule.

The preset information processing rule may be based on the recognition object and the relative motion parameter settings, and merge the required prompt information with the captured image, that is, display the prompt information directly on the captured image, for example, in the recognition object When it is a preset object, it can indicate the general outline of the identified object, etc. For example, when the relative motion parameter exceeds the predicted value, it can display prompt information such as distance or warning information.

In some embodiments, in order to strengthen the identification of key target objects and enhance targeted warning measures, single camera can be integrated with deep learning based object recognition capabilities; roads and obstacles or various special objects can be performed on the scenes in the image The segmentation is used for feature extraction, and on this basis, pattern matching based on machine learning and deep learning is performed to realize the recognition of the recognition object. Focus on identifying human, vehicle and other recognition objects that have a significant impact on driving safety.

After the recognition object is recognized, other auxiliary means such as machine learning, deep learning or radar can be used to obtain the distance and displacement of the recognition object for the single camera situation, so that more clear prompt information can be given later.

In some embodiments, the processing device 12 may detect the distance between the identified object and the carrier according to the real-time images captured by the first image acquisition device 111 and the second image acquisition device 112 respectively, using the dual-camera ranging principle .

When there are two cameras, the distance and displacement of obstacles can be judged by the principle of dual-camera distance measurement, that is, the principle of binocular parallax distance measurement. Similar to the binocular imaging of human beings, the position of the same object is different in the respective imaging of the scene during the simultaneous imaging of the scene with the two cameras, the position of the near object changes greatly in both, and the object The position difference between the two is small. In this way, you can detect the proximity and speed of recognized objects such as people or cars in relative motion, and then generate prompt information according to preset information processing rules, for example, generate a red warning graphic mark through calculation, including: object distance, Red warning for moving speed and limit distance, superimposed on the physical image, and supplemented by sound warning at the same time.

In some embodiments, the first image acquisition device 111 may be a visible light image acquisition device, and the second image acquisition device 112 may be an infrared image acquisition device.

For example, the visible light image acquisition device may be a visible light camera; the infrared image acquisition device may be an infrared camera, or a camera that supports both visible light and infrared functions, and may be switched to an infrared function when needed.

In some embodiments, in order to achieve a more ideal image collection effect, a wide-angle + telephoto configuration may be used, a visible light camera uses a wide-angle, an infrared camera uses a telephoto, and a black-and-white image captured and parsed by a color image of a visible light camera plus an infrared camera It is the basis of image processing; in this way, you can obtain more differentiated environmental information and enhance the ability of the stereo vision system to cope with night scenes or special weather.

On the basis of two cameras, one more visible light camera can be added to form a three-camera. The two visible light cameras are responsible for binocular parallax distance measurement, and the infrared camera is specifically responsible for dark light imaging.

In some embodiments, the system further includes an infrared emitting device 14 for sending an infrared beam to the identified object. The processing device 12 is further configured to send the infrared beam transmission time according to the infrared transmission device 14, and the second image acquisition device 112 receives the light beam generated by the infrared beam reflected by the identified object, and adopts the flight time (TOF, Time Of Flight) distance measurement method to determine the distance between the identification object and the carrier.

For example, the processing device 12 acquires obstacle depth information according to the time difference between the infrared beam emitted by the infrared emitting device 14 being transmitted to the obstacle and the infrared beam reflected by the second image acquisition device 112 (ie, infrared camera) receiving the obstacle, In addition, in combination with the image information acquired by the first image acquisition device 111 (that is, the visible light camera), the three-dimensional information of the obstacle can be determined more quickly and accurately.

The image projection device 13 is configured to project the output image to a designated area of a preset projection surface.

Here, the preset projection surface may be rear-view mirrors on both sides of the car, that is, the output image may be projected to a designated area of the rear-view mirrors on both sides. Two head-up display systems 10 may be provided on one car, respectively corresponding to the rear-view mirrors on both sides. The two head-up display systems 10 respectively project the blind spot images on both sides and the prompt information to the rearview mirrors on both sides, so that the driver can directly observe the blind spot in the rear vision and get the prompt information, which greatly enhances the driving safety.

In some embodiments, the image projection device 13 may illuminate a digital micromirror device based on digital optical processing (DLP, Digital Processing) technology with a dynamic zoom projection lens and a light source using LED (Light Emitting Diode) DMD, Digital (mirror, Device) realizes the conversion of electrical signals into optical signals.

The head-up display system 10 shown can be placed outside the car as shown in FIG. 3 and can be mechanically fixed to the outer edge of the window and assisted by magnetic attraction to the metal part of the door; or as shown in FIG. 4 can be mechanically fixed to the side car The upper or lower edge of the inside of the window. In the figure, reference numeral 31 represents a projected image, and 32 represents a virtual image of the projected image visually formed when the user views the rearview mirror.

As shown in FIG. 5, taking two cameras as an example, the working process of the head-up display system 10 is explained, including steps 501 to 509.

In step 501, the camera captures real-time image information of the close-up environment on the side of the vehicle body and transmits it to the main control circuit board. The processor of the main control circuit board and the like can perform step 502 and step 503 simultaneously using a parallel processing method.

In step 502, the real-time image is subjected to compression, difference, sharpening, etc. to adapt the real-time image to subsequent projection data transmission processing, etc., and the process proceeds to step 507.

In step 503, the preset target image recognition rules are used to extract the key target feature of the video information and identify the recognition object, and then step 504 is executed.

In step 504, the preset image recognition rules are used to determine the relative motion parameter information such as the distance and speed of the recognized object and the body. If you have a dual camera, you can use the binocular parallax ranging algorithm. If you have an infrared transmitter, you can press TOF to measure the distance. Algorithm, and can estimate the movement trajectory and trend, and then execute step 505.

In step 505, according to the recognition and calculation results, a risk assessment is performed with respect to the distance to the vehicle body and the level is judged, and then step 506 is executed.

In step 506, the AR overlay prompt information is given according to the risk level, including: the calculated information that needs to be displayed directly and the associated warning icon pre-stored in the database retrieved according to the calculation result, etc., which may be a warning color bar and an evasion icon Wait, and then perform step 507.

In step 507, the prompt information and the real-time image processed in step 502 are superimposed, and the superimposed projection image is converted into a projection signal suitable for the image projection device 13 (that is, a projection light machine).

In step 508, the projection signal is pushed to the projection light machine, optically processed and projected.

In step 509, the projection result is presented in the projection area of the exterior mirror of the car.

The head-up display system 10 may further include a distance sensor 15 for measuring relative motion parameter information of the recognized object.

Here, the distance sensor 15 may be an infrared distance sensor, a ranging radar, or the like. Taking infrared distance sensor as an example, infrared distance sensor has limited infrared light power and scattering surface, but infrared distance sensor has a faster response speed, and can be used as a quick predicting tool. It can start immediately after predicting that there is an obstacle near the rear of the body side The infrared emitting device 14 emits infrared light of greater power and a larger scattering surface to the obstacle, and then the second image acquisition device 112 (ie, infrared camera) receives the infrared light reflected by the obstacle to obtain the distance or depth information of the obstacle, and Combined with the color information obtained by the visible light camera, the three-dimensional information of the obstacle can be judged more quickly and accurately. The distance sensor 15 may not be placed on the body of the head-up display system 10, for example, it may be placed at a position closer to the suspicious obstacle on the rear side of the vehicle body, or even a plurality of distance sensors 15 may be provided to improve the accuracy of the distance measurement.

In some embodiments, a projection adjustment device 16 and / or a projection image acquisition device 17 for adjusting the projection image may also be provided. The projection adjustment device 16 is used to adjust the projection orientation of the image projection device 13 under the control of the processing device 12. The projection image collection device 17 is used to collect the projection image generated by the image projection device 13. The processing device 12 is further configured to control the projection adjustment device 16 to adjust the projection orientation of the image projection device 13 according to the projected image, using a preset adjustment rule.

Here, a pan-tilt head driven by a motor, etc. may be used to adjust the projection orientation by adjusting the position of the image projection device 13; or the head-up display system 10 may be directly adjusted by a pan-tilt head or other bracket devices to adjust The effect of projection orientation. The gimbal or other types of brackets can also have a manual adjustment function, which can be adjusted by the user.

The projection image collection device 17 may be a camera or the like, and the projection image collection device 17 may sample the projected image of the image projection device 13. The processing device 12 can control the projection adjustment device 16 to adjust the projection direction according to the actual condition of the projected image. The preset adjustment rule can be set according to the driver position, the projection position, etc., and the projection orientation is adjusted so that the projection image of the image projection device 13 is projected at a preset position. The projection position can be preset in the rearview mirror. Due to the adjustment of the rearview mirror, etc., if the projected image exceeds the preset projection position, the projection orientation can be adjusted to keep the projected image at the preset projection position.

In some embodiments, the projection image acquisition device 17 may use a wide-angle camera, which can simultaneously observe the condition of the turn signal and find that the turn signal is flashing, such as three consecutive flashes. Processing, usually do not need to process real-time images.

In some embodiments, the head-up display system 10 may further include an ambient light sensor 18 for detecting the intensity of ambient light. The processing device 12 is also used to adjust the projection brightness of the image projection device 13 according to the intensity of the ambient light, using a preset brightness adjustment rule. The preset brightness adjustment rule may be set according to the visibility of the projection brightness under different ambient light brightness. Different projection brightness can be set according to different ambient light brightness, so that the projected image can be observed under different ambient light brightness.

In some embodiments, the head-up display system 10 may further include a wireless transceiver device 19 for transmitting control information received from an external terminal to the processing device 12.

In some embodiments, the wireless transceiver 19 may use Bluetooth, wireless (wifi), or a mobile communication network to transmit control information. The user can select the head-up display system 10 in a remote control mode through an external terminal. The external terminal may be a mobile terminal such as a wireless remote controller or a mobile phone.

The external terminal can activate different functions in the head-up display system 10 by setting different working modes, such as the night vision mode, and the image acquisition device 17 can be switched to infrared imaging in the night vision mode. In addition, the external terminal can send instructions to adjust the projection direction.

Take the button-type Bluetooth remote control as an example, you can set the driving, reversing, parking, night vision and other modes. In the night vision mode, the image acquisition device 17 can be switched to infrared camera, and it can automatically switch to night vision mode by light recognition ; Or under the night vision mode there are driving, reversing, parking and other modes; you can also set up microphones and other radio devices, used to transmit the sound of the remote control in the car to the head-up display system 10 speakers.

In addition, the mobile terminal can also obtain real-time images and output images from the head-up display system 10 through wireless transmission.

In some embodiments, as shown in FIG. 2, a fill light device 20 may be provided in the head-up display system 10 to fill the light with a flash or the like when the image acquisition device 11 captures an image. The head-up display system 10 may also be provided with a sound-generating device 21, such as a speaker, etc., which emits different prompt sounds according to different levels of prompt information while projecting. The head-up display system 10 in FIG. 2 further includes an optional battery 22 and an external power supply interface 23, so that different power supplies can be used to power the head-up display system 10.

Combined with the actual use, the following describes the adjustment of the projection orientation that may occur during actual use.

Case 1: When the head-up display system 10 is placed in a car, when projecting to the mirrors on both sides, the projected light will be refracted due to the influence of the side windows. In this case, the projection orientation may need to be adjusted or corrected. In addition, changes in external light can also cause adjustment of the brightness of the projected image. The specific adjustment process is shown in FIG. 6 and includes steps 601 to 608.

In step 601, the projection light beam passes through or does not pass through the window to learn, and determine the position range of the projection area in the captured image of the projection image acquisition device 17 with and without the side window .

At step 602, the ambient light sensor 18 detects the intensity of ambient light.

In step 603, it is compared with a preset light intensity threshold to determine whether it is in a dark environment, if it is in a dark environment, step 604 is performed, otherwise, step 605 is performed.

In step 604, the projection direction is adjusted to a preset dark light angle, the projection area of the external rearview mirror is avoided, and the projection picture is directly projected onto the side window glass to avoid multi-directional scattering. As shown in FIG. 7, at a specific projection angle, the projection image of the image projection device 13 on the side window glass can be observed by the driver.

In step 605, the projection image acquisition device 17 identifies the position of the projection area in the acquired image.

In step 606, the position of the identified projection area in the captured image is compared with the position range obtained through learning. If it does not exceed the position range, step 607 is executed, otherwise step 608 is executed.

In step 607, no adjustment and compensation are made to the projection orientation.

In step 608, the projection orientation is adjusted and compensated so that the projection area falls within the position range.

Case 2: When the side mirrors are adjusted, the head-up display system 10 tracks the projection position in real time and performs real-time adjustment, as shown in FIG. 8, and the specific process includes steps 801 to 804.

In step 801, the projection image acquisition device 17 recognizes the edge of the projected image of the exterior mirror of the vehicle.

In step 802, whether the edge area of the projected image exceeds the side mirror's mirror surface or a predetermined range recognized by the system. If it exceeds, then step 804 is executed; otherwise, step 803 is executed.

In step 803, no projection orientation adjustment is made.

In step 804, the projection orientation is adjusted so that the projected image is projected into the predetermined range of the side mirror, and if the adjustable range is exceeded, the user may be prompted to perform human intervention.

Case three, according to the steering situation of the vehicle, real-time projection is performed, as shown in FIG. 9, including steps 901 to 904.

At step 901, it is determined that the vehicle is turning.

In some embodiments, in addition to communicating with the vehicle control system, judging the turning of the vehicle can also be achieved by using the projection image acquisition device 17 to capture the turn signal, for example, the image captured by the projection image acquisition device 17 has a steering The light flashes, if three consecutive times, it is determined that the vehicle is turning.

In step 902, the processing function of the head-up display system 10 on the steering side is started, and the movement trend analysis is performed for the movement state of the target.

In step 903, target features are extracted and fitted and classified, and high-risk features are retrieved for matching.

In step 904, more data analysis results and warning prompts than normal information can be superimposed on the physical image. Here, the driver can be prompted with projection information and sound information.

In summary, the image acquisition device 11 collects the environmental image on the outside and rear of the vehicle body and sends it to the processing device 12 for image processing and analysis and calculation, and then sends the environmental real video information and the virtual information calculated based on the analysis of the real image data The image projection device 13 is projected onto a partial mirror surface of the exterior mirror of the car. Typical application scenarios, such as pedestrians or other obstacles approaching the body or predicting its movement trajectory to approach or will penetrate into the body, the projection interface gives graphics or voice prompts superimposed on the physical image. In this way, the driver can obtain more information based on the projected image and improve driving safety.

As shown in FIG. 10, the head-up display method provided by an embodiment of the present disclosure includes steps 1001 to 1003.

In step 1001, a real-time image of a preset image acquisition area is captured.

Here, as shown in FIG. 1, a real-time image can be captured by the image acquisition device 11 in the head-up display system 10. The head-up display system 10 may be a single whole, or may be composed of different devices distributed at various positions on the vehicle body. The head-up display system 10 may be installed on a carrier such as an automobile. Similar to other in-vehicle systems, the head-up display system 10 may be powered by a carrier such as an automobile, or it may be powered by the battery of the head-up display system 10 itself.

AR is a combination of real scenes and virtual scenes. Based on the pictures taken by the camera, through computer processing capabilities, the virtual data is superimposed on the real environment, and then the same picture is used to display an interactive mode. The head-up display system 10 can provide driving information to the driver through the AR mode.

Taking the head-up display system 10 as a whole as an example, the detailed internal structure of the head-up display system 10 may be as shown in FIG. 2, which includes an image acquisition device 11, a processing device 12 (for example, a main control circuit board), and an image projection device 13 . The image acquisition device 11 includes a first image acquisition device 111 and a second image acquisition device 112, for example, two cameras; the main control circuit board is also provided with a power management circuit for managing the power of the entire head-up display system 10, This includes management of external power supply and optional built-in battery power supply switching, charging and discharging, and power supply distribution of each device in the head-up display system 10. The main control circuit board may include a processor for data processing, and circuits and interfaces corresponding to various functions such as power supply.

In step 1002, a preset image recognition rule is used to identify the recognition object in the real-time image and / or relative motion parameter information of the recognition object and the carrier of the head-up display system 10, and according to the recognition object and / or Or the relative motion parameter information, using preset information processing rules to generate prompt information, and merging the prompt information and the real-time image into an output image.

In some embodiments, the step 1002 may be performed by the processing device 12 in the head-up display system 10 as described in FIG. 1. The preset image recognition rule may be set according to the image acquisition device 11, when the image acquisition device 11 is a camera, the real-time image captured by a single camera may be identified; the image acquisition device 11 is two or two When there are more than one camera, the real-time images captured by the two cameras can be identified through methods such as binocular parallax ranging. Recognition of recognized objects can be achieved by methods such as model training and deep learning. The relative motion parameter may include a relative distance and a relative motion speed of the identification object and the carrier. The relative motion speed may be determined by dividing the relative distance between two time points by the time interval between the two time points. When the image acquisition device 11 is two or more cameras, a real-time image of one camera may be pre-selected and combined as an output image according to the preset information processing rule.

When there are two cameras, the distance and displacement of obstacles can be judged by the principle of dual-camera distance measurement, that is, the principle of binocular parallax distance measurement. Similar to the binocular imaging of human beings, the position of the same object is different in the respective imaging of the scene during the simultaneous imaging of the scene with the two cameras, the position of the near object varies greatly between the two, and the object in the distance The position difference between the two is small. In this way, you can detect the proximity and speed of recognized objects such as people or cars in relative motion, and then generate prompt information according to preset information processing rules, for example, generate a red warning graphic mark through calculation, including: object distance, Red warning for moving speed and limit distance, superimposed on the physical image, and supplemented by sound warning at the same time.

In some embodiments, in order to achieve a more ideal image collection effect, a wide-angle + telephoto configuration may be used, a visible light camera uses a wide angle, an infrared camera uses a telephoto, and a black-and-white image captured and parsed by a color image of a visible light camera plus an infrared camera It is the basis of image processing; in this way, you can obtain more differentiated environmental information and enhance the ability of the stereo vision system to cope with night scenes or special weather.

In some embodiments, the system further includes an infrared emitting device 14 for sending an infrared beam to the identified object. The processing device 12 is further configured to transmit the infrared beam according to the infrared beam emitting time of the infrared emitting device 14, and the second image acquisition device 112 receives the infrared beam received by the object reflected by the beam of the receiving time, using TOF measurement The distance method determines the distance between the identification object and the carrier.

In step 1003, the output image is projected to a designated area of a preset projection surface.

Here, as shown in FIG. 1, the image projection device 13 in the head-up display system 10 may perform projection. The preset projection surface may be rear-view mirrors on both sides of the car, that is, the output image may be projected to a designated area of the rear-view mirrors on both sides. Two head-up display systems 10 may be provided on one car, respectively corresponding to the rear-view mirrors on both sides. The two head-up display systems 10 respectively project the blind spot images on both sides and the prompt information to the rearview mirrors on both sides, so that the driver can directly observe the blind spot in the rear vision and get the prompt information, which greatly enhances the driving safety.

In some embodiments, the image projection device 13 may use a dynamic zoom projection lens and an LED light source to illuminate a DMD based on DLP technology to achieve conversion of electrical signals into optical signals.

The head-up display system 10 shown can be placed outside the car as shown in FIG. 3 and can be mechanically fixed to the outer edge of the window and supplemented by magnetic force to the metal part of the door; as shown in FIG. The upper or lower edge of the inside of the window. In the figure, reference numeral 31 represents a projected image, and 32 represents a virtual image of the projected image visually formed when the user views the rearview mirror.

Here, the distance sensor 15 may be an infrared distance sensor, a ranging radar, or the like. Taking infrared distance sensor as an example, infrared distance sensor has limited infrared light power and scattering surface, but infrared distance sensor has a faster response speed, and can be used as a quick predicting tool. It can start immediately after predicting that there is an obstacle near the rear of the body side The infrared emitting device 14 emits infrared light of greater power and a larger scattering surface to the obstacle, and then the second image acquisition device 112 (ie, infrared camera) receives the infrared light reflected by the obstacle to obtain the distance or depth information of the obstacle, and Combined with the color information obtained by the visible light camera, the three-dimensional information of the obstacle can be judged more quickly and accurately. The distance sensor 15 may not be placed on the body of the head-up display system 10, for example, it may be placed at a position closer to the suspicious obstacle on the rear side of the vehicle body, or even a plurality of distance sensors 15 may be provided to improve the accuracy of the ranging.

In some embodiments, a projection adjustment device 16 and / or a projection image acquisition device 17 for adjusting the projection image may also be provided. The projection adjustment device 16 is used to adjust the projection orientation of the image projection device 13 under the control of the processing device 12. The projection image acquisition device 17 is used to acquire the projection image generated by the image projection device 13. The processing device 12 is further configured to control the projection adjustment device 16 to adjust the projection orientation of the image projection device 13 according to the projected image, using a preset adjustment rule.

Case 2: When the side mirrors are adjusted, the head-up display system 10 tracks the projection position in real time and performs real-time adjustment, as shown in FIG. 8, and the specific process includes steps 801 to 804. :

In step 803, no projection orientation adjustment is made.

At step 901, it is determined that the vehicle is turning.

In summary, the image acquisition device 11 collects the environmental image on the outside and rear of the vehicle body and sends it to the processing device 12 for image processing and analysis and calculation, and then sends the environmental real video information and the virtual information calculated based on the analysis of the real image data The image projection device 13 is projected onto a partial mirror surface of the exterior mirror of the car. Typical application scenarios, such as pedestrians or other obstacles approaching the body or predicting their trajectory to approach or will penetrate the body, the projection interface gives graphics or voice prompts superimposed on the physical image. In this way, the driver can obtain more information based on the projected image and improve driving safety.

The above are only the best embodiments of the present disclosure and are not intended to limit the scope of protection of the present disclosure. Any modification, equivalent replacement and improvement made within the spirit and principle of the present disclosure should be included Within the scope of this disclosure.

Claims

A head-up display system includes: an image acquisition device, a processing device, and an image projection device; wherein,

The image acquisition device is used to capture a real-time image of a preset image acquisition area;

The processing device is used to recognize the identification object in the real-time image and / or the relative motion parameter information of the identification object and the carrier of the head-up display system using preset image recognition rules, and is used to identify The object and / or the relative motion parameter information, using preset information processing rules to generate prompt information, and merging the prompt information and the real-time image into an output image;

The image projection device is used to project the output image to a designated area of a preset projection surface.
The system according to claim 1, wherein the image acquisition device includes a first image acquisition device and a second image acquisition device;

The processing device is used to detect the distance between the identification object and the carrier according to the real-time images captured by the first image acquisition device and the second image acquisition device respectively, using the principle of dual-camera ranging.
The system according to claim 2, wherein

The first image acquisition device is a visible light image acquisition device, and the second image acquisition device is an infrared image acquisition device.
The system according to claim 3, further comprising an infrared emitting device for sending an infrared beam to the identified object;

The processing device is further used for transmitting the infrared beam according to the transmission time of the infrared beam and the second image acquisition device receiving the light beam generated by the infrared beam reflected by the identification object The time TOF distance measurement method determines the distance between the identification object and the carrier.
The system according to any one of claims 1 to 4, further comprising a projection adjustment device and / or a projection image acquisition device, wherein,

The projection adjustment device is used to adjust the projection orientation of the image projection device under the control of the processing device;

The projection image acquisition device is used to capture the projection image generated by the image projection device;

The processing device is further configured to control the projection adjustment device to adjust the projection orientation of the image projection device using preset adjustment rules according to the projected image.
The system according to any one of claims 1 to 4, further comprising an ambient light sensor for detecting the intensity of ambient light;

The processing device is further used for adjusting the projection brightness of the image projection device according to the intensity of the ambient light and using preset adjustment rules.
A head-up display method, including:

Capture real-time images of preset image collection areas;

A preset image recognition rule is used to identify the relative motion parameter information of the recognition object in the real-time image and / or the recognition object and the head-up display system carrier, and according to the recognition object and / or the relative motion parameter information , Using preset information processing rules to generate prompt information, and merging the prompt information and the real-time image into an output image; and

The output image is projected to the designated area of the preset projection surface.
The method according to claim 7, wherein the step of using a preset image recognition rule to recognize the recognition object in the real-time image and / or the relative motion parameter information of the recognition object and the carrier of the head-up display system includes :

According to the real-time images captured by the first image acquisition device and the second image acquisition device respectively, the distance between the identification object and the carrier is detected by using the principle of dual-camera ranging.
The method according to claim 8, wherein the first image acquisition device captures a real-time image of visible light, and the infrared image real-time image captured by the second image acquisition device.
The method of claim 9, further comprising:

Sending an infrared beam to the identified object;

According to the transmission time of transmitting the infrared beam and the receiving time of the light beam generated by the second image acquisition device receiving the infrared beam reflected by the identification object, a TOF ranging method is used to determine Carrier distance.
The method according to any one of claims 7 to 10, further comprising:

Use preset adjustment rules to adjust the projection orientation.
The method according to any one of claims 7 to 10, further comprising:

According to the intensity of ambient light, the preset adjustment rules are adopted to adjust the projection brightness.
An automobile comprising a body and the head-up display system according to any one of claims 1 to 6.