WO2021139818A1 - Multi-level imaging system, head-up display, transportation tool, and multi-level imaging method - Google Patents

Abstract

A multi-level imaging system, a head-up display, a transportation tool, and a multi-level imaging method. The multi-level imaging system has an eye box area and comprises an electronic control unit and a head-up display. The electronic control unit is communicationally connected to the head-up display. The electronic control unit is used for obtaining driving information and sending the driving information to the head-up display. The head-up display is configured to emit multiple paths of light to respectively display multiple layers of images to display the driving information, and the distances between at least some of the multiple layers of images and the eye box area are not equal.

Description

Multi-level imaging system, head-up display, vehicle, and multi-level imaging method

This application claims the priority of Chinese patent application Nos. 202010029234.4 and 202010029223.6 filed on January 10, 2020, and the contents disclosed in the above-mentioned Chinese patent applications are cited here in full as a part of this application.

Technical field

The embodiments of the present disclosure relate to a multi-level imaging system, a head-up display, a vehicle, and a multi-level imaging method.

Background technique

At present, the Augmented Reality Head Up Display (AR-HUD) is one of the Head Up Display (HUD), which can achieve a good visual experience.

The image projected by the AR-HUD needs to be integrated with the scene in the real environment (such as a car or an object). For example, the direction indicator arrow needs to be accurately integrated with the road to achieve a good visual effect.

Summary of the invention

At least one embodiment of the present disclosure provides a multi-level imaging system having an eye box area and including: an electronic control unit and a head-up display. The electronic control unit is communicatively connected with the head-up display; the electronic control unit is used to obtain driving information and send the driving information to the head-up display; the head-up display is configured to emit multiple lights to display separately A multi-layer image is used to display the driving information, and at least a part of the multi-layer image has a different distance from the eye box area.

In some examples, the head-up display includes a first sub-head-up display, and the first sub-head-up display is an augmented reality head-up display.

In some examples, the augmented reality head-up display is configured to display a single-layer image; or the augmented reality head-up display is configured to emit at least two of the multiple light rays to display at least one of the multi-layer images Two-layer image.

In some examples, the driving information includes driving information and environmental information.

In some examples, the multi-level imaging system further includes an imaging window, the imaging window is a windshield of a vehicle, the eye box area is an eye box area of the vehicle; the windshield is used to The side of the windshield away from the head-up display presents a multi-layer image formed by the multi-path light with a different distance from the eye box area, and the light forming any layer of the multi-layer image can be Enter the eye box area.

In some examples, the multi-level imaging system further includes a data acquisition device configured to acquire environmental information around the vehicle and send the acquired environmental information to the electronic control unit.

In some examples, the multi-level imaging system is an imaging system for vehicles.

In some examples, the augmented reality head-up display includes an image source, a flat mirror, and a curved mirror. The light emitted by the image source is reflected by the curved mirror and then enters the flat mirror, and is reflected on the flat mirror and then exits. . Alternatively, the head-up display includes a curved mirror and at least two image sources; the at least two image sources are used to respectively emit the at least two paths of light; the at least two paths of light are respectively emitted from the image sources of the respective paths of light. The length of the propagation path between the curved mirror and the curved mirror is different; the curved mirror is used to reflect the at least two rays of light incident on the curved mirror.

In some examples, at least one of the at least two image sources included in the head-up display is located at a position close to or a focal plane of the curved mirror.

In some examples, the at least two image sources included in the head-up display include a first image source and a second image source, the head-up display further includes a first transflective film; the first image source is configured to emit For the first light, the second image source is configured to emit a second light; the first transflective film is configured to transmit the first light and reflect the second light. Alternatively, the at least two image sources included in the head-up display include a first image source, a second image source, and a third image source, and the augmented reality head-up display further includes a first transflective film and a second transflective film The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light; the first transflective film is configured to transmit The first light and the second light and the third light are reflected; the second transflective film is configured to reflect the third light and transmit the first light. Alternatively, the at least two image sources include a first image source, a second image source, and a third image source, and the head-up display further includes a first transflective film and a second transflective film; the first image source configuration In order to emit the first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light; the first transflective film is configured to transmit the first light and the The third light and the second light are reflected; the second transflective film is configured to transmit the first light and reflect the third light. Alternatively, the at least two image sources include a first image source, a second image source, and a third image source, and the head-up display further includes a first transflective film, a second transflective film, and a first mirror; The first mirror includes a flat mirror; the first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light; The first transflective film is configured to transmit the second light and reflect the first light; the second transflective film is configured to transmit the second light and reflect the third light; the plane mirror is configured To reflect the first light, the second light, and the third light.

In some examples, the at least two image sources include a first image source and a second image source, the augmented reality head-up display further includes a first mirror; the first image source is configured to emit a first light, so The second image source is configured to emit a second light; the first reflector is configured to reflect the first light and the second light; the first reflector includes a flat reflector; the flat reflector The at least two light rays emitted by the at least two image sources are reflected to the curved mirror. Alternatively, the at least two image sources include a first image source and a second image source, the augmented reality head-up display further includes a first mirror and a second mirror; the first image source is configured to emit a first light , The second image source is configured to emit a second light; the first reflector is configured to reflect the first light; the second reflector is configured to reflect the second light.

In some examples, the electronic control unit is further configured to send information identifying the external target to the first sub head-up display when it is determined that the external target needs to be identified after processing the driving information; The image displayed by the first head-up display matches the distance of the external target and merges with the external target.

In some examples, the head-up display further includes a second sub-head-up display, the horizontal viewing angle of the second sub-head-up display is not less than 15 degrees, and the vertical viewing angle of the second sub-head-up display is not less than 15 degrees. Less than 5 degrees.

In some examples, the second head-up display includes a projection device and a light control device; the projection device is used to emit light that can be incident on the light control device; the light control device is used to change the projection device The exit direction of the emitted light so that the horizontal field angle of the second sub-head-up display is not less than 15 degrees and the vertical field angle is not less than 5 degrees.

In some examples, the light control device includes: a light converging unit and a diffusing element; the light converging unit is used to reflect the light incident on the light control device in a direction opposite to the incident direction of the light. The dispersion element; the dispersion element diffuses the incident light and forms a beam used to form a spot.

In some examples, the light converging unit includes: an inverted triangular pyramid microstructure, a supporting structure, and a substrate layer; the inverted triangular pyramid microstructure and the supporting structure are disposed on the substrate layer, and the inverted triangular pyramid microstructure The structure is located on the side of the support structure away from the substrate layer; the refractive index of the inverted triangular pyramid microstructure is greater than the refractive index of the support structure; the inverted triangular pyramid microstructure is configured to incident the incident light The light whose angle is greater than the critical angle is reflected to the dispersion element in a direction opposite to the incident direction of the light in a manner of total reflection.

In some examples, the light condensing unit includes: a light condensing layer, an isolation layer, a planar reflective layer, and a substrate sequentially arranged in the incident direction of the light; the planar reflective layer is located at the focal point of the light converging layer. On a plane; the light condensing layer and the plane reflective layer are made of different metamaterials; the light condensing layer is used to converge the incident light to the plane reflective layer by changing the phase of the incident light And used to reflect the light reflected by the flat reflective layer to the dispersion element in a direction opposite to the direction in which the light enters the light converging layer; the planar reflective layer is used to change the convergence of the light converging layer The phase of the light, and the light after the phase change is reflected to the light converging layer.

In some examples, for the same external target, the electronic control unit is further configured to control the first sub head-up display to display the first information of the external target according to the driving information, and control the second sub head-up The display displays the second information of the external target, and the content included in the first information and the second information is at least partially different.

In some examples, for multiple external targets, the electronic control unit is further configured to control the first sub-head-up display to display the first external target that meets the first selection condition, and control the second sub-head-up display to display the second selection The second external target of the condition, the first condition and the second condition are different, and the first external target and the second external target are different.

In some examples, the multi-level imaging system is a multi-level imaging system for a vehicle, and the driving information includes: speed information of the vehicle; and distance information between the vehicle and surrounding vehicles, wherein, The surrounding vehicles include: a front vehicle located in front of the vehicle, a rear vehicle located behind the vehicle, a left vehicle located on the left side of the vehicle, and a right vehicle located on the right side of the vehicle. At least one; the electronic control unit is also configured to: obtain a first distance between the vehicle and the front vehicle, a second distance between the vehicle and the rear vehicle, the At least one of the third distance between the vehicle and the vehicle on the left and the fourth distance between the vehicle and the vehicle on the right; determining the vehicle according to the speed information of the vehicle A first safe distance threshold to the vehicle in front; when the first distance is less than or equal to the determined first safe distance threshold, first warning information is generated, and the first warning information is sent to all The head-up display is displayed; when the second distance is less than or equal to the second safe distance threshold, second warning information is generated, and the second warning information is sent to the head-up display for display; wherein, the second safe distance The threshold is used to indicate the safe driving distance between the vehicle and the rear vehicle; when the third distance and/or the fourth distance is less than or equal to the third safe distance threshold, the third warning information is generated and the all The third warning information is sent to a head-up display for display; wherein, the third safe distance threshold is used to indicate a safe driving distance between the vehicle and the side vehicle.

In some examples, the target image source may be selected according to the relationship between the image distance and the first distance. For example, the first distance may be compared with the multiple image distances of the multiple image sources to select the target image source from the multiple image distances. Select the target image distance.

In some examples, the electronic control unit is configured to: when the first distance is less than or equal to the determined first safe distance threshold, generate first warning information; Path image; according to the acquired path image between the vehicle and the preceding vehicle, identify the path between the vehicle and the preceding vehicle; select a target from the multiple image sources of the head-up display Image source; use the first color identification to generate a first path identification instruction, and send the generated first path identification instruction to the target image source in the head-up display, and control the target image source to connect the vehicle with the The path between the preceding vehicles is displayed in the color corresponding to the first color mark, and the first warning information is sent to the target image source for display; when the first distance is greater than the first safe distance threshold When the target image source is selected from the multiple image sources of the head-up display according to the first distance; a second path identification instruction is generated using a second color identification, and the generated second path identification instruction is generated Send to the target image source, control the target image source to display the path between the vehicle and the preceding vehicle in the color corresponding to the second color mark; when the first distance is less than the braking operation When the length threshold is longer and when the first distance is less than or equal to the determined first safe distance threshold and the duration is greater than the first preset duration threshold, a braking instruction is generated; the generated braking instruction is used to control the vehicle Perform a braking operation.

In some examples, the multi-level imaging system is a multi-level imaging system for a vehicle, and the electronic control unit is further configured to obtain the surrounding environment of the vehicle when the location of the vehicle is a crowded area Image; when it is determined that there is a non-motorized vehicle around the vehicle based on the surrounding environment image, determine the distance between the non-motorized vehicle and the vehicle; when the distance between the non-motorized vehicle and the vehicle is less than When the collision safety distance threshold is reached, collision warning information is generated, and the collision warning information is sent to the head-up display for display.

In some examples, the electronic control unit is further configured to generate collision warning information when the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold; according to the location of the non-motor vehicle and According to the distance between the non-motor vehicle and the vehicle, determine the first size of the non-motor vehicle indicator frame fused with the non-motor vehicle, and determine the non-motor vehicle outline based on the surrounding environment image The first shape of the non-motor vehicle indicator frame; select the target image source from the multiple image sources of the head-up display; use the third color identification, the first size and the first shape of the non-motor vehicle indicator frame to generate the non-motor vehicle The identification instruction, through the non-motor vehicle identification instruction, the target image source is controlled to use the color corresponding to the third color identification to generate the non-motor vehicle instruction frame according to the determined first size and first shape, and according to the The location of the non-motor vehicle integrates the non-motor vehicle indication frame and the non-motor vehicle, identifies the non-motor vehicle, and sends the collision warning information to a target image source for display.

In some examples, the electronic control unit is further configured to determine the location of the non-motor vehicle when the distance between the non-motor vehicle and the vehicle is greater than the collision safety distance threshold; When the distance between the motor vehicle and the vehicle is less than or equal to the warning distance threshold, the target image source is selected from the head-up display according to the distance between the non-motor vehicle and the vehicle; the target image source is controlled to use the fourth color The color corresponding to the mark generates a warning graphic at the location of the non-motor vehicle; wherein the warning graphic is fused with the non-motor vehicle.

In some examples, the electronic control unit is further configured to: when the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, obtain a non-motor vehicle image and determine the non-motor vehicle The current location of the motor vehicle; according to the current location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle, determine the second size of the non-motor vehicle indicator frame fused with the non-motor vehicle , And based on the non-motor vehicle outline in the non-motor vehicle image, determine the second shape of the non-motor vehicle indicator frame; select the target image source from the head-up display according to the distance between the non-motor vehicle and the vehicle Use the fifth color identification, the second size and the second shape of the non-motor vehicle indicator box to generate a non-motor vehicle identification instruction, and control the target image source to use the fifth color identification corresponding to the non-motor vehicle identification instruction Color, generate a non-motor vehicle indication frame according to the determined second size and second shape, and merge the non-motor vehicle indication frame and the non-motor vehicle based on the current location of the non-motor vehicle.

In some examples, the vehicle is a vehicle, and the driving information further includes: weather information and status information of the road on which the vehicle is located; the electronic control unit is further configured to: obtain weather information and/or the The condition information of the road where the vehicle is located; when it is determined that there are potentially dangerous driving targets around the vehicle based on the weather information and/or the condition information of the road where the vehicle is located, warning information is generated, and the generated warning information is sent to the header Display for display.

In some examples, the potentially dangerous driving target includes: bad weather conditions; the electronic control unit is configured to: when it is determined that the road where the vehicle is located has bad weather conditions, obtain the bad weather road image of the road where the vehicle is located and Generate warning information; process the bad weather road image to determine the brightness value of the road where the vehicle is located; when the brightness value is less than the brightness threshold, generate a high beam turn-on prompt message, through the head-up display (for example, the The second sub-head-up display in the head-up display) displays the prompt information of turning on the high beam, and determines the path contour and path position of the road where the vehicle is located from the bad weather road image; according to the path position, from the Select the target image source from the multiple image sources of the head-up display (for example, the first sub-head-up display in the head-up display); use the sixth color to fill in the path contour of the road where the vehicle is located to obtain the path of the road where the vehicle is located Contour map; according to the path position of the road where the vehicle is located, the path contour map is projected onto the road where the vehicle is located through the target image source, so that the path contour map merges with the road where the vehicle is located, and at the same time The warning information is sent to the head-up display (for example, the second sub-head-up display) for display.

In some examples, the potentially dangerous driving target includes: a dangerous situation in front of the road where the vehicle is located; the electronic control unit is configured to: when the condition information of the road where the vehicle is When the distance between the road sections in the dangerous condition is less than the road condition display distance threshold, the road condition image of the road where the vehicle is located is acquired and warning information is generated; wherein, the road condition image includes: a road condition that occurs on the road where the vehicle is located Determine the road section contour and road section position of the road section where the dangerous condition occurs from the road condition image, and select the target image source from the multiple image sources on the head-up display according to the road section position; Use the seventh color to fill in the road section contour of the road section where the dangerous condition occurs to obtain the state contour map of the road section where the road condition occurs; project the state contour map to the road section where the road condition occurs through the target image source On the road section of the condition, the condition contour map is merged with the road section where the road condition occurs.

In some examples, the multi-level imaging system is a multi-level imaging system for a vehicle, and the driving information includes: operation information, driving speed, acceleration, vehicle status information, and navigation information of the vehicle; The control unit is also configured to: when generating operation advice information of the vehicle according to the navigation information, the driving speed, acceleration, and vehicle status information, collect the vehicle operation video of the observer, and pass The head-up display displays the operation suggestion information; when it is determined that the operation corresponding to the operation suggestion information is not detected within the specified period of operation, dangerous driving information is generated, and the dangerous driving information is displayed through the head-up display, And perform a decrement operation on the driving vehicle index; when it is determined that the operation corresponding to the operation advice information is detected within the specified time of operation, the driving vehicle index is increased; when it is determined that the vehicle has not been operated for longer than the parking time When the threshold is reached, stop collecting vehicle operation videos, and send the driving vehicle index and the collected vehicle operation videos to the head-up display for display.

In some examples, the electronic control unit is configured to: form display data according to the driving information, process the display data according to the driver’s personal information, and control the head-up display to send out according to the processed display data The light irradiates the imaging window.

In some examples, the processing includes at least one of the following: color saturation enhancement processing of the image, color conversion processing of the image, and shape conversion processing of the image.

In some examples, the electronic control unit is further configured to identify the driver's identity information, and determine the driver's personal information based on the identity information.

At least one embodiment of the present disclosure further provides a head-up display, which includes a curved mirror, a first image source, and a second image source. The first image source is used to emit a first path of light incident to the curved mirror, and the second image source is used to emit a second path of light incident to the curved mirror; There is a first propagation path between the first image source and the curved mirror, and the second path of light has a second propagation path between the second image source and the curved mirror; the first propagation path The length of the second propagation path is different from that, so that the first image displayed by the head-up display through the first path of light and the second image displayed through the second path of light have different image distances.

At least one embodiment of the present disclosure further provides a head-up display, which includes a curved mirror and at least one image source. The curved mirror is configured to reflect light emitted by the at least one image source, and the at least one image source is close to or located at the focal plane of the curved mirror.

At least one embodiment of the present disclosure further provides a vehicle, including the multi-level imaging system or the head-up display provided in any of the foregoing embodiments.

In some examples, the vehicle further includes an imaging window, the head-up display further includes a second sub-head-up display, and the second sub-head-up display includes a projection device and a light control device; the projection device is used to emit The light to the light control device; the light control device is used to change the exit direction of the light emitted by the projection equipment, so that the horizontal field angle of the second sub-head-up display is not less than 15 degrees and the vertical direction The field of view angle is not less than 5 degrees; the projection equipment is installed in the vehicle; the light control device is installed on the surface of the center console of the vehicle opposite to the imaging window.

In some examples, the electronic control unit is configured to: according to the driving information, control the second head-up display to emit light to the imaging window, so that the light emitted by the second head-up display shows The area of the imaging area of the image at the imaging window is larger than the area of the imaging area of the remaining images in the plurality of images.

At least one embodiment of the present disclosure further provides an imaging method. The multi-level imaging system includes an eye box area and a head-up display. The imaging method includes: acquiring driving information and sending it to the head-up display; and controlling according to the driving information The head-up display emits multiple rays of light to respectively display multiple layers of images to display the driving information, and at least part of the multiple layers of images are at different distances from the eye box area.

In some examples, the driving information includes driving information and environmental information; controlling the head-up display to emit multiple lights according to the driving information to display multiple layers of images to display the driving information includes: displaying according to the driving information A first image and a second image displayed according to the environmental information, wherein the distance between the first image and the eye box area is smaller than that of the second image.

In some examples, the driving information includes distance information between the vehicle and surrounding vehicles; controlling the head-up display to emit multiple lights according to the driving information to display multiple layers of images to display the driving information includes: The driving information displays a first image and a second image, wherein the first image displays the distance between the vehicle and the side vehicle, and the second image displays the distance between the vehicle and the vehicle in front; The distance between the first image and the eye box area is smaller than that of the second image.

In some examples, controlling the head-up display to emit multiple lights according to the driving information to display multiple layers of images to display the driving information includes: generating prompt information according to the driving information, and controlling the driving information according to the prompt information. The head-up display display emits at least one line of light to the windshield to display at least one of the at least two images.

At least one embodiment of the present disclosure further provides a multi-level imaging method, including: acquiring driving information of a vehicle; controlling a head-up display to emit at least two lights to the windshield of the vehicle according to the driving information to display and At least two images with unequal distances between the eye box area of the vehicle.

In some examples, the driving information includes: vehicle speed information and distance information between the vehicle and surrounding vehicles. The surrounding vehicles include: a front vehicle located in front of the vehicle, a rear vehicle located behind the vehicle, And side vehicles located on both sides of the vehicle. Sending at least two rays of light to the windshield of the vehicle according to the driving information to respectively display at least two images with different distances from the eye box area of the vehicle includes: acquiring the vehicle and the front vehicle respectively The first distance to the vehicle behind, the second distance to the vehicle behind, and the third distance and the fourth distance to the vehicle on both sides of the side; determine the vehicle and the front distance according to the speed information of the vehicle The first safe distance threshold of the vehicle; when the first distance is less than or equal to the first safe distance threshold, first warning information is generated, and the head-up display is controlled to emit at least two lights to The windshield of the vehicle to respectively display at least one of the at least two images with different distances from the eye box area of the vehicle; when the second distance is less than or equal to a second safety distance threshold, a second Warning information, and controlling the head-up display to emit at least two lights to the windshield of the vehicle according to the first warning information to respectively display at least two images with different distances from the eye box area of the vehicle At least one of; wherein the second safe distance threshold is used to indicate the safe driving distance of the vehicle and the rear vehicle; when the third distance and/or the fourth distance is less than or equal to the third safe distance When the threshold is set, the third warning information is generated, and the head-up display is controlled to emit at least two lights to the windshield of the vehicle according to the first warning information, so as to respectively display the unequal distances from the eye box area of the vehicle At least one of the at least two images of, wherein the third safe distance threshold is used to indicate the safe driving distance of the vehicle and the side vehicle.

In some examples, when the first distance is less than or equal to the first safe distance threshold, first warning information is generated, and the head-up display is controlled to emit at least two lights to the vehicle according to the first warning information To display at least one of the at least two images with different distances from the eye box area of the vehicle, including: acquiring a path image between the vehicle and the vehicle in front; according to the acquired The path image between the vehicle and the preceding vehicle is used to identify the path between the vehicle and the preceding vehicle; the image distance matching the first distance is used as the target image distance, and the Among the multiple image sources of the augmented reality head-up display, the image source matching the target image distance is selected as the target image source; wherein, the image distance is the difference between the image formed by the image source on the curved mirror and the curved mirror Use the first color mark to generate a first path identification instruction, and send the generated first path identification instruction to the target image source in the augmented reality head-up display, and control the target image source to transfer the The path between the vehicle and the preceding vehicle is displayed in the color corresponding to the first color mark, and the first warning information is sent to the target image source for display.

In some examples, emitting at least two rays of light to the windshield of the vehicle according to the driving information to respectively display at least two images with different distances from the eye box area of the vehicle further includes: When a distance is greater than the first safe distance threshold, a target image source is selected from at least two image sources of the augmented reality head-up display according to the first distance; a second path identification instruction is generated using the second color identification, And send the generated second path identification instruction to the target image source, and control the target image source to display the path between the vehicle and the preceding vehicle in the color corresponding to the second color identification To form at least one of the at least two images.

In some examples, the imaging method further includes: when the first distance is less than a braking operation length threshold and the first distance is less than or equal to the determined first safe distance threshold for a duration greater than a first preset duration threshold When the time, a braking instruction is generated; the generated braking instruction is used to control the vehicle to perform a braking operation.

In some examples, emitting at least two rays of light to the windshield of the vehicle according to the driving information to respectively display at least two images with different distances from the eye box area of the vehicle includes: when it is determined that the vehicle When the location of is in a crowded area, the surrounding environment image of the vehicle is acquired; when it is determined that there is a non-motorized vehicle around the vehicle according to the surrounding environment image, the distance between the non-motorized vehicle and the vehicle is determined When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, generate collision warning information, and control the head-up display to emit at least two lights to the vehicle's block according to the collision warning information A windshield to respectively display at least one of at least two images with different distances from the eye box area of the vehicle.

In some examples, when the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, collision warning information is generated, and the heads-up display is controlled to emit at least two lights to all areas according to the collision warning information. The windshield of the vehicle to respectively display at least one of the at least two images with different distances from the eye box area of the vehicle includes: generating collision warning information; according to the location of the non-motor vehicle and the non-motor vehicle The distance between the motor vehicle and the vehicle is determined to determine the first size of the non-motor vehicle indicator frame fused with the non-motor vehicle, and based on the non-motor vehicle outline in the surrounding environment image, the size of the non-motor vehicle indicator frame is determined The first shape; the image distance that matches the distance between the non-motor vehicle and the vehicle is used as the target image distance, and the image source that matches the target image distance is selected from the multiple image sources of the augmented reality head-up display The image source is used as the target image source; wherein, the image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror; using the third color mark, the first size and the first size of the non-motor vehicle indicator frame A shape generates a non-motor vehicle identification instruction, and the target image source is controlled by the non-motor vehicle identification instruction to use the color corresponding to the third color identification to generate the non-motor vehicle according to the determined first size and first shape Indicating frame, and integrate the non-motor vehicle indication frame with the non-motor vehicle according to the location of the non-motor vehicle, identify the non-motor vehicle, and send the collision warning information to the target at the same time Image source for display.

In some examples, emitting at least two rays of light to the windshield of the vehicle according to the driving information to respectively display at least two images with different distances from the eye box area of the vehicle further includes: When the distance between the motor vehicle and the vehicle is greater than the collision safety distance threshold and less than or equal to the warning distance threshold, select the target image source from the augmented reality head-up display according to the distance between the non-motor vehicle and the vehicle; control the use of the target image source The color corresponding to the fourth color mark generates a warning graphic at the location of the non-motor vehicle; wherein the warning graphic is fused with the non-motor vehicle.

In some examples, sending at least two rays of light to the windshield of the vehicle according to the driving information to display at least two images with different distances from the eye box area of the vehicle further includes: when it is determined that the When the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, the non-motor vehicle image is acquired, and the current location of the non-motor vehicle is determined; according to the current location of the non-motor vehicle and the The distance between the non-motor vehicle and the vehicle is determined, the second size of the non-motor vehicle indicator frame fused with the non-motor vehicle is determined, and the non-motor vehicle indicator is determined based on the non-motor vehicle outline in the non-motor vehicle image The second shape of the frame; select the target image source from the augmented reality head-up display according to the distance between the non-motorized vehicle and the vehicle; use the fifth color mark, the second size and the second shape of the non-motorized vehicle indicator frame to generate Non-motor vehicle identification instruction, through the non-motor vehicle identification instruction, the target image source is controlled to use the color corresponding to the fifth color identification to generate a non-motor vehicle instruction frame according to the determined second size and second shape, and based on The current location of the non-motor vehicle integrates the non-motor vehicle indicator frame and the non-motor vehicle.

In some examples, the environmental information further includes weather information and condition information of the road where the vehicle is located. Controlling the head-up display to emit at least two lights to the windshield of the vehicle according to the driving information to display at least two images with different distances from the eye box area of the vehicle includes: obtaining weather information and/or The condition information of the road where the vehicle is located; when it is determined that there are potentially dangerous driving targets around the vehicle based on the weather information and/or the condition information of the road where the vehicle is located, warning information is generated, and the generated warning information is sent to Augmented reality head-up display for display.

In some examples, the potentially dangerous target for driving includes: severe weather conditions; when it is determined that there is a potentially dangerous target for driving around the vehicle according to the weather information and/or the condition information of the road where the vehicle is located, a warning is generated Information, and sending the generated warning information to the augmented reality head-up display for display includes: generating warning information when it is determined that there are potentially dangerous driving targets around the vehicle based on the weather information and/or the condition information of the road where the vehicle is located , And sending the generated warning information to the augmented reality head-up display for display, including: when it is determined that the road where the vehicle is located has severe weather conditions, acquiring the severe weather road image of the road where the vehicle is located and generating warning information; The bad weather road image is processed to determine the brightness value of the road where the vehicle is located; when the brightness value is less than the brightness threshold, a high-beam turn-on prompt message is generated, and the high-beam turn-on prompt is displayed through the augmented reality head-up display Information, and determine the path profile and path position of the road where the vehicle is located from the bad weather road image; determine the distance between the road where the vehicle is located and the eye box area according to the path position; The image distance matching the distance between the road and the eye box area is used as the target image distance, and the image source matching the target image distance is selected from the multiple image sources of the augmented reality head-up display as the target image Source; wherein, the image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror; the sixth color is used to fill the path contour of the road where the vehicle is located to obtain the distance of the road where the vehicle is located Path contour map; according to the path position of the road where the vehicle is located, the path contour map is projected onto the road where the vehicle is located through the target image source, so that the path contour map merges with the road where the vehicle is located, and at the same time The warning information is sent to the target image source for display.

In some examples, the potentially dangerous target for driving includes: a dangerous situation in front of the road where the vehicle is located; when it is determined that there is surrounding the vehicle based on the weather information and/or the condition information of the road where the vehicle is located When a potentially dangerous target is driving, generate warning information and send the generated warning information to the augmented reality head-up display for display, including: when the condition information of the road where the vehicle is located is obtained and the vehicle corresponds to the road where the dangerous situation occurs When the distance between the road sections of the condition is less than the road condition display distance threshold, the road condition image of the road where the vehicle is located is acquired and warning information is generated; wherein, the road condition image includes: the road condition where the vehicle is located Road section; determine the road section contour and road section position of the road section where the road condition occurs from the road condition image, and determine the distance between the road section where the road condition occurs and the vehicle according to the road section position; The image distance matching the distance between the road section of the road condition and the vehicle is used as the target image distance, and the image source matching the target image distance is selected from the multiple image sources of the augmented reality head-up display as The target image source; use the seventh color to fill the road section contour of the road section where the road condition occurs to obtain the state contour map of the road section where the road condition occurs; project the state contour map to the target image source On the road section where the road condition occurs, the condition contour map is merged with the road section where the road condition occurs.

In some examples, the driving information includes: operating information, driving speed, acceleration, and vehicle state information of the vehicle; the environmental information includes navigation information; and controlling the head-up display to emit at least two signals according to the driving information The road light to the windshield of the vehicle to respectively display at least two images with different distances from the eye box area of the vehicle includes: when according to the navigation information, the vehicle’s driving speed, acceleration, and vehicle State information, when generating the operation advice information of the vehicle, collect the vehicle operation video of the observer, and display the operation advice information through the augmented reality head-up display; when it is determined that the operation is not detected within the specified period of time When the operation advice information corresponds to the operation, dangerous driving information is generated, the dangerous driving information is displayed through the augmented reality head-up display, and the driving vehicle index is decremented; when it is determined that the operation is detected within the specified time period When the operation corresponding to the suggested information, the driving vehicle index is incremented; when it is determined that the time that the vehicle has not been operated is greater than the parking time threshold, the collection of vehicle operation videos is stopped, and the driving vehicle index and the collected all The vehicle operation video is sent to the augmented reality head-up display for display.

In some examples, the head-up display further includes: a flat mirror; the flat mirror reflects at least two rays of light emitted by the at least two image sources to the curved mirror.

The head-up display includes: a first image source, a second image source, a third image source, a first transflective film, a second transflective film, a flat mirror, and a curved mirror; the first image source, the second The second image source and the third image source are respectively arranged at different positions in the head-up display; the first image source can emit a first light, and the second image source can emit a second light; so The third image source can emit a third light; the first transflective film is disposed between the first image source and the second image source; the second transflective film is disposed on the second Between the image source and the first transflective film; the first transflective film can transmit the incident first light and reflect the incident second light and third light; the first Two transflective films, which can transmit the incident first light and the second light and reflect the incident third light; the plane reflecting mirror removes the incident first light and the second light And the third light are reflected to the curved mirror; the curved mirror reflects the incident first light out of the head-up display, so that the first light reflected out of the head-up display can be based on the first light itself The first image is formed by the length of the propagation path; the curved mirror reflects the incident second light out of the head-up display, so that the second light reflected out of the head-up display can follow the propagation of the second light The path length forms a second image; the curved mirror reflects the incident third light out of the head-up display, so that the third light reflected out of the head-up display can be based on the propagation path length of the third light , Forming a third image; the propagation path lengths of the first light, the second light and the third light are different, so that the distance between the first image and the eye box area, the second image and The distance between the observers and the distance between the third image and the eye box area are different from each other.

In some examples, the second transflective element can also be arranged between the first image source and the first transflective film; the first transflective film can reflect the incident first light And transmit the incident second light and third light.

In some examples, the first image source, the second image source, or the third image source is arranged at a position close to the focal plane of the curved mirror or at the focal plane of the curved mirror Position such that the first image source, the second image source, or the third image source that is set close to the focal plane of the curved mirror or the focal plane of the curved mirror emits The light emitted from the head-up display can form a long-distance image.

In some examples, the at least two image sources include a first image source and a second image source, the head-up display further includes a first transflective film and a first mirror; the first image source is configured to emit a first image source A light, the second image source is configured to emit a second light; the first transflective film is configured to transmit the first light and reflect the second light; the first mirror is configured to reflect the The first light and the second light.

In some examples, the at least two image sources include a first image source, a second image source, and a third image source, and the head-up display further includes a first transflective film, a second transflective film, and a first flat mirror; The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light; the first transflective film is configured to transmit light. The first light and reflect the second light and the third light; the second transflective film is configured to transmit the second light and reflect the third light; the first flat mirror is configured to reflect the The first light, the second light, and the third light.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the drawings of the embodiments. Obviously, the drawings in the following description only refer to some embodiments of the present disclosure, rather than limiting the present disclosure. .

Fig. 1 shows an imaging schematic diagram of a multi-level imaging system provided by at least one embodiment of the present disclosure;

Fig. 2 shows an imaging schematic diagram of a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 3 shows a schematic structural diagram of a head-up display in a multi-level imaging system provided by at least one embodiment of the present disclosure;

4A-4C show schematic structural diagrams of a head-up display in a multi-level imaging system provided by still other embodiments of the present disclosure;

5A-5E show schematic structural diagrams of a head-up display in a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of multiple images presented by the AR-HUD that can be seen in the eye box area in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 7 shows a flow chart executed by the ECU when the AR-HUD sends out images identifying the surrounding vehicles of the vehicle in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 8 shows a flowchart executed by an ECU when identifying non-motor vehicles around the vehicle in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 9 shows a flowchart executed by the ECU when the potentially dangerous target for driving is bad weather in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 10 shows a flowchart executed by the ECU when the potentially dangerous target for driving is a dangerous situation in front of the road where the vehicle is located in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 11 shows a flowchart executed by the ECU during vehicle driving recommendations in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 12 shows a schematic diagram of the AR-HUD display when the vehicle is too close to the vehicle in front in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 13 shows a schematic diagram of AR-HUD when there is a non-motor vehicle that is too close to the vehicle in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 14 shows a schematic diagram of AR-HUD fusing the path contour map with the road on which the vehicle is located in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 15 shows a schematic diagram of an AR-HUD display image when the condition in front of the road is an obstacle in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 16 shows an imaging side view of the AR-HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 17A shows the first schematic diagram of the HUD display image of the whole car window in a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 17B shows the second schematic diagram of the HUD display image of the whole car window in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 18 shows a schematic structural diagram of a full-window HUD in a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 19 shows a schematic structural diagram of a light control device in a multi-level imaging system provided by at least one embodiment of the present disclosure;

20A shows a schematic structural diagram of a first implementation manner of a light converging unit in a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 20B shows a schematic structural diagram of a second implementation manner of the light converging unit in the multi-level imaging system provided by at least one embodiment of the present disclosure;

20C shows a schematic structural diagram of a third implementation manner of the light converging unit in the multi-level imaging system provided by at least one embodiment of the present disclosure;

20D shows a schematic structural diagram of a fourth implementation manner of the light converging unit in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 21 shows an arrangement of light control devices in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 22 shows another arrangement of light control devices in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 23 shows an imaging schematic diagram of AR-HUD in a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 24 shows a schematic diagram of multiple images presented by the AR-HUD that can be seen in the eye box area in the multi-level imaging system provided by at least one embodiment of the present disclosure; FIG.

FIG. 25A shows a schematic structural diagram of the first AR-HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 25B shows a schematic structural diagram of the second type of AR-HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 25C shows a schematic structural diagram of a third type of AR-HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 25D shows a schematic structural diagram of a fourth type of AR-HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 26 shows a flow chart of the full-window HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure to identify the distance between the surrounding vehicles of the vehicle;

FIG. 27 shows a schematic diagram of a multi-level imaging system provided by at least one embodiment of the present disclosure, in which the full-window HUD displays the first warning information and the AR-HUD identifies the path between the vehicle and the motor vehicle that is too close. ；

FIG. 28 shows that in the multi-level imaging system provided by at least one embodiment of the present disclosure, when the vehicle is too close to the vehicle in front, the AR-HUD is controlled to display the path between the vehicle and the vehicle in front of the vehicle that is too close. Flow chart

FIG. 29 shows a flow chart of displaying collision warning information on a full-window HUD in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 30 shows a schematic diagram of a multi-level imaging system provided by at least one embodiment of the present disclosure, in which a full-window HUD displays collision warning information and the AR-HUD identifies a vehicle with a non-motor vehicle that is too close;

FIG. 31 shows the process of controlling the AR-HUD to identify the non-motor vehicle that is too close to the vehicle when the vehicle is too close to the non-motorized vehicle in the multi-level imaging system provided by at least one embodiment of the present disclosure Figure;

FIG. 32 shows a flow chart in which an ECU controls a full-window HUD to display weather information and status information of the road on which the vehicle is located in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 33 shows that in the multi-level imaging system provided by at least one embodiment of the present disclosure, when it is determined that the area where the vehicle is located is currently under severe weather conditions, the warning information can be displayed on the HUD of the entire vehicle window while the AR-HUD display and the road are integrated The intent of the image;

FIG. 34 shows a flow chart of the ECU controlling the AR-HUD to display images in the multi-level imaging system provided by at least one embodiment of the present disclosure when the potentially dangerous target for driving is bad weather;

35 shows a flow chart of the ECU controlling the AR-HUD to display images when the potentially dangerous target of driving is a dangerous situation in front of the road where the vehicle is located in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 36 shows a flowchart executed by the ECU when a vehicle driving recommendation is given in the multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 37A shows an imaging side view of an AR-HUD and a full-window HUD in a multi-level imaging system provided by at least one embodiment of the present disclosure;

FIG. 37B shows the first schematic diagram of displaying images when the AR-HUD and the full-window HUD work at the same time in the multi-level imaging system provided by at least one embodiment of the present disclosure; and

FIG. 37C shows the second schematic diagram of displaying images when the AR-HUD and the full-window HUD work at the same time in the multi-level imaging system provided by at least one embodiment of the present disclosure.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the protection scope of the present disclosure.

Unless otherwise defined, the technical terms or scientific terms used herein shall have the usual meanings understood by those with ordinary skills in the field to which this disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity, or importance, but are only used to distinguish different components. "Include" or "include" and other similar words mean that the elements or items appearing before the word cover the elements or items listed after the word and their equivalents, but do not exclude other elements or items. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "Down", etc. are only used to indicate the relative position relationship. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

At present, with the continuous development of technologies such as automobile intelligence, Internet of Vehicles, and autonomous driving, the information received by mobile on-board terminals and various expanded applications emerge in an endless stream. People are concerned about connecting all the displays in the car and flexibly displaying all kinds of information. The demand is increasing, but the driver's vision is easy to deviate when performing related operations, which poses a potential safety risk.

The head-up display (HUD) can avoid the distraction caused by the driver looking down at the instrument panel or other display screens during driving, improve the driving safety factor, and at the same time bring a better driving experience. More and more attention has been paid to the huge application potential of in-vehicle intelligent display.

For example, the HUD can display some driving information reasonably and vividly in the driver's sight area through a specially designed optical system inside, so as to enhance the driver's perception of the actual driving environment. For example, when the driver drives the vehicle to deviate from the established lane, the HUD can mark a red line at the edge of the lane line of the vehicle's normal driving lane to remind the driver that the vehicle has deviated from the lane; for example, when driving, you can see a marked bright band on the rear of the vehicle in front. That is: the image projected by the HUD is visually blended with the scene in the real environment (such as a car or an object). For example, the direction indicator arrow needs to be blended with the road to achieve a good visual effect.

The inventor of the present application discovered in research that the HUD includes: an image source, a flat mirror and a curved mirror; the light emitted by the image source is reflected by the curved mirror and then enters the flat mirror, and is reflected on the flat mirror and then emerges from the HUD. The light is emitted from the mouth, and the emitted light enters the windshield of the vehicle, and presents a single-layer image at a fixed position on the side of the windshield away from the eye box area. .

The eye box area refers to an area where the observer can observe an image that is better integrated with the real environment, and usually is an area with a fixed position.

The light emitted by the image source in the HUD is reflected on the curved mirror, and the reflected light is emitted to the eye box area, so that the observer can observe the image formed on the imaging window, such as outside the windshield, in the eye box area. The phenomenon of light reflection on the windshield can be approximated as a flat mirror imaging. According to the structure and optical principle of the HUD, the light emitted by the image source is reflected on the curved mirror and then emitted to the windshield. It can be considered that an equivalent image source emitting light is formed on the side of the curved mirror away from the image source. According to the imaging law of the plane mirror, the image formed by the HUD and the equivalent imaging source are symmetrical about the reflective medium.

For example, the HUD is an Augmented Reality HeadUp Display (AR-HUD).

The equivalent image source is an image formed on the side of the curved mirror away from the image source after the light emitted by the image source is reflected on the curved mirror.

The inventor of the present application also found that to form multiple images with different distances from the eye box area, it is necessary to increase the number of equivalent image sources in the HUD, so that there are at least two equivalent image sources with different positions in the HUD.

The position of the equivalent image source is determined by the imaging characteristics of the curved mirror, and the imaging law of the curved mirror is given by the following formula 1:

1/u+1/v=1/f(1)

Among them, f represents the focal length of the curved mirror; u represents the distance between the image source and the curved mirror, that is, the object distance of the image source on the curved mirror; v represents the distance between the image formed by the image source on the curved mirror and the curved mirror , That is, the image distance of the image source on the curved mirror. It can be seen from formula (1) that the position of the equivalent image source, that is, the position of the image formed by the image source on the curved mirror, is directly related to the image distance of the image source on the curved mirror. The larger the image distance, the farther the position of the equivalent image source of the image source is from the curved mirror.

The inventor of the present application also found that in order to increase the image distance of the image source, the image source can be placed on the focal plane of the curved mirror or close to the focal plane of the curved mirror.

For example, as shown in FIG. 1, at least one embodiment of the present disclosure provides a head-up display, including a curved mirror and at least one image source, the curved mirror is configured to reflect light emitted by the at least one image source, and the at least one The image source is close to the focal plane of the curved mirror or located at the focal plane of the curved mirror.

For example, the vicinity of the focal plane of the curved mirror means that the ratio of the distance between the image source and the focal plane of the curved mirror to the focal length of the curved mirror is less than a predetermined ratio. For example, the predetermined ratio may be 1%, 5%, 10%, 20% or other predetermined values.

For example, the distance between the image source and the focal plane of the curved mirror is less than one focal length of the curved mirror and is close to the focal plane of the curved mirror.

The inventor of the present application also found that the position of the equivalent image source is related to the following two parameters: the focal length of the curved mirror and the image distance of the image source on the curved mirror (that is, the distance between the image source and the curved mirror) .

The inventor of the present application also found that in the HUD, the focal length of the curved mirror is pre-designed, and it is difficult to change it. Therefore, to form multiple images with unequal distances from the eye box area, it is necessary to increase the amount of light emitted from the image source to the curved mirror, so that the image source emits at least two rays of light to the curved mirror. After each path of light enters the curved mirror, it can form equivalent image sources with different positions.

For example, after there are equivalent image sources with different positions in the HUD, the equivalent image sources of the equivalent image sources with different positions can be formed on the side of the windshield away from the eye box area with different distances from the eye box area. Images with different distances from the eye box area can be separately fused to scenes at different distances from the eye box area in the real environment.

Based on the above-mentioned various findings of the inventors of the present application, at least one embodiment of the present disclosure proposes a multi-level imaging system, a head-up display, a vehicle, and a multi-level imaging method. At least one embodiment is disclosed that can realize multi-level imaging, so that the imaging effect of the head-up display can be improved, thereby enhancing the user experience. In some embodiments, the images presented by each of the at least two rays of light emitted by the HUD can be visually fused with scenes at different distances from the eye box area in the real environment to enhance the imaging effect. .

For example, the HUD may include one or more sub-head-up displays, and each sub-head-up display may be an AR-HUD (such as a normal AR-HUD or a mixed reality head-up display (Mixed Reality HUD, MRHUD)) or a full-window HUD. The embodiment does not limit this.

It should be noted that the full-window HUD can also be called a large-view head-up display.

2 is a schematic diagram of a multi-level imaging system provided by at least one embodiment of the present disclosure. It has an eye box area and includes an electronic control unit (ECU) and a head-up display, the electronic control unit and the head-up display Communication connection; the electronic control unit is used to obtain driving information and send the driving information to the head-up display; the head-up display is configured to emit multiple lights to display multiple layers of images to display the driving information, At least part of the multi-layer image has a different distance from the eye box area.

For example, the display contents of the multi-layer images may be the same or different; they may be displayed at the same time, or may be displayed at different times or alternately.

It should be noted that displaying driving information should be understood as displaying information related to driving information.

For example, the electronic control unit controls the head-up display to emit the multiple lights according to the driving information.

It should be noted that multiple rays of light should be understood as at least two rays of light.

For example, the unequal distance between at least part of the multi-layer image and the eye box area means that the distance between the geometric center of the image of two or more layers in the multi-layer image and the geometric center of the eye box area is not equal. Etc. (e.g. not equal within the allowable error range).

For example, the multi-layer image may include multi-layer images with different imaging areas but the same distance from the geometric center of the image to the geometric center of the eye box region.

For example, the multi-layer image may include a multi-layer image with different distances from the geometric center of the image of two or more layers in the multi-layer image to the geometric center of the eye box region.

As shown in FIG. 2, the head-up display emits two rays of light, and the two rays of light respectively form an image 1 and an image 2. The distance d1 between the image 1 and the eye box area and the distance d2 between the image 2 and the eye box area are different.

For example, the multiple light beams here may be emitted by the same image source, or may also be emitted by multiple image sources separately. Each light is part or all of the light emitted by the corresponding image source.

For example, the multiple rays of light display the multi-layer image in a one-to-one correspondence.

For example, the multi-level imaging system further includes an imaging window, which is used for presenting on the side of the windshield away from the head-up display a different distance from the eye box area formed by the multi-path light. For a multi-layer image, light that forms any layer of the image in the multi-layer image can enter the eye box area.

For example, the imaging window may be a windshield of a vehicle to which the multi-level imaging system is applied; in other examples, the imaging window may also be additionally provided, for example, arranged in front of the windshield.

For example, the eye box area is an eye box area of a multi-level imaging system applied to the vehicle.

For example, the vehicle may include, but is not limited to, a vehicle, an aircraft, or a water vehicle, which is not limited in at least one embodiment of the present disclosure.

For example, the multi-level imaging system further includes a data acquisition device. When the vehicle is a vehicle, the data acquisition device is used to acquire environmental information around the vehicle and send the acquired environmental information to the electronic device. control unit.

For example, the driving information includes all information that may have an influence on the driving of the driver or related to driving, such as driving information and environmental information. For example, the driving information includes: operation information, driving speed, acceleration, and vehicle status information of the vehicle; the environmental information includes: navigation information, weather information, and road condition information (including surrounding vehicles, pedestrians, obstacles, etc.) Information).

In at least one of the following embodiments, the term "visually fused with the real environment (scene in the real environment)" refers to the image presented by the HUD seen by the observer’s eyes in the eye box area and the real environment (in the real environment). The scene) is completely fit/coincident together.

The term "image" refers to the virtual image of the HUD on the side of the windshield away from the eye box.

For example, an electronic control unit (ECU) not only includes vehicle state control devices such as vehicle speed, motor, fuel consumption, and shift efficiency, but also includes in-vehicle systems such as entertainment interactive systems and car networking systems.

Windshield, any shape of vehicle windshield can be used to display HUD images, which will not be repeated here.

In order to make the above objectives, features, and advantages of the application more obvious and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific implementations.

For example, the head-up display includes a first sub-head-up display, and the first sub-head-up display is an augmented reality head-up display. For example, the augmented reality head-up display is configured to display a single-layer image; or the augmented reality head-up display is configured to emit at least two rays of the multi-path light to display at least two layers of the multi-layer image.

For example, the augmented reality head-up display is based on at least the principle of augmented reality for display. For example, the augmented reality head-up display may also be based on mixed augmented reality (MR).

For example, at least one embodiment of the present disclosure further provides a head-up display, including a curved mirror, a first image source, and a second image source. The first image source is used to emit a first path of light incident to the curved mirror, and the second image source is used to emit a second path of light incident to the curved mirror; There is a first propagation path between the first image source and the curved mirror, and the second path of light has a second propagation path between the second image source and the curved mirror; the first propagation path The length of the second propagation path is different from that, so that the first image displayed by the head-up display through the first path of light and the second image displayed through the second path of light have different image distances.

Hereinafter, taking the first sub-head-up display as an augmented reality head-up display as an example, the multi-level imaging system provided by at least one embodiment of the present disclosure will be exemplarily described, and it is also an exemplary description of the head-up display provided by the embodiments of the present disclosure. But this is not a limitation of the present disclosure.

Fig. 2 shows a multi-level imaging system provided by at least one embodiment of the present disclosure, which is used in a vehicle (such as a vehicle), including: a windshield of the vehicle, an electronic control unit installed on the vehicle, Data acquisition equipment, and AR-HUD.

The ECU is respectively connected to the data acquisition device and the AR-HUD in communication.

For example, one end of the ECU is communicatively connected to the data acquisition device, and the other end of the ECU is communicatively connected to the augmented reality head-up display.

The data collection device is used to obtain environmental information around the vehicle and send the obtained environmental information to the ECU.

The ECU is used to obtain the driving information of the vehicle, and send the driving information of the vehicle and the environmental information sent by the sensor to the AR-HUD for display.

The AR-HUD is used to emit at least two rays of light to the windshield, and each of the at least two rays of light can respectively display multiple images with different distances from the eye box area; among them, more Each of the images can display the driving information and environmental information of the vehicle sent by the ECU.

See FIG. 6 for a schematic diagram of multiple images presented by the AR-HUD that can be seen in the eye box area. The imaging window, such as a windshield, is used to present a large distance from the eye box area formed by each of the at least two rays of light on the side of the windshield away from the AR-HUD. An image, wherein the light that forms any one of the multiple images can enter the eye box area.

For example, the data collection equipment includes, but is not limited to: sensors, navigation equipment, and wireless communication equipment installed on vehicles; the sensors, navigation equipment, and wireless communication equipment are respectively connected to the ECU.

For example, the sensor includes, but is not limited to: one or more of an image ranging sensor, an image sensor, a gravity sensor, and a ranging sensor.

For example, the image sensor is installed around a vehicle (for example, a vehicle), and is used to collect images of the surrounding environment of the vehicle. The distance measuring sensor measures the distance between the vehicle and surrounding objects; The distance between the vehicle and surrounding objects and the image of the surrounding environment of the vehicle are sent to the ECU together. The image ranging sensor has the function of an image sensor and the function of a ranging sensor.

For example, the ECU stores the distances of each distance measuring sensor from the eye box area, which can be stored in the form of the corresponding relationship between the sensor identification of the distance measuring sensor and the distance to determine the distance between the eye box area and objects around the vehicle.

For example, the distance between each distance measuring sensor and the eye box area is stored in the vehicle in advance, for example, after each distance measuring sensor is installed in the vehicle, it is measured by (for example, a worker) and stored in the vehicle. ECU.

For example, the gravity sensor is used to measure the yaw direction, yaw angle, and yaw rate of the vehicle, and send the measured yaw direction, yaw angle, and yaw rate of the vehicle to the ECU.

For example, the wireless communication device is used to connect the ECU to the Internet to obtain weather information and road condition information of the area where the vehicle is located.

For example, the navigation device is used to plan the travel path of the vehicle, obtain navigation information of the vehicle, instruct the vehicle to drive according to the navigation information, and send the obtained navigation information to the ECU.

For example, the navigation device is also used to locate the vehicle, obtain location information of the vehicle, and send the obtained location information of the vehicle to the ECU.

For example, the driving information of the vehicle includes, but is not limited to: speed information of the vehicle, location information of the vehicle, operation information of the vehicle, driving speed, acceleration, and state information of the vehicle.

For example, the vehicle status information includes, but is not limited to: engine status information, gearbox status information, and the current power and/or fuel volume of the vehicle.

For example, the operation information of the vehicle includes, but is not limited to: gear shift operation, acceleration/deceleration operation, and steering operation of the vehicle.

For example, the vehicle speed information, the vehicle operation information, the driving speed, acceleration, and the vehicle state information are obtained by the ECU from an on-board automatic diagnosis system (On-Board Diagnostics, OBD).

For example, the environmental information around the vehicle includes, but is not limited to: distance information between the vehicle and surrounding vehicles (for example, distance information), weather information on the road where the vehicle is located, and the vehicle Information on the status of the road and navigation information.

For example, the vehicle distance information between the vehicle and the surrounding vehicles is obtained through the aforementioned distance measuring sensor.

For example, the surrounding vehicles include: a front vehicle located in front of the vehicle, a rear vehicle located behind the vehicle, and side vehicles located on both sides of the vehicle. Taking a vehicle as an example of a vehicle, for example, the surrounding vehicles include: a front vehicle located in front of the vehicle, a rear vehicle located behind the vehicle, and side vehicles located on both sides of the vehicle.

For example, the weather information of the road where the vehicle is located and the status information of the road where the vehicle is located are obtained by the ECU after accessing the Internet.

For example, the navigation information is sent to the ECU by the navigation device of the vehicle.

For example, after receiving the environmental information and driving information, the ECU performs formatting and analysis processing on the environmental information and driving information, and obtains the image data to be fused to be displayed for fusion display, so as to control the image emitted by the image source in the AR-HUD Can be integrated with the scene in the real environment,

For example, the formatting of information is a further multi-dimensional description of the information type. The formatting is to match the existing data according to the preset database in the ECU to determine whether the received environmental information and driving information can be integrated with the scene in the current real environment and displayed , And then determine the fitting position of the collected data in the display screen and other display strategies (the position where some data is to be displayed may not be in the current field of view corresponding to the actual scene, and it can be displayed in advance according to the relative position). When it is determined through the above operations to display part of the data in the environmental information and driving information, the part of the data that needs to be displayed is sent to the information display fusion unit in the ECU as the display data to be fused.

For example, after the information display fusion unit receives the display data to be fused, it sends the display data to be fused to the image source in the AR-HUD, and sends the display data to be fused to the scene in the real environment through the image source. The scenes are displayed in fusion.

For example, the fusion and matching of the image and the scene in the real environment can be further configured to map the image of the data that needs to be fused and displayed with the scene in the field of view through image recognition. According to the mapping relationship, the image is displayed in the same relative position of the scene, so that The image emitted by the image source can be displayed in fusion with the scene in the real environment.

For example, referring to the first structure diagram of the AR-HUD shown in FIG. 3 that can be applied to the multi-level imaging system proposed by at least one embodiment of the present disclosure, the AR-HUD includes: a curved mirror 202 and at least two The image source 200 emits at least two rays of light to the windshield,

At least two image sources can respectively emit at least two rays of light; each of the at least two rays of light is emitted by a different image source 200 of the at least two image sources.

For example, the length of the propagation path of the light beams between the image source emitting the light beams and the curved mirror is at least partially different.

For example, the length of the propagation path of each ray is the length of the propagation path of the main optical axis of the ray, and the curved mirror reflects at least two incident rays out of the AR-HUD, so that at least the length of the AR-HUD is reflected Each of the two rays of light can form images with different distances from the eye box area according to the length of the propagation path of each of the rays.

For example, the curved mirror 202 may be, but not limited to: a spherical mirror, a hyperbolic mirror, a parabolic mirror, and a free-form surface mirror.

The image source 200 can emit light for presenting an image.

For example, the light emitted by each image source in at least two image sources 200 has different propagation path lengths between the image source emitting each light and the curved mirror, indicating that the object distance of each image source on the curved mirror is different. , And the focal length of the curved mirror is constant. According to the imaging law of the curved mirror given in the above formula 1, the image distance of each image source on the curved mirror is different, that is, the equivalent image source of each image source and the curved mirror The distance is different. Therefore, the light emitted by each image source can form images with unequal distances from the eye box area.

For example, the image is an image that matches the distance of the scene that needs to be fused in the real environment.

For example, the scene may be, but is not limited to: motor vehicles, non-motor vehicles, pedestrians, animals, lane change areas, and repair sections in front of the road where the vehicle is located, damaged sections in front of the road where the vehicle is located, and the road where the vehicle is located. There are obstacles ahead, and there is a traffic accident section in front of the road where the vehicle is located.

For example, the ECU also stores the corresponding relationship between the image source identification and the image distance of each image source in the AR-HUD to achieve the purpose of multi-level imaging

For example, the image distance of each image source is set before the AR-HUD leaves the factory.

For example, the image distance of each image source can be set to a few meters, ten meters, tens of meters, and infinity. For example, it can be stored in the ECU through the correspondence between the image source identifier and the image distance.

For example, the process of determining the image matching the distance of the scene includes: the ECU calculates the difference between the distance between the scene and the eye box area and the image distance of each image source, and calculates the difference between the distance between the scene and the eye box area with the smallest value. The image corresponding to the image distance is determined as the image matching the scene distance.

According to the curved mirror imaging law given by the above formula 1, the inventor of the present application found that when the image source 200 is placed on the focal plane of the curved mirror 202 or close to the focal plane, the image can be presented at an infinite distance from the eye box area. Place, so that the observer can see the image without parallax.

For example, the parallax refers to a situation where the image seen by the observer's eyes in the eye box area and the scene in the real environment cannot be merged in the horizontal direction and/or the vertical direction.

For example, the at least two image sources in the head-up display include: a long-distance imaging image source set close to the focal plane of the curved mirror or set at the focal plane of the curved mirror, in order to be able to Imaging at infinity.

The light emitted by the long-distance imaging image source can form a long-distance image after exiting the AR-HUD.

The long-distance image is used to eliminate parallax when the observer views the image.

For example, referring to the imaging side view of the AR-HUD shown in FIG. 16, among the images presented by at least two image sources, the distance between the remote image presented by the remote imaging source and the eye box area is the farthest. That is, the long-distance image is the image with the farthest horizontal distance from the eye box area among the images presented by at least two image sources.

For example, the structure of the AR-HUD shown in Figure 3 includes only two image sources for illustration, and the AR-HUD can also include more image sources, which will not be repeated here.

For example, referring to FIG. 4A, a schematic structural diagram of an AR-HUD that can be applied to the multi-level imaging system proposed by at least one embodiment of the present disclosure, an AR-HUD proposed by at least one embodiment of the present disclosure includes: a first image The source 300, the second image source 302, the curved mirror 202 and the first transflective film 304.

The first image source 300 can emit a first light, and the second image source 302 can emit a second light.

For example, the first image source 300 and the second image source 302 are respectively arranged at different positions, and the propagation path lengths of the first light and the second light are different.

The different propagation path lengths of the first light and the second light mean that the lengths of the propagation paths of the first light and the second light in the HUD are different.

The first transflective film 304 can transmit the incident first light and reflect the incident second light.

The first transflective film 304 is arranged between the first image source and the second image source.

The curved mirror 202 reflects the incident first light out of the head-up display, so that the first light reflected out of the head-up display can form a first virtual image according to the propagation path length of the first light.

The curved mirror 202 reflects the incident second light out of the head-up display, so that the second light reflected out of the head-up display can form a second virtual image according to the propagation path length of the second light.

For example, the propagation path lengths of the first light and the second light are different, so that the distance between the first virtual image and the observer is different from the distance between the second virtual image and the observer;

For example, the polarization characteristics of the first light and the second light are different.

For example, the first light may be S-polarized light; the second light may be P-polarized light.

For example, on the basis of the schematic structural diagram of the AR-HUD shown in FIG. 4A, refer to the schematic structural diagram of the third AR-HUD shown in FIG. 4B that can be applied to the multi-level imaging system proposed in at least one embodiment of the present disclosure. , AR-HUD also includes: a second transflective film 308 and a third image source 306.

The third image source 306 can emit a third light.

For example, the second transflective film 308 is disposed between the first transflective film 304 and the first image source 300.

The second transflective film can transmit the incident first light and reflect the incident third light; the first transflective film transmits the incident third light so that the The third light is incident on the curved mirror.

The curved mirror reflects the incident third light out of the head-up display, so that the third light reflected out of the head-up display can form a third image according to the propagation path length of the third light.

The third light rays have different propagation path lengths from the first light rays and the second light rays, so that the distance between the third image and the observer, the distance between the first image and the observer, and the distance between the third image and the observer are different. The distances between the second image and the observer are not the same.

For example, referring to FIG. 4C, another structural schematic diagram of an AR-HUD that can be applied to the multi-level imaging system proposed by at least one embodiment of the present disclosure; the second transflective film 308 is arranged on the second image Between the source 302 and the first transflective film 304.

The second transflective film can transmit the incident second light and reflect the incident third light; the first transflective film reflects the incident third light to the curved surface mirror.

The curved mirror reflects the incident third light out of the head-up display, so that the third light reflected out of the head-up display can form a third image according to the propagation path length of the third light.

For example, the polarization characteristics of the third light and the fourth light are different.

For example, the third light may be S-polarized light; the fourth light may be P-polarized light.

The first light emitted by the first image source 300, the second light emitted by the second image source 302, and the third light emitted by the third image source 304 have different propagation path lengths, which means that the first light The object distances of the first image source 300, the second image source 302, and the third image source 304 are different, resulting in the first image source 300, the second image source 302, and the third image source The image distances of 304 are different, so that the first image source 300, the second image source 302, and the third image source 304 can respectively image at different positions from the eye box area to achieve multi-level imaging.

Set the first image source 300, the second image source 302, or the third image source 304 at a position close to the focal plane of the curved mirror 202 or at the position of the focal plane of the curved mirror , So that the first image source, the second image source, or the third image source arranged at a position close to the focal plane of the curved mirror or at the position of the focal plane of the curved mirror emits After the light exits the AR-HUD, it can form a long-distance image to eliminate the parallax between the observer's viewing image and the scene in the real environment.

For example, the first image source 300, the second image source 302, or the third image set at a position close to the focal plane of the curved mirror 202 or at the position of the focal plane of the curved mirror The source 304 is referred to as a remote imaging image source.

For example, it is also possible to achieve different image distances by using a flat mirror or combining a flat mirror and a transflective film (transflective element).

For example, the augmented reality head-up display includes an image source, a flat mirror, and a curved mirror. The light emitted by the image source is reflected by the curved mirror and enters the flat mirror, and is reflected on the flat mirror before being emitted.

For example, as shown in FIG. 5A, the plane mirror is a plane mirror, and the image source includes at least two image sources, such as image source 1 (first image source) and image source 2 (second image source). An image source is configured to emit a first light, the second image source is configured to emit a second light; the first mirror is configured to reflect the first light and the second light. The propagation path lengths of the at least two rays of light between the image source emitting the respective rays of light and the curved mirror are different, so that different image distances are generated.

For example, as shown in FIG. 5B, the head-up display includes a first flat mirror (flat mirror 1) and a second flat mirror (flat mirror 2); the first flat mirror is configured to reflect the first light emitted by the image source 1; The plane mirror is configured to reflect the second light emitted by the image source 2.

For example, as shown in FIG. 5C, the head-up display includes a transflective element (transflective film) and a flat mirror; the transflective film is configured to transmit the first light emitted by the image source 1 and reflect the image source 2 second light emitted; the plane mirror is configured to reflect the first light and the second light.

For example, as shown in FIG. 5D, the head-up display includes a first transflective element, a second transflective element, a flat mirror, and a first image source (image source 1), a second image source (image source 2), and a third image source. Source (like source 3). The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light. The first transflective element is configured to transmit the first light and reflect the second light and the third light; the second transflective element is configured to transmit the second light and reflect the third light Light; the first plane mirror is configured to reflect the first light, the second light and the third light.

For example, as shown in FIG. 5E, the head-up display includes a first image source, a second image source, and a third image source, as well as a first transflective element, a second transflective element, and a flat mirror; the first image source configuration To emit the first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light. The first transflective element is configured to transmit the second light and reflect the first light; the second transflective film is configured to transmit the second light and reflect the third light; the first The plane mirror is configured to reflect the first light, the second light, and the third light.

For example, the electronic control unit is further configured to: form display data according to the driving information, process the display data according to the driver’s personal information, and control the light emitted by the head-up display according to the processed display data Illuminate to the imaging window.

For example, the processing includes at least one of the following: an image color saturation enhancement process, an image color conversion process, and an image shape conversion process.

For example, the electronic control unit is further configured to identify the identity information of the driver, and determine the personal information of the driver according to the identity information.

Through the above settings, the imaging system provided by at least one embodiment of the present disclosure can provide a driving assistance function for people with color vision defects such as color weakness/color blindness.

The following embodiments are described by taking the transportation as a vehicle as an example. In other embodiments, the same applies to the following embodiments when the vehicle is a non-vehicle type vehicle.

For example, when there are moving targets around the vehicle, the ECU may perform the following steps:

When there is a moving target around the vehicle, determine the distance between the eye box area and the moving target; wherein, the moving target includes: motor vehicles and non-motor vehicles;

When the distance between the moving target and the vehicle is less than the safe distance, the distance between the eye box area and the moving target is taken as the target image distance, and the AR-HUD image source is selected from the multiple image sources of the AR-HUD. The image source with matching distance is used as the target image source;

The target image source is controlled to send an image fused with the moving target to a moving target that is less than a safe distance from the vehicle, and the moving target is identified.

For example, the moving target should be a pedestrian, an object, a motor vehicle and/or a non-motor vehicle moving relative to the vehicle.

For example, in the above steps, in order to determine the distance between the eye box area and the moving target, the following steps are included:

Acquiring the distance information sent by the distance measuring sensor, the distance information including: the sensor identifier of the distance measuring sensor sending the distance information and the measured distance between the moving target and the vehicle;

According to the sensor identifier, the distance between the distance measuring sensor corresponding to the sensor identifier and the eye box area is queried from the corresponding relationship between the sensor identifier of the distance measuring sensor and the distance;

Calculate the sum of the distance between the distance measuring sensor corresponding to the sensor identifier and the eye box area and the distance between the moving target and the vehicle recorded in the distance information, and determine the calculation result as the distance between the eye box area and the moving target distance.

For example, in the above steps, the distance information further includes: surrounding images of the vehicle collected by the image sensor.

After the image sensor collects the image around the vehicle, it measures the distance between the moving target and the vehicle measured by the sensor, and generates distance information based on the sensor identification of the image sensor itself, the collected images of the surrounding vehicle, and the distance between the moving target and the vehicle, and sends it To the ECU.

In the above steps, the corresponding relationship between the sensor identifier of the ranging sensor and the distance is cached in the HUD control device.

In the above steps, the process of selecting the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image source, and determining the image matching the scene distance as described above The process is similar and can include the following steps to steps:

Acquiring, from the correspondence between the image source identifier and the image distance, the image distance of each image source in the AR-HUD except for the long-distance imaging image source among the multiple image sources;

Respectively calculating the difference between the image distance of each of the image sources and the target image distance;

Determine the image source corresponding to the image distance with the smallest difference between the target image distance and the target image distance smaller than the difference threshold among the image sources as the target image source matching the target image distance ；

When the difference between the image distance of each obtained image source and the target image distance is greater than the difference threshold, the long-distance imaging image source is determined as the target image source that matches the target image distance.

It can be seen from the content of the above steps that when the distance between the moving target and the vehicle is less than the safe distance, the distance is taken as the target image distance, and the target image distance is selected from a plurality of image sources to match the target image distance. The image source of is used as the target image source; the target image source is controlled to send out an image fused with the moving target to the moving target, and the moving target is identified, thereby assisting the driving of the vehicle.

For example, when the moving target is a vehicle, the environmental information includes: the distance information between the vehicle and surrounding vehicles; the surrounding vehicles include: the vehicle in front of the vehicle and the vehicle behind the vehicle The rear vehicle and the side vehicles located on both sides of the vehicle.

For example, referring to the flow chart of the AR-HUD shown in FIG. 7 for identifying the distance between the surrounding vehicles of the vehicle, the ECU, for example, is used to perform the following steps to identify the vehicle as the moving target:

Acquiring a first distance between the vehicle and the front vehicle, a second distance from the rear vehicle, and a third distance and a fourth distance from the side vehicles on both sides;

Determining the first safe distance threshold between the vehicle and the preceding vehicle according to the vehicle speed information of the vehicle;

When the first distance is less than or equal to the determined first safe distance threshold, generate first warning information, and send the first warning information to the AR-HUD for display;

When the second distance is less than or equal to the second safety distance threshold, second warning information is generated, and the second warning information is sent to the AR-HUD for display; wherein, the second safety distance threshold is used to indicate The safe driving distance between the vehicle and the rear vehicle;

When the third distance and/or the fourth distance is less than or equal to the third safe distance threshold, third early warning information is generated, and the third early warning information is sent to the AR-HUD for display; wherein, the first 3. A safe distance threshold, which is used to indicate the safe driving distance between the vehicle and the side vehicle;

When it is determined that the first distance is greater than the first safe distance threshold, select a target image source from the AR-HUD according to the first distance;

Use the second color identification to generate a second path identification instruction, and send the generated second path identification instruction to the target image source, and control the target image source to transfer the path between the vehicle and the preceding vehicle Use the color corresponding to the second color mark to display;

Generating a braking instruction when the first distance is less than the braking operation length threshold and the first distance is less than or equal to the determined first safe distance threshold and the duration is greater than the first preset duration threshold;

Using the generated braking instruction, the vehicle is controlled to perform a braking operation.

In the above steps, the first distance between the vehicle and the vehicle in front is obtained through the first distance information sent by the distance-measuring sensor installed on the head of the vehicle, and the second distance is sent by the distance-measuring sensor installed at the rear of the vehicle. The distance information acquires the second distance to the rear vehicle, and the third distance information and the fourth distance information are respectively sent by the distance measuring sensors installed on both sides of the vehicle to acquire the second distance of the side vehicle located on both sides of the vehicle. Three distance and fourth distance.

The specific processes of generating the first distance information, the second distance information, the third distance information, and the fourth distance information are at least partially similar to the foregoing process of generating distance information, and will not be repeated here.

For example, the first distance information includes, but is not limited to: the sensor identifier of the distance measuring sensor that sends the first distance information, the first distance between the vehicle and the preceding vehicle, and the distance between the vehicle and the preceding vehicle. The path image between.

The content included in the second distance information, the third distance information, and the fourth distance information is similar to the above-mentioned first distance information, and will not be repeated here.

For example, in the above steps, the ECU may query the first safety distance threshold corresponding to the vehicle speed information of the vehicle according to the correspondence between the vehicle speed and the safety distance cached by the ECU itself.

For example, the corresponding relationship between vehicle speed and safety distance can be expressed as follows:

When the vehicle speed is less than or equal to 20 km/h, the safe distance is 10 meters;

When the vehicle speed is greater than 20 km/h and less than or equal to 40 km/h, the safe distance is 20 meters;

When the vehicle speed is greater than 40 km/h and less than or equal to 60 km/h, the safety distance is 30 meters;

When the vehicle speed is greater than 60 km/h and less than or equal to 100 km/h, the safety distance is 60 meters.

For example, the ECU also stores a second safety distance threshold and a third safety distance threshold.

The second safe distance threshold is used to indicate the safe driving distance between the vehicle and the rear vehicle.

The third safe distance threshold is used to indicate the safe driving distance between the vehicle and the side vehicle.

In the above steps, the first warning information is used to indicate that the distance between the vehicle and the preceding vehicle is less than a first safe distance threshold.

For example, referring to the display schematic diagram of the AR-HUD when the vehicle is too close to the vehicle in front shown in FIG. 12; in order to generate the first warning information and send the first warning information to the AR-HUD for display, In step 704, the following steps can be performed:

Generating first warning information, and acquiring a path image between the vehicle and the preceding vehicle;

Identifying the path between the vehicle and the preceding vehicle according to the acquired path image between the vehicle and the preceding vehicle;

An image distance matching the first distance is used as a target image distance, and an image source matching the target image distance is selected from a plurality of image sources of the AR-HUD as the target image source; wherein, The image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror;

Use the first color mark to generate a first path identification instruction, and send the generated first path identification instruction to the target image source in the AR-HUD, and control the target image source to connect the vehicle with the front The path between the vehicles is displayed in the color corresponding to the first color mark, and the first warning information is sent to the target image source for display.

For example, in the above steps, the path image between the vehicle and the preceding vehicle is obtained from the first distance information.

For example, in the above steps, the ECU may use image processing technology to identify the path between the vehicle and the preceding vehicle based on the acquired path image between the vehicle and the preceding vehicle, specifically The process will not be repeated here.

For example, in the above steps, the image distance matching the first distance is the distance between the eye box area and the vehicle in front of the vehicle. Therefore, the image distance matching the first distance is determined The process is similar to the process of determining the distance between the eye box area and the moving target, and will not be repeated here. For example, the image distance matching the first distance corresponds to the actual distance between the external target.

In the above steps, the colors corresponding to the first color identifier may be red and pink.

For example, the first warning message may be "too close to the vehicle in front, please slow down".

In the above steps, the second warning information is used to indicate that the distance between the vehicle and the preceding vehicle is less than a second safe distance threshold.

In the above steps, the third warning information is used to indicate that the distance between the vehicle and the side vehicle on at least one side of the vehicle itself is less than the third safe distance threshold.

For example, in the above steps, the display modes of the first warning information, the second warning information, and the third warning information may include, but are not limited to, the following multiple ways: warning text, image, video, such as " Too close to the vehicle in front, please slow down, "Please keep a distance with the vehicle on the right"; or brightly colored, prominent icons, or prompt animations, such as red prompt messages; and the brightness of related prompt messages is higher than other AR-HUD Part of the screen brightness, or effects such as scrolling, flashing, and beating can also be used to further improve the warning effect.

For example, the displayed position is at least concentrated in front of the driver, or it can also be displayed in front of the co-pilot at the same time to remind passengers.

For example, on the basis of the AR-HUD displaying the second warning information and the third warning information, the vehicle or the audio playback device on the vehicle can be used for reminding by voice broadcast, which can be an alarm bell with no specific meaning. , Or it can be a specific voice reminder such as "Attention! Keep the distance between cars."

For example, it can cooperate with other equipment, such as mechanical vibration equipment integrated on the steering wheel, mechanical equipment integrated in the seat, to remind through vibration.

For example, warning texts, images, and videos are displayed on the side windows and rear windows of the vehicle, such as "rear vehicles please keep the distance", "side vehicles please keep the distance" or brightly colored and prominent icons, or prompt animations, such as red Prompt information; and the brightness of the related prompt information is higher than the brightness of other parts of the HUD, or effects such as scrolling, flashing, and beating can also be adopted to further improve the warning effect.

The display mode described in the above steps is similar to the display mode of the traditional HUD, and will not be repeated here.

In the above steps, the process of selecting the target image source is similar to the process of determining the distance between the eye box area and the moving target, and will not be repeated here. It is determined that the first distance is greater than the first safe distance threshold, indicating that the vehicle currently maintains a safe driving distance.

In the above steps, the color corresponding to the second color identifier may be green or no color.

For example, while controlling the target image source to display the path between the vehicle and the preceding vehicle in the color corresponding to the second color mark, at the same time, after confirming that a safe driving distance is maintained, it is displayed through the traditional HUD method. Prompt texts, images, videos, such as "safe distance, please continue to maintain"; or icons with a large color difference from the alarm display, or prompt animations, such as green prompt messages, or scrolling, flashing, beating, etc. The effect can disappear after being displayed for a certain period of time; or it can always remain on the screen and be shown to the observer.

For example, in the above steps, the braking operation length threshold is buffered in the ECU and can be set to 15 meters. The braking operation length threshold value can also be set to be less than the safety distance value recorded in the correspondence between any vehicle speed and safety distance, which will not be repeated here.

For example, the first preset duration threshold is cached in the ECU and can be set to 20 seconds.

When the moving target is a non-motorized vehicle, refer to the flowchart shown in FIG. 8 for identifying non-motorized vehicles around the vehicle. For example, to identify the non-motorized vehicle as the moving target, the ECU, For example, it is used to perform the following steps:

When it is determined that the location of the vehicle is a crowded area, acquiring an image of the surrounding environment of the vehicle;

When it is determined that there is a non-motorized vehicle around the vehicle according to the surrounding environment image, determine the distance between the non-motorized vehicle and the vehicle;

When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, generate collision warning information, and send the collision warning information to the AR-HUD for display;

When it is determined that the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, acquiring a non-motor vehicle image, and determining the current location of the non-motor vehicle;

According to the current location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle, determine the second size of the non-motor vehicle indicator frame fused with the non-motor vehicle, and based on the non-motor vehicle The outline of the non-motor vehicle in the image determines the second shape of the non-motor vehicle indicator frame;

Selecting a target image source from the AR-HUD according to the distance between the non-motor vehicle and the vehicle;

Use the fifth color identification, the second size and the second shape of the non-motor vehicle indicator frame to generate a non-motor vehicle identification instruction, and use the non-motor vehicle identification instruction to control the target image source to identify the color corresponding to the fifth color , Generating a non-motor vehicle indication frame according to the determined second size and second shape, and fusing the non-motor vehicle indication frame with the non-motor vehicle based on the current location of the non-motor vehicle;

When it is determined that the distance between the non-motor vehicle and the vehicle is less than the emergency braking length threshold, the vehicle is controlled to perform a braking operation.

In the above steps, the ECU determines the location of the vehicle from the location information sent by the navigation device. For example, when it is determined that the location of the vehicle is in a school, a hospital, a parking lot, and a prosperous urban area, it is determined that the location of the vehicle is a crowded area, and an image acquisition instruction is sent to the image sensor installed on the vehicle , So that the image sensor installed on the vehicle collects an image of the surrounding environment of the vehicle. For example, a crowded area means that the population density in a predetermined area is greater than the predetermined density.

For example, after the image sensor receives the image acquisition instruction, after collecting the surrounding environment image, when it is determined that there is a non-motorized vehicle around the vehicle, the distance measuring sensor determines the distance between the non-motorized vehicle and the vehicle, and then calculates the distance between the non-motor vehicle and the vehicle. The surrounding environment image, the determination of the distance between the non-motor vehicle and the vehicle existing around the vehicle, and the sensor identifier of the distance measuring sensor itself generate non-motor vehicle distance information and send it to the ECU.

For example, in the above steps, the collision safety distance threshold is cached in the ECU and can be set to any distance between 2 meters and 5 meters.

For example, the ECU, after receiving the non-motor vehicle distance information sent by the distance measuring sensor, refer to the schematic diagram of AR-HUD when there is a non-motor vehicle around the vehicle that is too close to the vehicle as shown in FIG. Steps or can also be performed:

When the distance between the non-motor vehicle and the vehicle recorded in the non-motor vehicle distance information is greater than the collision safety distance threshold and less than or equal to the early warning distance threshold, the distance between the non-motor vehicle and the vehicle is determined from the AR-HUD Select the target image source;

Controlling the target image source to use the color corresponding to the fourth color mark to generate an early warning graphic at the location of the non-motor vehicle; wherein the early warning graphic is fused with the non-motor vehicle;

When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, generating collision warning information;

According to the location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle, the first size of the non-motor vehicle indicator frame fused with the non-motor vehicle is determined, and based on the non-motor vehicle in the surrounding environment image Motor vehicle outline, determining the first shape of the non-motor vehicle indicator frame;

Use the image distance matching the distance between the non-motor vehicle and the vehicle as the target image distance, and select the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image Source; wherein, the image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror;

Use the third color identification, the first size and the first shape of the non-motor vehicle indicator frame to generate a non-motor vehicle identification instruction, and use the non-motor vehicle identification instruction to control the target image source to identify the color corresponding to the third color , Generating the non-motor vehicle indicator frame according to the determined first size and first shape, and fusing the non-motor vehicle indicator frame and the non-motor vehicle together according to the location of the non-motor vehicle to combine all The non-motor vehicle is identified, and the collision warning information is sent to the target image source for display.

For example, in the above steps, the warning distance threshold is cached in the ECU and can be set to any distance between 5 meters and 10 meters.

For example, the process of selecting the target image source from the AR-HUD according to the distance between the non-motor vehicle and the vehicle is to first determine the distance between the eye box area and the target according to the distance between the non-motor vehicle and the vehicle. The distance between the non-motor vehicle and the process of selecting the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image source. This process is the same as the process of determining the eye box area and the above The distance process of moving the target is similar, so I won't repeat it here.

For example, in the above steps, the color corresponding to the fourth color identifier may be a bright color such as red, green, and yellow.

For example, the warning graphic is pre-buffered in the ECU, and may be, but not limited to: a circle, an ellipse, and a quadrilateral.

For example, the specific process of controlling the target image source to use the color corresponding to the fourth color identifier to generate an early warning graphic at the location of the non-motor vehicle includes the following steps:

Determining the location of the non-motor vehicle according to the surrounding environment image;

Acquiring the cached warning graphic, and filling the warning graphic with the color corresponding to the fourth color mark;

According to the determined location of the non-motor vehicle, the target image source is controlled to emit the early warning graphic fused with the non-motor vehicle.

For example, in the above steps, the location of the non-motor vehicle can be determined according to the surrounding environment image by any algorithm for determining the position of the object from the digital image, and the specific process will not be repeated here.

In the above steps, the collision warning information is used to indicate that the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold.

In at least one embodiment, the collision warning information may be “too close to the pedestrian ahead, please pay attention”.

For example, in the above steps, image processing technology can be used to determine the non-motor vehicle indication frame fused with the non-motor vehicle according to the location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle. Determine the first shape of the non-motor vehicle indicator frame based on the contour of the non-motor vehicle in the surrounding environment image. The specific process is not repeated here.

For example, in the above steps, an image distance that matches the distance between the non-motorized vehicle and the vehicle is used as the target image distance, and the AR-HUD is selected from a plurality of image sources that are closer to the target image distance. The process of using the matched image source as the target image source is similar to the process of determining the distance between the eye box area and the moving target, and will not be repeated here.

For example, in the above steps, the colors corresponding to the third color identifier may be red and pink.

For example, in the above step 804, the display mode of the collision warning information may include, but is not limited to, the following various modes: warning text, image, video; or brightly colored and prominent icons, or prompt animation, such as red Or the brightness of the prompt information is higher than the brightness of other parts of the HUD display, or the effects of scrolling, flashing, and beating can also be adopted to further improve the warning effect. For example, the displayed position should be at least concentrated in front of the driver, and it can also be displayed in front of the co-pilot at the same time to remind passengers.

For example, use red and other bright-colored shapes or signs to mark and remind key information, and at the same time, accompanied by voice reminders, such as "there is a pedestrian ahead, pay attention to avoid", to further enhance the reminder effect to the driver.

The display mode described in the above steps is similar to the display mode of the traditional HUD, and will not be repeated here.

For example, in the above steps, the colors corresponding to the fifth color identifier may be green and blue.

The non-motor vehicle indication frame and the non-motor vehicle are integrated based on the current location of the non-motor vehicle, so that the observer can see the non-motor vehicle indication frame and the non-motor vehicle through the eye box area. The images merged by the motor vehicles to determine that the distance between the vehicle and the non-motor vehicle identified by the non-motor vehicle indicator frame is greater than the collision safety distance threshold.

The specific process of the above steps, and the ECU described in the above steps, is used to generate collision warning information when the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, and send the collision warning information to The specific process described in the process of AR-HUD display is similar, and will not be repeated here.

For example, when it is determined that the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, it means that the vehicle is at a safe distance from the surrounding non-motor vehicles, or the specific process of the above steps may not be performed. Non-motor vehicles at a safe distance will not be marked.

In the above steps, for example, the emergency braking length threshold may be any distance between 0.5 meters and 1.5 meters.

For example, in some cases, potentially dangerous driving targets existing around the vehicle are highlighted. The potentially dangerous driving target includes: bad weather conditions and dangerous conditions in front of the road where the vehicle is located.

For example, when there are potentially dangerous driving targets around the vehicle, the ECU is also specifically configured to perform the following steps:

Obtaining weather information and/or status information of the road where the vehicle is located;

When it is determined that there are potentially dangerous driving targets around the vehicle based on the weather information and/or the condition information of the road where the vehicle is located, warning information is generated, and the generated warning information is sent to the AR-HUD for display.

In the above steps, the ECU obtains weather information and/or condition information of the road where the vehicle is located through a wireless communication device.

For example, the ECU will determine the administrative area of the road where the vehicle is located according to the location information of the road where the vehicle is located, and then access the Internet through a wireless communication device to obtain weather information in the administrative area and traffic information in the administrative area. The traffic information includes: road condition information and condition information of the road where the vehicle is located, so as to obtain weather information of the road where the vehicle is located and condition information of the road where the vehicle is located.

The weather information of the road where the vehicle is located is used to indicate whether the road where the vehicle is located is in good weather or bad weather.

For example, the good weather conditions include but are not limited to: sunny, cloudy and sunny, and cloudy.

For example, the severe weather conditions include, but are not limited to: hail, heavy rain, floods, and tornadoes.

For example, the status information of the road where the vehicle is located includes, but is not limited to: including: reminder information about maintenance ahead of the road, reminder information about damage ahead of the road, reminder information about traffic accidents in front of the road, and obstacles Reminder information.

For example, there is a reminder message for maintenance ahead of the road, which is used to indicate that there is a maintenance section ahead of the road where the vehicle is located.

For example, the warning message that there is damage in front of the road is used to indicate that there is a damaged section in front of the road where the vehicle is located.

For example, the reminder information of a traffic accident in front of the road is used to indicate that there is a traffic accident section in front of the road where the vehicle is located.

For example, the reminder message that there is an obstacle in front of the road is used to indicate that there is an obstacle in front of the road where the vehicle is located.

For example, in the above steps, the following methods can be used to display the warning information:

Based on the AR enhancement system, the information of driving safety is analyzed. After virtual information is simulated and simulated, the application is displayed on the HUD. The real and virtual information complement each other to enhance the warning information.

Displaying information on the HUD will indicate the information that needs to be marked that will help driving, including: potentially threatening vehicles, potential road hazards, bad road conditions, and information that needs attention when driving at night;

Among them, safety signs can be green safety signs, and danger signs can be red warning signs;

According to real-time road conditions and networked road information, the driving route is assisted in the screen, and auxiliary lines and steering signs are marked on the correct driving road.

For example, the displayed information can be text, image, video, and the warning effect can be improved by using color deepening, scrolling, flashing, and jumping.

For example, the displayed position should be at least concentrated in front of the driver, and it can also be displayed in front of the co-pilot at the same time to remind passengers.

For example, by marking and prompting key information on AR-HUD, there is a potential threat to the vehicle, and the driver can be shown or voiced to remind the driver to threaten the vehicle information;

For example, by marking and prompting key information on the AR-HUD, if there is a potential road hazard, the driver can be shown or voiced to remind the driver of the road condition information.

For example, when the potentially dangerous target for driving is a bad weather condition, referring to the flowchart executed by the ECU when the potentially dangerous target for driving is bad weather as shown in FIG. And/or the condition information of the road where the vehicle is located, when it is determined that there are potentially dangerous driving targets around the vehicle, generating warning information and sending the generated warning information to the AR-HUD for display, including the following steps:

When it is determined that there is a bad weather condition on the road where the vehicle is located, acquiring a bad weather road image of the road where the vehicle is located and generating warning information;

Processing the bad weather road image to determine the brightness value of the road where the vehicle is located;

When the brightness value is less than the brightness threshold, generate high beam turn-on prompt information, display the high-beam turn-on prompt information through the AR-HUD, and determine the path of the road where the vehicle is located from the bad weather road image Contour and path position;

Determine the distance between the road where the vehicle is located and the eye box area according to the path position;

The image distance that matches the distance between the road where the vehicle is located and the eye box area is used as the target image distance, and an image that matches the target image distance is selected from the multiple image sources of the AR-HUD A source as a target image source; wherein the image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror;

Use the sixth color to fill the path contour of the road where the vehicle is located to obtain a path contour map of the road where the vehicle is located;

According to the path position of the road where the vehicle is located, the path contour map is projected onto the road where the vehicle is located through the target image source, so that the path contour map merges with the road where the vehicle is located, and the warning The information is sent to the target image source for display. Refer to the AR-HUD shown in Figure 14 for a schematic diagram of fusing the path contour map with the road where the vehicle is located, where the solid line part is the road part that can be seen through the eye box area, and the dashed line part is the path contour map showing Part of the road.

For example, in the above steps, the ECU sends an image acquisition instruction to the image sensor installed on the vehicle, so that the image sensor installed on the vehicle collects bad weather road images of the road where the vehicle is located.

In at least one embodiment, the warning message may be: "The weather is bad, please go slowly."

After receiving the image acquisition instruction, the image sensor collects the bad weather road image, and then sends the bad weather road image to the ECU.

In the above steps, the image brightness algorithm is used to process the bad weather road image, and the processed image brightness value is determined as the brightness value of the road where the vehicle is located. The specific process will not be repeated here.

In the above steps, the process of displaying the high beam turn-on prompt information through the AR-HUD is similar to the process of displaying information of the traditional HUD, and will not be repeated here.

For example, the ECU may use any image processing technology to determine the path profile and path position of the road where the vehicle is located from the bad weather road image. The specific process is not repeated here.

In at least one embodiment, the prompt message for turning on the high beam may be "please turn on the high beam".

For example, in the above steps, the geometric center of the road where the vehicle is located is obtained from the path profile of the road where the vehicle is located, and the distance between the geometric center of the road where the vehicle is located and the vehicle is measured by a distance measuring sensor, and the vehicle The distance between the geometric center of the road and the vehicle, and the distance between the distance measuring sensor that measures the distance between the geometric center of the road and the vehicle and the eye box area are calculated, and the result of the sum calculation is determined as the vehicle The distance between the road and the eye box area.

In the foregoing steps, the specific process of determining the target image source is similar to the foregoing process of determining the distance between the eye box area and the moving target, and will not be repeated here.

In the above steps, the sixth color may be a striking color such as red, blue, and green.

In the above steps, the process of sending the warning information to the target image source for display is similar to the process of displaying information in a traditional HUD, and will not be repeated here.

For example, refer to the flowchart executed by the ECU when the potentially dangerous target for driving is the dangerous situation in front of the road where the vehicle is located as shown in FIG. 10, and the AR-HUD display image when the situation in front of the road is obstructed as shown in FIG. In a schematic diagram, when the potentially dangerous driving target is a dangerous situation in front of the road where the vehicle is located, the ECU is used to determine the surrounding area of the vehicle according to the weather information and/or the condition information of the road where the vehicle is located When there are potentially dangerous driving targets, generating warning information and sending the generated warning information to AR-HUD for display, including the following steps:

When the condition information of the road where the vehicle is located is obtained and the distance between the vehicle and the road section corresponding to the road condition where the dangerous condition occurs is less than the road condition display distance threshold, the road condition image of the road where the vehicle is located is obtained and generated Warning information; wherein, the road condition image includes: a road section where a road condition occurs in the road where the vehicle is located;

Determining the road section contour of the road section where the road state occurs from the road state image;

The image distance that matches the distance between the road section where the road condition occurs and the vehicle is used as the target image distance, and an image that matches the target image distance is selected from the multiple image sources of the AR-HUD The source is the target image source;

Use the seventh color to fill the road section contour of the road section where the road condition occurs, to obtain the state contour map of the road section where the road condition occurs;

Projecting the condition contour map onto the road section where the road condition occurs through the target image source, so that the condition contour diagram is merged with the road section where the road condition occurs.

In the above steps, the road section where the road condition occurs may be, but not limited to: a repaired road section, a damaged road section, a traffic accident road section, and a road section with obstacles.

The road condition display distance threshold can be any distance between 0 and 2 kilometers.

The ECU sends an image acquisition instruction to the image sensor installed on the vehicle, so that the image sensor installed on the vehicle collects road conditions images of the road where the vehicle is located.

After the image sensor receives the image acquisition instruction and collects the road condition image, the distance measuring sensor measures the distance between the vehicle and the road section where the road condition occurs, and according to the sensor identification of the distance measuring sensor itself, the road condition image, and the distance between the vehicle and the road condition The distance of the road section where the road condition occurs generates road condition distance information, and the road condition distance information is sent to the ECU.

In the above steps, any image processing algorithm can be used to determine the road section contour of the road section where the road condition occurs from the road condition image, and the specific process will not be repeated here.

In the above steps, the process of determining the target image source is similar to the process of determining the distance between the eye box area and the moving target, and will not be repeated here.

In the above steps, the seventh color may be red, pink, and green.

In some cases, it is necessary to evaluate the operation of the vehicle and give driving recommendations for the vehicle. Refer to the flowchart shown in Figure 11 for the flow chart executed by the ECU when the vehicle driving recommendations are given. The ECU is also specifically used to execute The following steps:

According to the navigation information, the driving speed, acceleration, and vehicle status information of the vehicle, when generating the operation advice information of the vehicle, the vehicle operation video of the observer is collected, and the vehicle operation video is collected through the AR-HUD. Operation suggestion information display;

When it is determined that the operation corresponding to the operation advice information is not detected within the specified operation time period, generate dangerous driving information, display the dangerous driving information through the AR-HUD, and perform a decrement operation on the driving vehicle index;

When it is determined that the operation corresponding to the operation suggestion information is detected within the specified operation time period, an incremental operation is performed on the driving vehicle index;

When it is determined that the length of time the vehicle has not been operated is greater than the parking time threshold, stop collecting vehicle operation videos, and send the driving vehicle index and the collected vehicle operation videos to the AR-HUD for display.

In the above steps, the navigation information is obtained from a navigation device; the driving speed, acceleration, and vehicle state information of the vehicle are obtained from OBD (On Board Diagbostics, car detection system).

The process of generating the operation suggestion information of the vehicle according to the navigation information, the driving speed, acceleration, and vehicle state information of the vehicle is not repeated here.

The operation advice information includes, but is not limited to: recommended driving route information, recommended gear information, obstacle prompt information, recommended speed information, and recommended braking information.

For example, in the above steps, the specified operation duration can be any length of time between 3 and 5 seconds.

The driving vehicle index is used to evaluate the reasonable degree of vehicle operation; the larger the driving vehicle index, the more reasonable the vehicle operation.

Decreasing the driving vehicle index refers to subtracting a fixed index variable from the current driving vehicle index to obtain the driving vehicle index after the decrement operation.

In the above steps, the process of incrementing the driving vehicle index is similar to the process of decrementing the driving vehicle index, and will not be repeated here.

In summary, at least one embodiment of the present disclosure proposes a multi-level imaging system. By setting an AR-HUD that can emit at least two rays of light, different images can be displayed at positions that are not equal to the eye box area. The light that forms any one of the multiple images can enter the eye box area, achieving the purpose of multi-level imaging. The image presented by each of the at least two rays of light emitted by the AR-HUD can be compared with the real environment. The scenes at different distances from the eye box area are fused separately to avoid the deviation caused by the image fusion with the scene in the real environment as much as possible, so that the observer can see the merged image and scene, and improve the AR-HUD Use experience.

The inventor of the present application found that the field of view (FOV, Field of View) of the traditional HUD based on free-form surface mirrors is small, resulting in a small display size of the HUD image, unable to display richer content, and reducing the use of HUD Experience.

In the multi-level imaging system of at least one embodiment of the present disclosure, the head-up display further includes a second sub-head-up display, the horizontal field of view of the second sub-head-up display is not less than 15 degrees, and the second sub-head-up display The vertical field of view angle is not less than 5 degrees. For example, the second sub head-up display is a full-window head-up display or a large-view head-up display.

For example, through the above arrangement, the imaging area can be increased. For example, the image presented by the light emitted by the second head-up display can cover the visible area of the windshield.

For example, the electronic control unit is configured to: control the second head-up display to emit light to irradiate the imaging window according to the driving information, so that the image presented by the light from the second head-up display is in all directions. The area of the imaging area at the imaging window is larger than the area of the imaging area at the imaging window of each of the remaining images in the plurality of images.

For example, the light control device includes: a light converging unit and a dispersion element; the light converging unit is used to reflect the light incident on the light control device to the diffuser in a direction opposite to the incident direction of the light. Element; The dispersive element diffuses the incident light and forms a beam used to form a spot.

For example, the dispersive element is configured to diffuse the light beam passing through the dispersive element without changing the optical axis of the light beam.

For example, the light converging unit includes: an inverted triangular pyramid microstructure, a supporting structure, and a substrate layer; the inverted triangular pyramid microstructure and the supporting structure are disposed on the substrate layer, and the inverted triangular pyramid microstructure is located at the substrate layer. The support structure is far away from the substrate layer; the refractive index of the inverted triangular pyramid microstructure is greater than the refractive index of the support structure; the inverted triangular pyramid microstructure is configured to make the incident angle of the incident light greater than the critical The angled light is reflected to the dispersion element in a direction opposite to the incident direction of the light in a way of total reflection.

For example, the light converging unit includes: light converging layers sequentially arranged in the incident direction of the light,

An isolation layer, a plane reflective layer, and a substrate; the plane reflective layer is located on the focal plane of the light condensing layer; the light condensing layer and the plane reflective layer are made of different metamaterials; the light The converging layer is used to converge the incident light onto the plane reflective layer by changing the phase of the incident light, and is used to reflect the light back from the plane reflective layer along the direction opposite to the direction in which the light enters the light converging layer The direction is reflected to the dispersion element; the plane reflection layer is used to change the phase of the light condensed by the light converging layer, and reflect the light after the phase change to the light converging layer.

For example, for the same external target, the electronic control unit is further configured to control the first sub-head-up display to display the first information of the external target according to the driving information, and control the second sub-head-up display to display the first information of the external target. For the second information of the external target, the content included in the first information and the second information is at least partially different.

For example, for multiple external targets, the electronic control unit is further configured to control the first sub-head-up display to display the first external target that meets the first selection condition, and control the second sub-head-up display to display the first external target that meets the second selection condition. Two external targets, the first condition and the second condition are different, and the first external target and the second external target are different.

At least one embodiment of the present disclosure provides a multi-level imaging system, which is provided with a second sub-head-up display, and the arrangement of the second sub-head-up display matches the visible area of the windshield, so that the first The image presented by the light emitted by the second head-up display can cover the visible area of the windshield. Since the arrangement of multiple light sources in the second head-up display matches the visible area of the windshield, The light emitted by the second sub-head-up display can display an image covering the visible area of the windshield, achieving the purpose of displaying the image at any position within the visible area of the windshield, so that the second sub-head-up display can display an image that covers the visible area of the windshield. The sub-head-up display displays richer content, which improves the HUD experience.

Referring to the schematic diagram of the second sub-head-up display shown in FIG. 17A and FIG. 17B, the image covering the visible area of the windshield may refer to the image covering all the visible area of the windshield. Or it can be an image that partially covers the entire visible area in the windshield.

In at least one embodiment of the present disclosure, the image is a virtual image and is presented on the side of the windshield away from the observer.

The term "covering" means that the observer can see an image that is as large as the visible area of the windshield or slightly smaller than the visible area of the windshield at the position of the windshield through the eye box area; feel the image from the observer's perspective It is presented on the windshield, and it does not mean that the image is actually presented on the windshield.

The windshield may adopt any shape of the vehicle windshield to display the image of the second sub-head-up display, which will not be repeated here.

The ECU includes not only vehicle state control devices such as vehicle speed, motor, fuel consumption, and gear shift efficiency, but also vehicle-mounted systems such as entertainment interactive systems and car networking systems.

At least one embodiment of the present disclosure proposes a multi-level imaging system for vehicles, including: an imaging window, an electronic control unit ECU installed on the vehicle, a data acquisition device, and a full-window HUD.

For example, the imaging window may be a windshield of a vehicle to which the multi-level imaging system is applied; in other examples, the imaging window may also be additionally provided, for example, arranged in front of the windshield.

The ECU is respectively connected to the data acquisition device and the full-window HUD in communication.

For example, one end of the ECU is communicatively connected to the data acquisition device, and the other end of the ECU is communicatively connected to the second sub head-up display.

The data collection device is used to obtain environmental information around the vehicle and send the obtained environmental information to the ECU.

The data collection equipment includes, but is not limited to: sensors, navigation equipment, and wireless communication equipment installed on a vehicle; the sensors, navigation equipment, and wireless communication equipment are respectively connected to the ECU.

The sensors include, but are not limited to: one or more of image sensors and distance measuring sensors.

For example, the image sensor is installed around the vehicle to collect images of the surrounding environment of the vehicle, and the distance measuring sensor is used to measure the distance between the vehicle and the surrounding objects; thus, the measured distance between the vehicle and the surrounding objects can be compared with the vehicle. The surrounding environment images are sent to the ECU together.

For example, in the ECU, the distance between each distance measuring sensor and the eye box area is stored, and the distance between the sensor identification of the distance measuring sensor and the distance can be stored to determine the eye box area of the multi-level imaging system and the vehicle. The distance of surrounding objects

The distance between each ranging sensor and the eye box area is measured by the staff after each ranging sensor is installed in the vehicle and stored in the ECU of the vehicle.

The wireless communication device is used to enable the ECU to access the Internet to obtain weather information and road condition information of the area where the vehicle is located.

The navigation device is used to plan the driving path of the vehicle, obtain navigation information of the vehicle, instruct the vehicle to drive according to the navigation information, and send the obtained navigation information to the ECU.

The ECU is used to obtain the driving information of the vehicle, and send the driving information of the vehicle and the environmental information sent by the sensor to the full-window HUD for display.

The driving information of the vehicle includes, but is not limited to: speed information of the vehicle, position information of the vehicle, operation information of the vehicle, driving speed, acceleration, and vehicle state information.

The vehicle status information includes, but is not limited to: engine status information, gearbox status information, and the current power and/or fuel volume of the vehicle.

The operation information of the vehicle includes, but is not limited to, the gearbox shift operation, acceleration/deceleration operation, and steering operation of the vehicle.

The speed information of the vehicle, the operation information of the vehicle, the driving speed, acceleration, and the vehicle state information are obtained by the ECU from an on-board automatic diagnostic system (On-Board Diagnostics, OBD).

The environmental information around the vehicle includes, but is not limited to: distance information between the vehicle and surrounding vehicles, weather information of the road where the vehicle is located, information about the conditions of the road where the vehicle is located, and navigation information.

The distance information between the vehicle and surrounding vehicles is obtained by the above-mentioned distance measuring sensor.

The surrounding vehicles include: a front vehicle located in front of the vehicle, a rear vehicle located behind the vehicle, and side vehicles located on both sides of the vehicle.

The weather information of the road where the vehicle is located and the condition information of the road where the vehicle is located are obtained by the ECU after accessing the Internet.

The navigation information is sent to the ECU by the navigation device of the vehicle.

The full-window HUD is used to display the environmental information sent by the ECU and the driving information of the vehicle.

The arrangement of the full-window HUD matches the visible area of the windshield, so that the image presented by the light emitted by the full-window HUD can cover the visible area of the windshield; wherein , The image includes: the environmental information and the driving information of the vehicle.

The windshield is used to reflect the light emitted by the full-window HUD into the eye box area.

For example, referring to the schematic structural diagram of the second sub-head-up display shown in FIG. 18, the second sub-head-up display includes: a projection device 102 and a light control device 100 to display one on the visible area of the windshield The image of the visible area of the windshield can be covered.

The projection device 102 is installed in the vehicle; the light control device 100 is installed on the surface of the center console of the vehicle opposite to the windshield.

For example, the projection device 102 may be installed above the observer.

The projection equipment is used to emit light that can be incident on the light control device.

The light control device 100 is used to change the exit direction of the light emitted by the projection device 102 so that the light enters the eye box area after being reflected by the windshield 104.

For example, the light control device 100 has a counter-reflection effect on light, and is used to reflect the incident light in a direction opposite to the incident direction.

The size and shape of the light control device match the visible area of the windshield 104; so that the light emitted by the projection device is reflected to the windshield 104 by the light control device, so that it can appear to cover the windshield 104 Image of the visible area of the windshield.

Referring to the structural schematic diagram of the light control device shown in FIG. 19, the light control device includes: a light converging unit 1000 and a diffractive optical element 1002; the diffractive optical element 1002 is disposed on the light converging unit 1000.

For example, when light is incident, the light converging unit 1000 reflects the light to the diffractive optical element 1002 in a direction opposite to the incident direction of the light, thereby changing the exit direction of the light emitted by the projection device, so that The light emitted by the projection device 102 can enter the eye box area.

The diffractive optical element 1002 diffuses the incident light and forms a light spot.

The diffractive optical element 1002 is used to control the degree of light diffusion. The spread angle and spot size of the light after dispersion determine the brightness and viewing angle of the final imaging. The smaller the diffusion angle of the diffractive optical element 1002, the higher the imaging brightness. , The viewing angle is also smaller; the greater the dispersion angle of the diffractive optical element 1002, the smaller the imaging brightness, and the larger the viewing angle. The diffractive optical element 1002 can disperse the condensed light after being controlled by the light converging unit 1000 at a certain angle, so as to cover the required eye box area.

The eye box area refers to the area where the observer can observe the image presented by the light spot.

After the light passes through the diffractive optical element, it will diffuse and form a light spot 1061 of different shape. The size and shape of the light spot 1061 of different shape are determined by the microstructure of the diffractive optical element. The diffractive optical element adopts a beam shaper that can form a variety of light spot 1061 shapes. Various shapes of the light spot 1061 can be, but are not limited to: linear, circular, elliptical, square, rectangular, and batwing shapes.

The light converging unit 1000 is used to reflect the incident light in opposite directions, and emit the incident light in a direction opposite to the incident direction of the light, so as to prevent the incident light from diverging everywhere. The term "convergence" in the light converging unit 1000 refers to emitting the incident light in a direction opposite to the incident direction of the light, which essentially means counter-reflection.

For example, the above-mentioned light converging unit 1000 can be implemented in the following multiple ways to converge light,

The surface of the light converging unit 1000 may be composed of inverted triangular pyramid microstructures, preferably equilateral triangular triangular pyramid microstructures or isosceles triangular triangular pyramid microstructures; or may also be composed of cubic microstructures as shown in FIG. 20A. The microstructures can emit the incident light in the opposite direction of the incident direction of the light.

Referring to the structural schematic diagram of the second implementation of the light converging unit shown in FIG. 20B, the light converging unit includes: an inverted triangular pyramid microstructure 400, a supporting structure 402, and a substrate layer 404. The inverted triangular pyramid microstructure 400 is disposed on the On the supporting structure 402; the supporting structure 402 is arranged on the substrate layer 404; the refractive index of the inverted triangular pyramid microstructure 400 needs to be greater than the refractive index of the supporting structure 3802. The inverted triangular pyramid microstructure 400 reflects the incident light rays whose incident angle is greater than the critical angle to the dispersion element in a manner of total reflection in a direction opposite to the incident direction of the light rays.

The supporting structure 402 may adopt a regular triangular pyramid microstructure, so that the supporting structure 402 can be closely attached to the inverted triangular pyramid microstructure 400.

For example, when the supporting structure 402 is sufficient to support the inverted triangular pyramid microstructure 400, the substrate layer 404 may not be provided.

Referring to the schematic structural diagram of the third implementation of the light converging unit shown in FIG. 20C, the light converging unit 1000 may use counter-reflective particles to emit the incident light in a direction opposite to the incident direction of the light. The light condensing unit 1000 can be transparent spherical particles or ellipsoidal particles. It can be made of glass beads, transparent resin, polymer or other materials similar to glass. It can use exposed particles and sealed particles. , And embedded particles.

For example, the surface of the counter-reflection particles may be coated with a metal reflection layer; the metal reflection layer may be coated on the surface near the point O (ie, the reflection point) of the counter-reflection particles. As shown in FIG. 20C, the light converging unit mainly uses the metal reflective layer on the surface of the spherical counter-reflective particles. When the light hits the spherical counter-reflective particle, it is refracted at point P on the surface of the spherical counter-reflection particle. The refracted light is specularly reflected at point O, and then refracted at point Q, which is in the opposite direction of the incident direction of the light. Shoot out counter-reflective particles.

For example, light can also be phase modulated by metamaterials. After multiple phase cumulative changes in the metamaterial structure, the cumulative phase of incident light and outgoing light changes by π, thereby playing the role of counter-reflection.

Referring to the schematic structural diagram of the fourth implementation of the light converging unit shown in FIG. 20D, the light converging unit 1000 may be made of metamaterials, and includes: a light converging layer 500, an isolation layer 501, and a plane arranged in sequence in the light incident direction The reflective layer 502 and the substrate 503. The planar reflective layer 502 is located on the focal plane of the light converging layer 500.

The light condensing layer 500 and the planar reflective layer 502 are made of different metamaterials, respectively.

The light condensing layer, by changing the phase of the incident light, condenses the incident light onto the plane reflective layer, and reflects the light back from the plane reflective layer in the opposite direction of the light incident on the light condensing layer Direction reflection to the dispersive element;

The plane reflection layer can change the phase of the light condensed by the light converging layer, and reflect the light whose phase is changed to the light converging layer.

The different metamaterials refer to materials with different sizes, compositions, shapes, or arrangements.

The light rays in each part of the light converging unit 1000 made of metamaterials: the light condensing layer 400, the second isolation layer 401, the planar reflective layer 402, and the substrate 404, the phase cumulatively changes by π, the light made of metamaterials The converging unit 1000 has a counter-reflection effect on the light, so that the light can be reflected in a direction opposite to the incident direction of the light.

Since the windshield is not flat and has a certain curvature, there will be barrel distortion or pincushion distortion when imaging directly with the windshield. In at least one embodiment of the present disclosure, the light converging unit 1000 may be arranged according to a first distortion form, and the first distortion form is in an opposite and corresponding relationship with the second distortion form of the windshield.

For example, referring to Figures 21 and 22, when the light control device 100 arranged in a rectangular manner is used for imaging on the windshield 104, the light control device 100 arranged in a rectangular manner can form a virtual image on the windshield 104, but due to The windshield has a second distortion form, so the virtual image is a distorted image. The grid pattern on the windshield 104 in FIG. 21 represents a pincushion distortion virtual image. In at least one embodiment of the present disclosure, the first distortion shape corresponding and opposite to the second distortion shape of the windshield 104 is determined, and the light control device 100 is arranged according to the first distortion shape to eliminate the windshield The distortion caused. For example, referring to FIG. 22, the light control device 100 in at least one embodiment of the present disclosure is arranged according to the first distortion form, so that a virtual image without distortion can be formed on the windshield 104. The windshield 104 in FIG. 22 The grid pattern on the top represents a virtual image without distortion.

From the above content, it can be seen that by arranging the light control device 100 according to different arrangements, the imaging distortion caused by the curved windshield can be eliminated, so that the HUD imaging on the windshield is more regular.

After describing the structure of the second sub head-up display based on the above content, in order to achieve the purpose of multi-level imaging, the multi-level imaging system proposed in at least one embodiment of the present disclosure may also include: Augmented Reality Head Up Display (Augmented Reality Head Up Display). Display, AR-HUD). AR-HUD is a kind of HUD, which can achieve a good visual experience.

For example, the augmented reality head-up display is based on at least the principle of augmented reality for display. For example, the augmented reality head-up display may also be based on mixed augmented reality (MR).

For example, the light emitted by the image source in the AR-HUD is reflected on the curved mirror, and the reflected light is emitted to the eye box area, so that the observer can observe the image formed outside the windshield in the eye box area. The phenomenon of light reflection on the windshield can be approximated as a flat mirror imaging. According to the structure and optical principle of AR-HUD, the light emitted by the image source is reflected on the curved mirror and then emitted to the windshield. It can be considered that an equivalent image of the emitted light is formed on the side of the curved mirror away from the image source. According to the law of plane mirror imaging, the image formed by AR-HUD and the equivalent imaging source are symmetrical about the reflective medium.

The equivalent image source is an image formed on the side of the curved mirror away from the image source after the light emitted by the image source is reflected on the curved mirror.

Therefore, to form multiple images with different distances from the eye box area, the number of equivalent image sources in the AR-HUD must be increased, so that there are at least two equivalent image sources with different positions in the AR-HUD.

The position of the equivalent image source is determined by the imaging characteristics of the curved mirror, and the imaging law of the curved mirror is given by the following formula 1:

1/u+1/v=1/f (1)

Among them, f represents the focal length of the curved mirror; u represents the distance between the image source and the curved mirror, that is, the object distance of the image source on the curved mirror; v represents the distance between the image formed by the image source on the curved mirror and the curved mirror , That is, the image distance of the image source on the curved mirror. It can be seen from formula (1) that the position of the equivalent image source, that is, the position of the image formed by the image source on the curved mirror, is directly related to the image distance of the image source on the curved mirror. The larger the image distance, the farther the position of the equivalent image source of the image source is from the curved mirror.

For example, referring to the AR-HUD provided by at least one embodiment of the present disclosure shown in FIG. 23, the image source can be placed on the focal plane of the curved mirror or close to the focal plane of the curved mirror to increase the image distance of the image source.

Therefore, the position of the equivalent image source is related to the following two parameters: the focal length of the curved mirror and the image distance of the image source on the curved mirror (that is, the distance between the image source and the curved mirror).

For example, in AR-HUD, the focal length of the curved mirror is pre-designed and it is difficult to change it. Therefore, to form multiple images with unequal distances from the eye box area, it is necessary to increase the amount of light emitted from the image source to the curved mirror, so that the image source emits at least two rays of light to the curved mirror. After each path of light enters the curved mirror, it can form equivalent image sources with different positions.

For example, after there are equivalent image sources with different positions in the AR-HUD, according to the imaging principle shown in Figure 23, the equivalent image sources of the equivalent image sources with different positions can be separated on the windshield away from the eye box. One side of the area forms an image with a different distance from the eye box area, and the images with a different distance from the eye box area can be visually merged with scenes at different distances from the eye box area in the real environment.

In at least one embodiment of the present disclosure, the term "visually fused with the real environment (scene in the real environment)" refers to the image presented by the AR-HUD and the real environment ( The scene in the real environment) is completely fit/coincident together.

Based on this, referring to the schematic diagram of multiple images presented by the AR-HUD that can be seen in the eye box area shown in FIG. 24, the multi-level imaging system proposed by at least one embodiment of the present disclosure further includes: a communication connection with the ECU AR-HUD.

For example, the ECU is configured to send an image identifying the target to the AR-HUD when it is determined that there is a target that needs to be identified around the vehicle after processing the environmental information and the driving information of the vehicle.

For example, the AR-HUD is used to emit at least two rays of light to the windshield, and each of the at least two rays of light can respectively display multiple images with different distances from the eye box area; wherein , Each of the multiple images can be fused with a target whose distance matches.

For example, referring to the schematic structural diagram of the first AR-HUD proposed in at least one embodiment of the present disclosure shown in FIG. 25A, the AR-HUD includes a curved mirror 902 and at least two image sources 900.

For example, at least two image sources 900 can respectively emit at least two rays; each of the at least two rays of light is emitted by a different image source of the at least two image sources 900, and each of the rays is The length of the propagation path between the image source emitting each light beam and the curved mirror is different.

The curved mirror reflects at least two incident rays of light out of the AR-HUD, so that each of the at least two rays of light reflected out of the AR-HUD can form an eye box according to the length of the propagation path of each light. Images with unequal area distances.

The curved mirror 902 can be, but is not limited to: a spherical mirror, a hyperbolic mirror, a parabolic mirror, and a free-form surface mirror.

The image source 900 can emit light for presenting an image.

For example, the light emitted by each image source in at least two image sources 900 has different propagation path lengths between the image source emitting each light and the curved mirror, indicating that the object distances of each image source in the curved mirror are different. , And the focal length of the curved mirror is constant. According to the imaging law of the curved mirror given in the above formula 1, the image distance of each image source on the curved mirror is different, that is, the equivalent image source of each image source and the curved mirror The distances are different; therefore, the light emitted by each image source can form images with unequal distances from the eye box area.

For example, the image is an image that matches the distance of the scene that needs to be fused in the real environment.

For example, the scene may be, but is not limited to: motor vehicles, non-motor vehicles, pedestrians, animals, lane change areas, and repair sections in front of the road where the vehicle is located, damaged sections in front of the road where the vehicle is located, and the road where the vehicle is located. There are obstacles ahead, and there is a traffic accident section in front of the road where the vehicle is located.

For example, in order to achieve the purpose of multi-level imaging, the ECU also stores the corresponding relationship between the image source identification and the image distance of each image source in the AR-HUD.

For example, the image distance of each image source is set before the AR-HUD leaves the factory.

For example, the image distance of each image source can be set to a few meters, ten meters, tens of meters, and farther away. It can be stored in the ECU through the correspondence between the image source identification and the image distance.

The process of determining the image matching the scene distance includes: the ECU calculates the difference between the distance between the scene and the eye box area and the image distance of each image source, and calculates the image distance with the smallest distance between the scene and the eye box area. The corresponding image is determined as an image matching the distance of the scene.

According to the imaging law of curved mirror given by the above formula 1, it is found that when the image source is placed on the focal plane of the curved mirror or close to the focal plane, the image can be presented at an infinite distance from the eye box area, so that the observer can see There is no parallax image.

The parallax refers to the situation where the image seen by the observer's eyes in the eye box area and the scene in the real environment cannot be merged in the horizontal direction and/or the vertical direction.

For example, the at least two image sources in the AR-HUD include: a long-distance imaging image source set close to the focal plane of the curved mirror or set at the focal plane of the curved mirror, so as to be infinite Distant imaging,

The light emitted by the long-distance imaging image source can form a long-distance image after exiting the AR-HUD.

The long-distance image is used to eliminate parallax when the observer views the image.

Referring to the imaging side view of AR-HUD and full-window HUD shown in FIG. 37A, among the images presented by at least two image sources, the distance between the remote image presented by the remote imaging source and the eye box area is the longest. far. That is, the long-distance image is the image with the farthest horizontal distance from the eye box area among the images presented by at least two image sources.

The schematic diagrams of displaying images when the AR-HUD and the full-window HUD are working at the same time are shown in Fig. 37B and Fig. 37C.

For example, the case that the AR-HUD structure shown in FIG. 25A includes only two image sources is only for illustration, and the AR-HUD may also include more image sources, which will not be repeated here.

For example, referring to the schematic structural diagram of the second type of AR-HUD proposed in at least one embodiment of the present disclosure shown in FIG. 25B, the AR-HUD further includes a plane mirror 904.

The flat reflecting mirror 904 reflects at least two rays of light emitted by the at least two image sources to the curved mirror.

For example, referring to the schematic structural diagram of the third AR-HUD proposed in at least one embodiment of the present disclosure shown in FIG. 25C, the AR-HUD includes: a first image source 906, a second image source 908, a third image source 910, The first transflective film 912, the second transflective film 914, the plane mirror 904, and the curved mirror 902.

The first image source 906, the second image source 908, and the third image source 910 are respectively arranged at different positions in the AR-HUD; the first image source 906 can emit a first light , The second image source 908 can emit a second light; the third image source 910 can emit a third light.

The first transflective film 912 is disposed between the first image source and the second image source.

The second transflective film 914 is disposed between the second image source and the first transflective film.

The first transflective film 912 can transmit the incident first light and reflect the incident second light and third light.

The second transflective film 914 can transmit the incident first light and the second light and reflect the incident third light.

The plane mirror reflects the incident first light, the second light, and the third light to the curved mirror.

The curved mirror reflects the incident first light out of the AR-HUD, so that the first light reflected out of the AR-HUD can form a first image according to the propagation path length of the first light.

The curved mirror reflects the incident second light out of the AR-HUD, so that the second light reflected out of the AR-HUD can form a second image according to the propagation path length of the second light.

The curved mirror reflects the incident third light out of the head-up display, so that the third light reflected out of the head-up display can form a third image according to the propagation path length of the third light.

The propagation path lengths of the first light, the second light, and the third light are different, so that the distance between the first image and the eye box area, and the distance between the second image and the observer , And the distance between the third image and the eye box area is different.

The first light, the second light, and the third light have different polarization characteristics.

For example, the first light is S linearly polarized light, the second light is P linearly polarized light, and the third light is circularly polarized light.

For example, referring to the schematic structural diagram of the fourth type of AR-HUD proposed in at least one embodiment of the present disclosure shown in FIG. 25D, the second transflective element 914 can also be disposed on the first image source 906 and the first Between the transflective film 912.

The first transflective film can reflect the incident first light and transmit the incident second light and third light.

The first light emitted by the first image source, the second light emitted by the second image source, and the third light emitted by the third image source have different propagation path lengths, which means that the first image source The object distances of the second image source and the third image source are different, resulting in different image distances of the first image source, the second image source, and the third image source, so that the The first image source, the second image source, and the third image source can respectively image images at different positions from the eye box area to achieve the purpose of multi-level imaging.

For example, the first image source, the second image source, or the third image source may be arranged at a position close to the focal plane of the curved mirror or at the position of the focal plane of the curved mirror, The light emitted by the first image source, the second image source, or the third image source that is arranged at a position close to the focal plane of the curved mirror or at the position of the focal plane of the curved mirror After the AR-HUD is shot, it can form a long-distance image, thereby eliminating the parallax between the observer's viewing image and the scene in the real environment.

Among the images respectively presented by at least two image sources, the distance between the remote image presented by the remote imaging image source and the eye box area is the farthest. The long-distance image is merged with the scene at the farthest position of the eye box area.

For example, when there are motor vehicles or non-motor vehicles around the vehicle, the ECU may perform the following steps:

Determine the distance between the eye box area and the motor vehicle or non-motor vehicle;

When the distance between a motorized or non-motorized vehicle and the vehicle is less than the safe distance threshold, the motorized or non-motorized vehicle whose distance to the vehicle is less than the safe distance threshold is determined as the target to be identified;

Taking the distance between the eye box area and the target as a target image distance, and selecting an image source matching the target image distance from a plurality of image sources of the AR-HUD as the target image source;

An image fused with the target is generated, and the target image source is controlled to send an image fused with the target to the target, and the target is identified.

For example, in the above steps, in order to determine the distance between the eye box area and the motor vehicle or non-motor vehicle, the following steps are included:

Acquire the distance information sent by the distance measuring sensor, the distance information including: the sensor identifier of the distance measuring sensor that sends the distance information and the measured distance between the motor vehicle or the non-motor vehicle and the vehicle;

According to the sensor identifier, the distance between the distance measuring sensor corresponding to the sensor identifier and the eye box area is queried from the corresponding relationship between the sensor identifier of the distance measuring sensor and the distance;

Calculate the sum of the distance between the distance measuring sensor and the eye box area corresponding to the sensor identifier and the distance between the motor vehicle or non-motor vehicle and the vehicle recorded in the distance information, and determine the calculation result as the eye box area and the machine The distance between motor vehicles or non-motor vehicles.

In the above steps, the distance information further includes: images around the vehicle collected by the image sensor.

For example, the image sensor collects images around the vehicle, measures the distance between the moving target and the vehicle measured by the sensor, and generates distance information based on the sensor identifier of the distance measuring sensor itself, the collected image around the vehicle, and the distance between the moving target and the vehicle, and Send to ECU.

For example, in the above steps, the corresponding relationship between the sensor identifier of the ranging sensor and the distance is cached in the HUD control device.

For example, in the above steps, the process of selecting the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image source, and the above-mentioned image matching the scene distance The determination process is similar and can include the following steps:

Acquiring, from the correspondence between the image source identifier and the image distance, the image distance of each image source in the AR-HUD except for the long-distance imaging image source among the multiple image sources;

Respectively calculating the difference between the image distance of each of the image sources and the target image distance;

Determine the image source corresponding to the image distance with the smallest difference between the target image distance and the target image distance smaller than the difference threshold among the image sources as the target image source matching the target image distance ；

When the difference between the image distance of each obtained image source and the target image distance is greater than the difference threshold, the long-distance imaging image source is determined as the target image source that matches the target image distance.

For example, when the distance between a moving target and the vehicle is less than the safe distance, the distance is used as the target image distance, and the image source matching the target image distance is selected from a plurality of image sources as the target image source; The target image source sends an image fused with the moving target to the moving target to identify the moving target, thereby assisting the driving of the vehicle.

For example, when the target is a motor vehicle, the driving information includes: vehicle speed information; the environment information includes: distance information between the vehicle and surrounding vehicles; and the surrounding vehicles include: A front vehicle in front of the vehicle, a rear vehicle located behind the vehicle, and a side vehicle located on both sides of the vehicle.

For example, referring to the flow chart of the full-window HUD shown in FIG. 26 for identifying the distance between the surrounding vehicles of the vehicle, the ECU is used to obtain the driving information of the vehicle and combine the driving information of the vehicle with the sensor The sent environmental information is sent to the full window HUD for display, including the following specific steps:

Acquire the first distance from the front vehicle, the second distance from the rear vehicle, and the third distance and the fourth distance from the side vehicles on both sides of the vehicle, respectively.

According to the vehicle speed information of the vehicle, a first safe distance threshold between the vehicle and the preceding vehicle is determined.

When the first distance is less than or equal to the determined first safety distance threshold, first warning information is generated, and the first warning information is sent to a full-window HUD for display.

When the second distance is less than or equal to the second safety distance threshold, second warning information is generated, and the second warning information is sent to the full-window HUD for display; wherein, the second safety distance threshold is used for Indicates the safe driving distance between the vehicle and the rear vehicle.

When the third distance and/or the fourth distance is less than or equal to the third safety distance threshold, third warning information is generated, and the third warning information is sent to the full-window HUD for display; wherein, the The third safe distance threshold is used to indicate the safe driving distance between the vehicle and the side vehicle.

For example, in the above steps, the first distance between the vehicle and the vehicle in front is acquired through the first distance information sent by the distance-measuring sensor installed on the head of the vehicle, and the distance between the vehicle and the vehicle in front is obtained through the distance-measuring sensor installed at the rear of the vehicle The second distance information acquires the second distance to the rear vehicle, and the third distance information and the fourth distance information respectively sent by the distance measuring sensors installed on both sides of the vehicle are used to acquire the side vehicles located on both sides of the vehicle. The third distance and the fourth distance.

For example, the specific process of generating the first distance information, the second distance information, the third distance information, and the fourth distance information is at least partly similar to the foregoing process of generating distance information, and will not be repeated here. .

For example, the first distance information includes, but is not limited to: the sensor identifier of the distance measuring sensor that sends the first distance information, the first distance between the vehicle and the preceding vehicle, and the vehicle and the preceding vehicle The path image between.

For example, the content included in the second distance information, the third distance information, and the fourth distance information is similar to the above-mentioned first distance information, and will not be repeated here.

For example, in the above steps, the ECU may query the first safety distance threshold corresponding to the vehicle speed information of the vehicle according to the correspondence between the vehicle speed and the safety distance cached by the ECU itself.

For example, the corresponding relationship between vehicle speed and safety distance can be expressed as follows:

When the vehicle speed is less than or equal to 20 km/h, the safe distance is 10 meters;

When the vehicle speed is greater than 20 km/h and less than or equal to 40 km/h, the safe distance is 20 meters;

When the vehicle speed is greater than 40 km/h and less than or equal to 60 km/h, the safety distance is 30 meters;

When the vehicle speed is greater than 60 km/h and less than or equal to 100 km/h, the safety distance is 60 meters.

For example, the ECU also stores a second safety distance threshold and a third safety distance threshold.

For example, the second safe distance threshold is used to indicate the safe driving distance between the vehicle and the rear vehicle.

For example, the third safe distance threshold is used to indicate the safe driving distance between the vehicle and the side vehicle.

For example, in the above steps, the first warning information is used to indicate that the distance between the vehicle and the preceding vehicle is less than a first safe distance threshold.

For example, in the above steps, the second warning information is used to indicate that the distance between the vehicle and the preceding vehicle is less than a second safe distance threshold.

For example, in the above steps, the third warning information is used to indicate that the distance between the vehicle and the side vehicle on at least one side of the vehicle itself is less than the third safe distance threshold.

For example, the display mode of the second early warning information and the third early warning information is similar to the display mode of the first early warning information, and will not be repeated here.

For example, in the above steps, the display modes of the first early warning information, the second early warning information, and the third early warning information may include, but are not limited to, the following multiple modes:

Warning text, images, and videos, such as "Too close to the vehicle ahead, please slow down", "Please keep a distance from the vehicle on the right"; or brightly colored, prominent icons, or prompt animations, such as red prompt information; and related prompts The brightness of the information is higher than the brightness of other parts of the AR-HUD, or effects such as scrolling, flashing, and beating can also be used to further improve the warning effect.

For example, the displayed position should be at least concentrated in front of the driver, and it can also be displayed in front of the co-pilot at the same time to remind passengers.

For example, on the basis of the AR-HUD displaying the second warning information and the third warning information, the vehicle or the audio playback device on the vehicle can be used for reminding by voice broadcast, which can be an alarm bell with no specific meaning. , Or it can be a specific voice reminder such as "Attention! Keep the distance between cars."

For example, with other equipment, such as mechanical vibration equipment integrated on the steering wheel, mechanical equipment integrated in the seat, reminders through vibration.

For example, warning texts, images, and videos are displayed on the side windows and rear windows of the vehicle, such as "rear vehicles please keep the distance", "side vehicles please keep the distance" or brightly colored and prominent icons, or prompt animations, such as red Prompt information; and the brightness of the related prompt information is higher than the brightness of other parts of the HUD, or effects such as scrolling, flashing, and beating can also be adopted to further improve the warning effect.

For example, referring to the schematic diagram in which the full-window HUD shown in FIG. 27 displays the first warning information and the AR-HUD identifies the path between the vehicle and the motor vehicle that is too close, the first warning information can be sent to the full-window HUD. While the window HUD is displayed, the AR-HUD can also be used to identify the path between the vehicle and the motor vehicle that is too close.

For example, in addition to the top of the windshield, the first warning information can also be displayed on other positions of the windshield, which will not be repeated here.

For example, referring to the flowchart shown in FIG. 28 when the vehicle is too close to the vehicle in front, the AR-HUD is controlled to display the flow chart showing the path between the vehicle and the vehicle in front of the vehicle that is too close; the ECU is used to After processing the environmental information and the driving information of the vehicle, when it is determined that there is a target that needs to be identified around the vehicle, sending an image that identifies the target to the AR-HUD includes performing the following steps:

Acquiring a path image between the vehicle and the preceding vehicle;

Identifying the path between the vehicle and the preceding vehicle according to the acquired path image between the vehicle and the preceding vehicle;

An image distance matching the first distance is used as a target image distance, and an image source matching the target image distance is selected from a plurality of image sources of the AR-HUD as the target image source; wherein, The image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror;

Use the first color mark to generate a first path identification instruction, and send the generated first path identification instruction to the target image source in the AR-HUD, and control the target image source to connect the vehicle with the front The path between the vehicles is displayed in the color corresponding to the first color mark;

When it is determined that the first distance is greater than the first safe distance threshold, select a target image source from the AR-HUD according to the first distance;

Use the second color identification to generate a second path identification instruction, and send the generated second path identification instruction to the target image source, and control the target image source to transfer the path between the vehicle and the preceding vehicle The color corresponding to the second color mark is displayed.

In the above steps, the path image between the vehicle and the preceding vehicle is obtained from the first distance information.

In the above steps, the ECU may use image processing technology to recognize the path between the vehicle and the preceding vehicle based on the acquired path image between the vehicle and the preceding vehicle. The specific process is here No longer.

In the above steps, the image distance matching the first distance is the distance between the eye box area and the vehicle in front of the vehicle. Therefore, the process of determining the image distance matching the first distance is It is similar to the above process of determining the distance between the eye box area and the moving target, and will not be repeated here.

In the above steps, the colors corresponding to the first color identifier may be red and pink.

In at least one embodiment, the first warning information may be "too close to the vehicle in front, please slow down".

Further, the ECU caches a table of correspondence between the distance and color of the motor vehicle and the vehicle.

In at least one embodiment, the corresponding table of the distance and color between the motor vehicle and the vehicle can be expressed as follows:

When the distance between the motor vehicle and the vehicle is between 0 and 50 cm, it can be expressed in red;

When the distance between the motor vehicle and the vehicle is 50 to 75 cm, it can be expressed as yellow.

For example, the ECU may also render the first warning information according to colors corresponding to the first distance between the vehicle and the preceding vehicle. The rendered first warning information is sent to the full-window HUD for display.

For example, in the above steps, the process of selecting the target image source is described in the above steps of selecting the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image source. The process is similar, so I won't repeat it here. It is determined that the first distance is greater than the first safe distance threshold, indicating that the vehicle currently maintains a safe driving distance.

For example, in the above steps, the color corresponding to the second color identifier may be green or no color.

For example, while controlling the target image source to display the path between the vehicle and the preceding vehicle in the color corresponding to the second color mark, at the same time, after confirming that a safe driving distance is maintained, it is displayed through the traditional HUD method. Prompt texts, images, videos, such as "safe distance, please continue to maintain"; or icons with a large color difference from the alarm display, or prompt animations, such as green prompt messages, or scrolling, flashing, beating, etc. The effect can disappear after being displayed for a certain period of time; or it can always remain on the screen and be shown to the observer.

For example, while displaying images through AR-HUD and full-window HUD, the ECU can also perform the following steps:

When the first distance is less than the braking operation length threshold and the first distance is less than or equal to the determined first safety distance threshold and the duration is greater than the first preset duration threshold, a braking instruction is generated.

Using the generated braking instruction, the vehicle is controlled to perform a braking operation.

For example, in the above steps, the braking operation length threshold is buffered in the ECU and can be set to 15 meters. The braking operation length threshold value can also be set to be less than the safety distance value recorded in the correspondence between any vehicle speed and safety distance, which will not be repeated here.

For example, the first preset duration threshold is cached in the ECU and can be set to 20 seconds.

For example, when the target is a non-motor vehicle, referring to the flowchart showing collision warning information shown in FIG. 29, the ECU is used to obtain the driving information of the vehicle, and combine the driving information of the vehicle with the sensor The sent environmental information is sent to the full window HUD for display, including the following specific steps:

When it is determined that the location of the vehicle is a crowded area, acquiring an image of the surrounding environment of the vehicle;

When it is determined that there is a non-motorized vehicle around the vehicle according to the surrounding environment image, determine the distance between the non-motorized vehicle and the vehicle;

When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, collision warning information is generated, and the distance between the non-motor vehicle and the vehicle and the collision warning information are sent to the global Car window HUD is displayed.

For example, in the above steps, the ECU determines the location of the vehicle from the location information sent by the navigation device. When it is determined that the location of the vehicle is in a school, a hospital, a parking lot, or a prosperous urban area, it is determined that the location of the vehicle is a crowded area, and an image acquisition instruction is sent to the image sensor installed on the vehicle, so that The image sensor installed on the vehicle collects an image of the surrounding environment of the vehicle.

For example, after the image sensor receives the image acquisition instruction, after collecting the surrounding environment image, when it is determined that there is a non-motorized vehicle around the vehicle, the distance measuring sensor determines the distance between the non-motorized vehicle and the vehicle, and then calculates the distance between the non-motor vehicle and the vehicle. The surrounding environment image, the determination of the distance between the non-motor vehicle and the vehicle existing around the vehicle, and the sensor identifier of the distance measuring sensor itself generate non-motor vehicle distance information and send it to the ECU.

For example, in the above steps, the collision safety distance threshold is cached in the ECU and can be set to any distance between 2 meters and 5 meters.

For example, the collision warning information is used to indicate that the distance between the non-motor vehicle and the vehicle is less than a collision safety distance threshold.

For example, in at least one embodiment, the collision warning information may be “too close to the pedestrian ahead, please pay attention”.

For example, in the above steps, the manner of displaying the collision warning information may include, but is not limited to, the following multiple manners:

It can be warning text, image, video; or bright and prominent icon, or prompt animation, such as red prompt message; or the brightness of the prompt message is higher than the brightness of other parts of the HUD display, or it can also be scrolled or flashed , Beating and other effects to further improve the warning effect. The displayed position should be at least concentrated in front of the driver, and it can also be displayed in front of the co-pilot at the same time to remind passengers.

Use red and other bright-colored shapes or signs to mark and prompt key information. At the same time, for example, accompanied by voice reminders, such as "there is a pedestrian ahead, pay attention to avoid", to further enhance the reminder effect to the driver.

The display mode described above is similar to the display mode of the traditional HUD, and will not be repeated here.

For example, referring to the schematic diagram of the full-window HUD displaying collision warning information and AR-HUD marking the vehicle and the non-motor vehicle that are too close to each other as shown in FIG. 30, the collision warning information can be sent to the full-window HUD for display At the same time, the AR-HUD can also be used to identify vehicles that are too close to non-motor vehicles.

For example, referring to the flowchart shown in FIG. 31 for controlling the AR-HUD to identify the non-motor vehicle that is too close to the vehicle when the vehicle is too close to the non-motorized vehicle; the ECU is used for when the vehicle is too close to the non-motorized vehicle. After processing the environmental information and the driving information of the vehicle, when it is determined that there is a target that needs to be identified around the vehicle, sending the image that identifies the target to the AR-HUD includes the following specific steps:

Determining the location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle;

When the distance between the non-motor vehicle and the vehicle is greater than the collision safety distance threshold and less than or equal to the warning distance threshold, select the target image source from the AR-HUD according to the distance between the non-motor vehicle and the vehicle;

Controlling the target image source to use the color corresponding to the fourth color mark to generate an early warning graphic at the location of the non-motor vehicle; wherein the early warning graphic is fused with the non-motor vehicle;

When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, the fusion with the non-motor vehicle is determined according to the location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle Determining the first size of the non-motor vehicle indicating frame in the image of the surrounding environment, and determining the first shape of the non-motor vehicle indicating frame based on the contour of the non-motor vehicle in the surrounding environment image;

Use the image distance matching the distance between the non-motor vehicle and the vehicle as the target image distance, and select the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image Source; wherein, the image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror;

Use the third color identification, the first size and the first shape of the non-motor vehicle indicator frame to generate a non-motor vehicle identification instruction, and use the non-motor vehicle identification instruction to control the target image source to identify the color corresponding to the third color , Generating the non-motor vehicle indicator frame according to the determined first size and first shape, and fusing the non-motor vehicle indicator frame and the non-motor vehicle together according to the location of the non-motor vehicle to combine all Said non-motor vehicle is marked out;

When it is determined that the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, obtain a non-motor vehicle image and determine the current location of the non-motor vehicle; step 1514, according to the location of the non-motor vehicle According to the current location and the distance between the non-motor vehicle and the vehicle, the second size of the non-motor vehicle indicator frame fused with the non-motor vehicle is determined, and based on the non-motor vehicle outline in the non-motor vehicle image, Determining the second shape of the non-motor vehicle indicating frame;

Selecting a target image source from the AR-HUD according to the distance between the non-motor vehicle and the vehicle;

Use the fifth color identification, the second size and the second shape of the non-motor vehicle indicator frame to generate a non-motor vehicle identification instruction, and use the non-motor vehicle identification instruction to control the target image source to identify the color corresponding to the fifth color , Generating a non-motor vehicle indication frame according to the determined second size and second shape, and fusing the non-motor vehicle indication frame with the non-motor vehicle based on the current location of the non-motor vehicle.

For example, in the above steps, the ECU may determine the location of the non-motor vehicle based on the surrounding environment image through any algorithm for determining the position of the object from a digital image, and the specific process is not repeated here.

For example, in the above steps, the warning distance threshold is cached in the ECU and can be set to any distance between 5 meters and 10 meters.

For example, the specific process of selecting the target image source from the AR-HUD according to the distance between the non-motor vehicle and the vehicle is to first determine the eye box area as the target distance according to the distance between the non-motor vehicle and the vehicle The distance to the non-motor vehicle, and the process of selecting the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image source. This process is the same as the process of selecting the image source from the AR-HUD as described above. The process described in steps (21) to (24) of selecting the image source matching the target image distance from the multiple image sources as the target image source is similar, and will not be repeated here.

For example, in the above steps, the color corresponding to the fourth color identifier may be a bright color such as red, green, and yellow.

For example, the warning graphic is pre-buffered in the ECU, and may be, but not limited to: a circle, an ellipse, and a quadrilateral.

For example, in the above steps, image processing technology can be used to determine the second non-motor vehicle indicator frame fused with the non-motor vehicle based on the location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle. The first shape of the non-motor vehicle indicating frame is determined based on the contour of the non-motor vehicle in the surrounding environment image. The specific process is not repeated here.

For example, in the above steps, an image distance that matches the distance between the non-motorized vehicle and the vehicle is used as the target image distance, and the AR-HUD is selected from a plurality of image sources that are closer to the target image distance. The process of using the matched image source as the target image source is similar to the process of determining the distance between the eye box area and the moving target, and will not be repeated here.

For example, in the above steps, the colors corresponding to the third color identifier may be red and pink. In the above steps, the colors corresponding to the fifth color identifier may be green and blue.

For example, based on the current location of the non-motor vehicle, the non-motor vehicle indicator frame and the non-motor vehicle are fused together, so that the observer can see the non-motor vehicle indicator frame and the non-motor vehicle through the eye box area. The non-motor vehicles are fused together to determine that the distance between the vehicle and the non-motor vehicle identified by the non-motor vehicle indicator frame is greater than the collision safety distance threshold.

For example, when it is determined that the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, it means that the vehicle is at a safe distance from the surrounding non-motor vehicles, or the specific process from the above steps to the steps may not be performed. Non-motor vehicles that are at a safe distance from the vehicle will not be marked.

The specific process of the above steps is similar to the specific process described above, and will not be repeated here.

In addition to the above steps, when it is determined that the distance between the non-motor vehicle and the vehicle is less than the emergency braking length threshold, the ECU may control the vehicle to perform a braking operation.

For example, the emergency braking length threshold may be any distance between 0.5 meters and 1.5 meters.

For example, the full-window HUD will also display weather information and the condition information of the road where the vehicle is located. Refer to the ECU controlling the full-window HUD shown in Figure 32 to display the weather information and the condition information of the road where the vehicle is located. In the flowchart, the ECU is used to obtain the driving information of the vehicle, and send the driving information of the vehicle and the environmental information sent by the sensor to the full-window HUD for display, including the following specific steps:

Obtaining weather information and/or status information of the road where the vehicle is located;

When it is determined that there are potentially dangerous driving targets around the vehicle based on the weather information and/or the condition information of the road where the vehicle is located, warning information is generated, and the weather information and/or the condition information of the road where the vehicle is located And the generated warning information is sent to the full window HUD for display.

For example, in the above steps, in order to obtain the weather information of the road where the vehicle is located and the condition information of the road where the vehicle is located, the ECU will determine the administrative area of the road where the vehicle is located according to the location information of the road where the vehicle is located, and then wirelessly The communication device accesses the Internet to obtain weather information of the administrative area and traffic information of the administrative area. The traffic information includes road condition information and status information of the road where the vehicle is located.

For example, the weather information of the road where the vehicle is located is used to indicate whether the road where the vehicle is located is in good weather or bad weather.

For example, the good weather conditions include but are not limited to: sunny, cloudy and sunny, and cloudy.

For example, the severe weather conditions include, but are not limited to: hail, heavy rain, floods, and tornadoes.

For example, the status information of the road where the vehicle is located includes, but is not limited to: including: reminder information about maintenance ahead of the road, reminder information about damage ahead of the road, reminder information about traffic accidents in front of the road, and obstacles Reminder information.

For example, there is a reminder message for maintenance ahead of the road, which is used to indicate that there is a maintenance section ahead of the road where the vehicle is located.

For example, the warning message that there is damage in front of the road is used to indicate that there is a damaged section in front of the road where the vehicle is located.

For example, the reminder information of a traffic accident in front of the road is used to indicate that there is a traffic accident section in front of the road where the vehicle is located.

For example, the reminder message that there is an obstacle in front of the road is used to indicate that there is an obstacle in front of the road where the vehicle is located.

For example, in the above steps, the warning information may be: "The weather is not good, please drive slowly" and "There is a traffic accident on the road ahead, please drive slowly".

For example, the following methods can be used to display the warning information:

Based on the AR enhancement system, the information of driving safety is analyzed. After virtual information is simulated and simulated, the application is displayed on the HUD. The real and virtual information complement each other to enhance the warning information.

For example, displaying information on the HUD will indicate the information that needs to be marked that will help driving, including: potentially threatening vehicles, potential road hazards, bad road conditions, and information that needs attention when driving at night;

For example, safety signs can be marked with green safety signs, and danger signs can be marked with red warning signs.

For example, based on real-time road conditions and networked road information, the driving route is assisted in the screen, and auxiliary lines and steering signs are marked on the correct driving road.

For example, the displayed information can be text, image, video, and the warning effect can be improved by using color deepening, scrolling, flashing, and jumping.

For example, the displayed position should be at least concentrated in front of the driver, and it can also be displayed in front of the co-pilot at the same time to remind passengers.

For example, by marking and prompting key information on AR-HUD, there is a potential threat to the vehicle, and the driver can be shown or voiced to remind the driver to threaten the vehicle information;

Mark and prompt key information on the AR-HUD. If there is a potential road hazard, the driver can be reminded of road condition information with graphics or voice.

For example, referring to Figure 33, when it is determined that the area where the vehicle is located is currently in severe weather, the AR-HUD can display the warning information while displaying the warning information on the full-window HUD, and the AR-HUD displays the intention of the image fused with the road. The part of the road that can be seen in the eye box area, and the dashed part is the part of the road shown in the path contour map.

For example, referring to the flow chart of the ECU controlling the AR-HUD to display images when the potentially dangerous target for driving is bad weather conditions shown in FIG. 34, the ECU is used to process the environmental information and the driving information of the vehicle. After it is determined that there is a target that needs to be identified around the vehicle, sending the image that identifies the target to the AR-HUD includes the following specific steps:

When it is determined that there is a bad weather condition on the road where the vehicle is located, acquiring a bad weather road image of the road where the vehicle is located and generating warning information;

Processing the bad weather road image to determine the brightness value of the road where the vehicle is located;

When the brightness value is less than the brightness threshold, generate high beam turn-on prompt information, display the high-beam turn-on prompt information through the full-window HUD, and determine from the bad weather road image the location of the road where the vehicle is located Path profile and path position;

Determine the distance between the road where the vehicle is located and the eye box area according to the path position;

The image distance that matches the distance between the road where the vehicle is located and the eye box area is used as the target image distance, and an image that matches the target image distance is selected from the multiple image sources of the AR-HUD A source as a target image source; wherein the image distance is the distance between the image formed by the image source on the curved mirror and the curved mirror;

Use the sixth color to fill the path contour of the road where the vehicle is located to obtain a path contour map of the road where the vehicle is located;

According to the path position of the road where the vehicle is located, the path contour map is projected onto the road where the vehicle is located through the target image source, so that the path contour map merges with the road where the vehicle is located.

For example, in the above steps, the ECU sends an image acquisition instruction to the image sensor installed on the vehicle, so that the image sensor installed on the vehicle collects bad weather road images of the road where the vehicle is located.

For example, after receiving the image acquisition instruction, the image sensor collects the bad weather road image, and then sends the bad weather road image to the ECU.

For example, in the above steps, the image brightness algorithm is used to process the bad weather road image, and the processed image brightness value is determined as the brightness value of the road where the vehicle is located. The specific process will not be repeated here.

For example, in the above steps, the process of displaying the high beam turn-on prompt information through the AR-HUD is similar to the process of displaying information of a traditional HUD, and will not be repeated here.

For example, the ECU may use any image processing technology to determine the path profile and path position of the road where the vehicle is located from the bad weather road image. The specific process is not repeated here.

For example, the prompt message for turning on the high beam may be "please turn on the high beam".

For example, in the above steps, the geometric center of the road where the vehicle is located is obtained from the path profile of the road where the vehicle is located, and the distance between the geometric center of the road where the vehicle is located and the vehicle is measured by a distance measuring sensor, and the vehicle The distance between the geometric center of the road and the vehicle, and the distance between the distance measuring sensor that measures the distance between the geometric center of the road and the vehicle and the eye box area are calculated, and the result of the sum calculation is determined as the vehicle The distance between the road and the eye box area.

For example, in the foregoing steps, the specific process of determining the target image source is similar to the foregoing process of determining the distance between the eye box area and the moving target, and will not be repeated here.

For example, in the above steps, the sixth color may be an eye-catching color such as red, blue, and green.

For example, in the above steps, as shown in FIG. 33, the dashed part is the road part shown in the path profile diagram.

For example, referring to the flow chart in which the ECU controls the AR-HUD to display images when the potentially dangerous target for driving is a dangerous situation in front of the road where the vehicle is located, as shown in FIG. After the driving information of the vehicle is processed, it is determined that there is a target that needs to be identified around the vehicle, and the image that identifies the target is sent to the AR-HUD, including the following specific steps:

For example, when the condition information of the road where the vehicle is located is obtained and the distance between the vehicle and the road section corresponding to the road condition where the dangerous condition occurs is less than the road condition display distance threshold, the road condition image of the road where the vehicle is located is obtained And generate warning information; wherein, the road condition image includes: a road section where a road condition occurs in the road where the vehicle is located;

For example, the road section profile and the road section position of the road section where the road condition occurs are determined from the road state image, and the distance between the road section where the dangerous situation occurs and the vehicle is determined according to the road section position; for example, the The distance can be obtained through the intelligent navigation system;

For example, take the image distance that matches the distance between the road section where the road condition occurs and the vehicle as the target image distance, and select the AR-HUD image source to match the target image distance The image source of as the target image source;

For example, using the seventh color to fill in the road section contour of the road section where the road condition occurs, to obtain the state contour map of the road section where the road condition occurs;

For example, the condition contour map is projected onto the road section where the road condition occurs through the target image source, so that the condition contour diagram is merged with the road section where the road condition occurs.

For example, in the above steps, the road section where the road condition occurs may be, but not limited to: a repaired road section, a damaged road section, a traffic accident road section, and a road section with obstacles.

The road condition display distance threshold can be any distance between 0 and 2 kilometers.

The ECU sends an image acquisition instruction to the image sensor installed on the vehicle, so that the image sensor installed on the vehicle collects road conditions images of the road where the vehicle is located.

For example, after the image sensor receives the image acquisition instruction, it collects the image of the road condition, and the distance measuring sensor measures the distance between the vehicle and the road section where the road condition occurs. According to the sensor identification of the image distance sensor itself, the road condition image, and the vehicle The distance from the road section where the road condition occurs generates road condition distance information, and the road condition distance information is sent to the ECU.

For example, in the above steps, any image processing algorithm can be used to determine the road section contour of the road section where the road condition occurs from the road condition image, and the specific process will not be repeated here.

For example, in the above steps, the process of determining the target image source is the same as the process described above in the step of selecting the image source matching the target image distance from the multiple image sources of the AR-HUD as the target image source. Similar, I won't repeat it here.

For example, in the above steps, the seventh color may be red, pink, and green.

For example, in some cases, it is necessary to evaluate the operation of the vehicle and give driving recommendations for the vehicle. Refer to the flowchart shown in FIG. 36 which is executed by the ECU when giving driving recommendations for the vehicle. The ECU is also specifically used To perform the following steps:

When the operation advice information of the vehicle is generated according to the navigation information, the driving speed, acceleration, and vehicle status information of the vehicle, the vehicle operation video of the observer is collected, and the vehicle operation video is collected through the full-window HUD. The operation suggestion information display;

When it is determined that the operation corresponding to the operation advice information is not detected within the specified operation time, dangerous driving information is generated, the dangerous driving information is displayed through the full-window HUD, and the driving vehicle index is decremented;

When it is determined that the operation corresponding to the operation suggestion information is detected within the specified operation time period, an incremental operation is performed on the driving vehicle index;

When it is determined that the length of time the vehicle has not been operated is greater than the parking time threshold, stop collecting vehicle operation videos, and send the driving vehicle index and the collected vehicle operation videos to the full-window HUD for display.

In the above steps, the navigation information is obtained from a navigation device; the driving speed, acceleration, and vehicle state information of the vehicle are obtained from OBD.

The process of generating the operation suggestion information of the vehicle according to the navigation information, the driving speed, acceleration, and vehicle state information of the vehicle is not repeated here.

The operation advice information includes, but is not limited to: recommended driving route information, recommended gear information, obstacle prompt information, recommended speed information, and recommended braking information.

In the above steps, the specified operation duration can be any length of time between 3 to 5 seconds.

The driving vehicle index is used to evaluate the reasonable degree of vehicle operation; the larger the driving vehicle index, the more reasonable the vehicle operation.

Decreasing the driving vehicle index refers to subtracting a fixed index variable from the current driving vehicle index to obtain the driving vehicle index after the decrement operation.

In the above steps, the process of incrementing the driving vehicle index is similar to the process of decrementing the driving vehicle index, and will not be repeated here. In summary, the multi-level imaging system proposed by at least one embodiment of the present disclosure is provided with a full-window HUD, and the arrangement of the full-window HUD matches the visible area of the windshield, so that all The image presented by the light emitted by the full-window HUD can cover the visible area of the windshield. Compared with the conventional HUD based on free-form surface mirrors and smaller FOV in the related art, the full-window HUD has more The arrangement of the individual light sources matches the visible area of the windshield, so that the light emitted by the full-window HUD can display an image covering the visible area of the windshield. The purpose of displaying images at any position in the visible area, so that richer content can be displayed through the full-window HUD, which improves the HUD experience.

At least one embodiment of the present disclosure also provides a vehicle, including the multi-level imaging system or the head-up display provided in any of the above embodiments.

For example, the vehicle further includes an imaging window, the head-up display includes a first sub-head-up display and a second sub-head-up display, and the second sub-head-up display includes a projection device and a light control device; The light to the light control device; the light control device is used to change the exit direction of the light emitted by the projection equipment, so that the horizontal field angle of the second sub-head-up display is not less than 15 degrees and the vertical direction The field of view angle is not less than 5 degrees; the projection equipment is installed in the vehicle; the light control device is installed on the surface of the center console of the vehicle opposite to the imaging window.

For example, the vehicle may include, but is not limited to, a vehicle, an aircraft, or a water vehicle, which is not limited in at least one embodiment of the present disclosure.

At least one embodiment of the present disclosure further provides an imaging method of a multi-level imaging system, the multi-level imaging system includes an eye box area and a head-up display, the imaging method includes: acquiring driving information and sending it to the head-up display; The driving information controls the head-up display to emit multiple rays of light to respectively display multi-layer images to display the driving information, and at least part of the multi-layer images are at different distances from the eye box area.

For example, controlling the head-up display to emit multiple lights according to the driving information to display multiple layers of images to display the driving information includes: forming display data according to the driving information, and comparing the display data according to the personal information of the driver. Process, and control the full-window head-up display to emit light to the windshield of the vehicle according to the processed display data.

For example, the processing includes at least one of the following: image color saturation enhancement processing, image color conversion processing, and image shape conversion processing.

For example, processing the display data according to the driver's personal information includes: identifying the driver's identity information, and determining the driver's personal information based on the identity information.

For example, personal information, such as those with color weakness/color blindness, can be stored in the in-car processor. The in-car identification system can identify the current user’s identity information through iris, fingerprint, voiceprint, face and other information. Query the user’s color perception analysis results according to the identity information, and determine the subsequent image information conversion mode. If the driver is color weak, the image color saturation will be enhanced, and the red-green-blind driver will convert the red-green image. Blue-yellow blindness The driver converts the yellow image; then analyzes the driver's color perception through the information collection and testing device installed in the vehicle, and determines the subsequent image information conversion mode based on the analysis result. If the driver is color weak, the image color saturation is enhanced Red-green-blind drivers convert the red-green images, and blue-yellow-blind drivers convert the yellow images. At the same time, the collected information can also be stored for later recall.

For example, it is possible to collect driving information through sensors installed inside and outside the vehicle, including vehicle location information, surrounding environment information, etc., for example, including:

(1) Collect the location information of the vehicle through the vehicle's own or external location sensor, such as GPS sensor. When the vehicle is approaching or in a scene with traffic lights, the image is activated when the vehicle is approaching or in a scene with traffic lights. Sensors to collect environmental information around the vehicle;

(2) Or directly collect environmental information around the vehicle through image sensors installed inside and outside the vehicle, such as a camera, etc.;

For example, analyze and process the collected data to identify whether there are signal lights near the vehicle. If so, collect the image of the signal light, and the state of the traffic light at this time, including color, shape, etc.; or collect traffic lights such as traffic lights, waiting time, street signs, etc. Image information; or, collect the lamp information of other vehicles, such as the turn signal of the vehicle ahead, dual flashing lights, etc.;

For example, converting the data after analysis and processing includes:

(1) To color the collected signal light image data, adjust the saturation of the image, and convert it into a high-saturation image;

(2) Perform conversion processing on the collected signal light image data. For example, the general signal light is circular, at this time, the red light is converted into a square color block image, and the green light is converted into a triangular color block image with text display;

(3) Perform conversion processing on the collected signal lamp image data, such as converting red and green lights into red-green color blindness recognizable color images;

(4) On the basis of the above scheme, convert the collected image data into text display and voice data. If the red light is on, the digital logo has 90 seconds, and the text is displayed on the HUD with the above display method. The data broadcast "Red light at this time, there are 90 seconds, 89, 88".

For example, the converted image information is displayed on the HUD, so that drivers with color weakness/color blindness can receive real-time traffic signals, and at the same time, they can cooperate with voice reminders to further enhance the prompt effect.

The above are only exemplary implementations of the present disclosure, and are not used to limit the protection scope of the present disclosure, which is determined by the appended claims.

Claims (38)

1. A multi-level imaging system has an eye box area and includes: an electronic control unit and a head-up display,

   Wherein, the electronic control unit is in communication connection with the head-up display;

   The electronic control unit is used to obtain driving information and send the driving information to the head-up display, and

   The head-up display is configured to display the driving information;

   The head-up display is configured to emit multiple channels of light to respectively display multiple layers of images to display the driving information, and at least part of the multiple layers of images are at different distances from the eye box area.
2. The multi-level imaging system according to claim 1, wherein the head-up display comprises a first sub-head-up display, and the first sub-head-up display is an augmented reality head-up display.
3. The multi-level imaging system according to claim 2, wherein:

   The augmented reality head-up display is configured to display a single-layer image; or

   The augmented reality head-up display is configured to emit at least two rays of the multiple rays of light to display at least two layers of the multi-layer image.
4. The multi-level imaging system according to any one of claims 2 or 3, wherein the augmented reality head-up display includes an image source, a flat mirror and a curved mirror, and the light emitted by the image source is reflected by the curved mirror and then enters the Said plane mirror, and emits after being reflected on said plane mirror; or,

   The augmented reality head-up display includes a curved mirror and at least two image sources;

   The at least two image sources are used to respectively emit the at least two paths of light; the at least two paths of light have different propagation path lengths between the image source that respectively emits the respective paths of light and the curved mirror;

   The curved mirror is used to reflect the at least two rays of light incident on the curved mirror.
5. The multi-level imaging system according to any one of claims 2-4, wherein the at least two image sources included in the augmented reality head-up display include a first image source and a second image source, and the augmented reality head-up display It also includes the first transflective membrane;

   The first image source is configured to emit a first light, and the second image source is configured to emit a second light;

   The first transflective film is configured to transmit the first light and reflect the second light;

   Alternatively, the at least two image sources included in the augmented reality head-up display include a first image source, a second image source, and a third image source, and the augmented reality head-up display further includes a first transflective film and a second transflective film. Anti-film

   The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light;

   The first transflective film is configured to transmit the first light and reflect the second light and the third light;

   The second transflective film is configured to reflect the third light and transmit the first light;

   or,

   The at least two image sources include a first image source, a second image source, and a third image source, and the augmented reality head-up display further includes a first transflective film and a second transflective film;

   The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light;

   The first transflective film is configured to transmit the first light and the third light and to reflect the second light;

   The second transflective film is configured to transmit the first light and reflect the third light;

   or,

   The at least two image sources include a first image source, a second image source, and a third image source. The head-up display further includes a first transflective film, a second transflective film, and a first mirror; the first The reflector includes a flat reflector;

   The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light;

   The first transflective film is configured to transmit the second light and reflect the first light;

   The second transflective film is configured to transmit the second light and reflect the third light;

   The plane mirror is configured to reflect the first light, the second light, and the third light;

   or,

   The at least two image sources include a first image source, a second image source, and a third image source. The head-up display further includes a first transflective film, a second transflective film, and a first mirror; the first The reflector includes a flat reflector;

   The first image source is configured to emit a first light, the second image source is configured to emit a second light, and the third image source is configured to emit a third light;

   The first transflective film is configured to transmit the second light and reflect the first light;

   The second transflective film is configured to transmit the second light and reflect the third light;

   The plane mirror is configured to reflect the first light, the second light, and the third light.
6. The multi-level imaging system according to any one of claims 2-4, wherein the at least two image sources include a first image source and a second image source, and the augmented reality head-up display further includes a first mirror;

   The first image source is configured to emit a first light, and the second image source is configured to emit a second light;

   The first reflecting mirror is configured to reflect the first light and the second light;

   The first reflecting mirror includes a flat reflecting mirror;

   The plane reflecting mirror reflects the at least two rays of light emitted by the at least two image sources to the curved mirror;

   or,

   The at least two image sources include a first image source and a second image source, and the augmented reality head-up display further includes a first mirror and a second mirror;

   The first image source is configured to emit a first light, and the second image source is configured to emit a second light;

   The first reflecting mirror is configured to reflect the first light;

   The second reflecting mirror is configured to reflect the second light.
7. The multi-level imaging system according to any one of claims 2-6,

   Wherein, the electronic control unit is further configured to send the information identifying the external target to the first head-up display when the external target that needs to be identified is determined after processing the driving information;

   The image displayed on the first sub-head-up display matches the distance of the external target and merges with the external target.
8. The multi-level imaging system according to claims 2-7, wherein the head-up display further comprises a second sub-head-up display, and the horizontal field angle of the second sub-head-up display is not less than 15 degrees, and the second head-up display The vertical viewing angle of the sub-head-up display is not less than 5 degrees.
9. 8. The multi-level imaging system according to claim 8, wherein the second sub-head-up display comprises a projection device and a light control device;

   The projection equipment is used to emit light that can be incident on the light control device;

   The light control device is used to change the exit direction of the light emitted by the projection device, so that the horizontal field angle of the second head-up display is not less than 15 degrees and the vertical field angle is not less than 5 degrees.
10. 9. The multi-level imaging system according to claim 9, wherein the light control device comprises: a light converging unit and a dispersion element;

    The light converging unit is used to reflect the light incident on the light control device to the dispersion element in a direction opposite to the incident direction of the light;

    The dispersion element diffuses the incident light and forms a light beam for forming a light spot.
11. The multi-level imaging system according to claim 10, wherein the light converging unit comprises: an inverted triangular pyramid microstructure, a supporting structure and a substrate layer;

    The inverted triangular pyramid microstructure and the supporting structure are arranged on the substrate layer, and the inverted triangular pyramid microstructure is located on the side of the supporting structure away from the substrate layer;

    The refractive index of the inverted triangular pyramid microstructure is greater than the refractive index of the support structure; the inverted triangular pyramid microstructure is configured to totally reflect light rays with an incident angle greater than a critical angle in the incident light rays. The direction opposite to the incident direction is reflected to the dispersive element.
12. 10. The multi-level imaging system according to claim 10, wherein the light converging unit comprises: a light converging layer, an isolation layer, a plane reflection layer, and a substrate sequentially arranged in the incident direction of the light;

    The plane reflection layer is located on the focal plane of the light converging layer;

    The light condensing layer and the plane reflective layer are made of different metamaterials;

    The light condensing layer is used to converge the incident light onto the plane reflective layer by changing the phase of the incident light, and is used to reflect the light back from the plane reflective layer along the light incident on the light converging layer The direction opposite to the direction is reflected to the dispersion element;

    The plane reflection layer is used for changing the phase of the light condensed by the light converging layer, and reflecting the light after the phase change to the light converging layer.
13. The multi-level imaging system according to any one of claims 9-12, wherein for the same external target, the electronic control unit is further configured to control the first sub head-up display to display the external target according to the driving information And controlling the second head-up display to display the second information of the external target, and the content included in the first information and the second information is at least partially different.
14. The multi-level imaging system according to any one of claims 9-12, wherein, for a plurality of external targets, the electronic control unit is further configured to control the first sub-head-up display to display the first external targets satisfying the first selection condition And controlling the second head-up display to display a second external target that meets a second selection condition, where the first condition and the second condition are different, and the first external target and the second external target are different.
15. The multi-level imaging system according to any one of claims 1-14, wherein the multi-level imaging system is a multi-level imaging system for a vehicle, and the driving information includes:

    Speed information of the vehicle; and

    Vehicle distance information between the vehicle and surrounding vehicles, wherein the surrounding vehicles include: a front vehicle located in front of the vehicle, a rear vehicle located behind the vehicle, and a left vehicle located on the left side of the vehicle , And at least one of the right side vehicles located on the right side of the vehicle;

    The electronic control unit is also configured to:

    Acquire the first distance between the vehicle and the front vehicle, the second distance between the vehicle and the rear vehicle, the third distance between the vehicle and the left vehicle, and the At least one of the fourth distance between the vehicle and the vehicle on the right;

    Determine the first safe distance threshold between the vehicle and the preceding vehicle according to the vehicle speed information of the vehicle;

    When the first distance is less than or equal to the determined first safe distance threshold, generating first warning information, and sending the first warning information to the head-up display for display;

    When the second distance is less than or equal to the second safety distance threshold, second warning information is generated, and the second warning information is sent to the head-up display for display; wherein, the second safety distance threshold is used to indicate all The safe driving distance between the vehicle and the rear vehicle;

    When the third distance and/or the fourth distance is less than or equal to the third safety distance threshold, third warning information is generated, and the third warning information is sent to the head-up display for display; wherein, the third The safe distance threshold is used to indicate the safe driving distance between the vehicle and the side vehicle.
16. The multi-level imaging system according to claim 15, wherein the electronic control unit is used for:

    When the first distance is less than or equal to the determined first safe distance threshold,

    Generate the first warning information;

    Acquiring a path image between the vehicle and the preceding vehicle;

    Identifying the path between the vehicle and the preceding vehicle according to the acquired path image between the vehicle and the preceding vehicle;

    Selecting a target image source from the multiple image sources of the head-up display;

    Use the first color identification to generate a first path identification instruction, and send the generated first path identification instruction to the target image source in the head-up display, and control the target image source to connect the vehicle with the preceding vehicle The paths between are displayed with the color corresponding to the first color mark, and the first warning information is sent to the target image source for display;

    When the first distance is greater than the first safe distance threshold, selecting a target image source from the multiple image sources of the head-up display according to the first distance;

    Use the second color identification to generate a second path identification instruction, and send the generated second path identification instruction to the target image source, and control the target image source to transfer the path between the vehicle and the preceding vehicle Use the color corresponding to the second color mark to display;

    When the first distance is less than the braking operation length threshold and when the first distance is less than or equal to the determined first safety distance threshold and the duration is greater than the first preset duration threshold, generating a braking instruction;

    Using the generated braking instruction, the vehicle is controlled to perform a braking operation.
17. The multi-level imaging system according to any one of claims 1-16, wherein the multi-level imaging system is a multi-level imaging system for vehicles, and the electronic control unit is further configured to:

    When the location of the vehicle is a crowded area, acquiring an image of the surrounding environment of the vehicle;

    When it is determined that there is a non-motorized vehicle around the vehicle according to the surrounding environment image, determine the distance between the non-motorized vehicle and the vehicle;

    When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, collision warning information is generated, and the collision warning information is sent to a head-up display for display.
18. The multi-level imaging system according to claim 17, wherein the electronic control unit is further configured to:

    When the distance between the non-motor vehicle and the vehicle is less than the collision safety distance threshold, generating collision warning information;

    According to the location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle, determine the first size of the non-motor vehicle indicator frame fused with the non-motor vehicle, and based on the surrounding environment image The outline of the non-motor vehicle in, determining the first shape of the non-motor vehicle indicator frame;

    Selecting a target image source from the multiple image sources of the head-up display;

    Use the third color identification, the first size and the first shape of the non-motor vehicle indicator frame to generate a non-motor vehicle identification instruction, and use the non-motor vehicle identification instruction to control the target image source to identify the color corresponding to the third color , Generating the non-motor vehicle indicator frame according to the determined first size and first shape, and fusing the non-motor vehicle indicator frame and the non-motor vehicle together according to the location of the non-motor vehicle to combine all The non-motor vehicle is identified, and the collision warning information is sent to the target image source for display.
19. The multi-level imaging system according to claim 17 or 18, wherein the electronic control unit is further configured to: when the distance between the non-motor vehicle and the vehicle is greater than a collision safety distance threshold,

    Determine the location of the non-motor vehicle;

    When the distance between the non-motor vehicle and the vehicle is less than or equal to the warning distance threshold, selecting the target image source from the head-up display according to the distance between the non-motor vehicle and the vehicle;

    The target image source is controlled to use the color corresponding to the fourth color mark to generate an early warning graphic at the location of the non-motor vehicle; wherein the early warning graphic is fused with the non-motor vehicle.
20. The multi-level imaging system according to claim 17 or 19, wherein the electronic control unit is further configured to:

    When the distance between the non-motor vehicle and the vehicle is greater than the warning distance threshold, acquiring a non-motor vehicle image, and determining the current location of the non-motor vehicle;

    According to the current location of the non-motor vehicle and the distance between the non-motor vehicle and the vehicle, determine the second size of the non-motor vehicle indicator frame fused with the non-motor vehicle, and based on the non-motor vehicle Determining the second shape of the non-motor vehicle indicator frame in the contour of the non-motor vehicle in the image of the motor vehicle;

    Selecting a target image source from the head-up display according to the distance between the non-motor vehicle and the vehicle;

    Use the fifth color identification, the second size and the second shape of the non-motor vehicle indicator frame to generate a non-motor vehicle identification instruction, and use the non-motor vehicle identification instruction to control the target image source to identify the color corresponding to the fifth color , Generating a non-motor vehicle indication frame according to the determined second size and second shape, and fusing the non-motor vehicle indication frame with the non-motor vehicle based on the current location of the non-motor vehicle.
21. The multi-level imaging system according to any one of claims 1-20, wherein the vehicle is a vehicle, the multi-level imaging system is used for a vehicle, and the driving information further includes: weather information and the road on which the vehicle is located Status information;

    The electronic control unit is also configured to:

    Obtaining weather information and/or status information of the road where the vehicle is located;

    When it is determined that there are potentially dangerous driving targets around the vehicle based on the weather information and/or the condition information of the road where the vehicle is located, warning information is generated, and the generated warning information is sent to a head-up display for display.
22. The multi-level imaging system according to claim 21, wherein the potentially dangerous driving target includes: severe weather conditions;

    The electronic control unit is configured as:

    When it is determined that there is a bad weather condition on the road where the vehicle is located, acquiring a bad weather road image of the road where the vehicle is located and generating warning information;

    Processing the bad weather road image to determine the brightness value of the road where the vehicle is located;

    When the brightness value is less than the brightness threshold, generate high beam turn-on prompt information, display the high-beam turn-on prompt information through the head-up display, and determine the path profile of the road where the vehicle is located from the bad weather road image And path location;

    Selecting a target image source from a plurality of image sources on the head-up display according to the path position;

    Use the sixth color to fill the path contour of the road where the vehicle is located to obtain a path contour map of the road where the vehicle is located;

    According to the path position of the road where the vehicle is located, the path contour map is projected onto the road where the vehicle is located through the target image source, so that the path contour map merges with the road where the vehicle is located, and the warning The information is sent to the head-up display for display.
23. The multi-level imaging system according to claim 21 or 22, wherein the potentially dangerous driving target includes: a dangerous situation in front of the road where the vehicle is located;

    The electronic control unit is configured as:

    When the condition information of the road where the vehicle is located is acquired and the distance between the vehicle and the road section where the dangerous condition occurs is less than the road condition display distance threshold, acquiring the road condition image of the road where the vehicle is located and generating warning information; Wherein, the road condition image includes: a road section where a road condition occurs in the road where the vehicle is located;

    Determine the road section contour and road section position of the road section where the dangerous condition occurs from the road condition image, and select a target image source from a plurality of image sources on the head-up display according to the road section position;

    Use the seventh color to fill in the road section contour of the road section where the dangerous condition occurs, to obtain the state contour map of the road section where the road condition occurs;

    Projecting the condition contour map onto the road section where the road condition occurs through the target image source, so that the condition contour diagram is merged with the road section where the road condition occurs.
24. The multi-level imaging system according to any one of claims 1-23, wherein the multi-level imaging system is a multi-level imaging system for a vehicle, and the driving information includes: operation information, driving speed, Acceleration, vehicle status information, navigation information;

    The electronic control unit is also configured to:

    When generating operation advice information of the vehicle according to the navigation information, the driving speed, acceleration, and vehicle status information of the vehicle, collect the vehicle operation video of the observer, and display the operation through the head-up display Suggested information display;

    When it is determined that the operation corresponding to the operation advice information is not detected within the specified operation time, generating dangerous driving information, displaying the dangerous driving information through the head-up display, and performing a decrement operation on the driving vehicle index;

    When it is determined that the operation corresponding to the operation suggestion information is detected within the specified operation time period, an incremental operation is performed on the driving vehicle index;

    When it is determined that the length of time the vehicle has not been operated is greater than the parking time threshold, stop collecting vehicle operation videos, and send the driving vehicle index and the collected vehicle operation videos to the head-up display for display.
25. The multi-level imaging system according to claims 1-24, wherein the electronic control unit is configured to:

    The display data is formed according to the driving information, the display data is processed according to the driver's personal information, and the light emitted by the head-up display is controlled to irradiate the imaging window according to the processed display data.
26. The multi-level imaging system according to claim 25, wherein the processing includes at least one of the following: color saturation enhancement processing, color conversion processing of the image, and shape conversion processing of the image.
27. The multi-level imaging system according to claim 25 or 26, wherein:

    The electronic control unit is further configured to identify the driver's identity information, and determine the driver's personal information based on the identity information.
28. The multi-level imaging system according to any one of claims 1-27, wherein at least one of the at least two image sources included in the augmented reality head-up display is located at a position close to the focal plane or focus of the curved mirror. The location of the plane.
29. The multi-level imaging system according to any one of claims 1-28, wherein the driving information includes driving information and environmental information.
30. The multi-level imaging system according to any one of claims 1-29, further comprising an imaging window, wherein the imaging window is a windshield of a vehicle, and the eye box area is an eye box area of the multi-level imaging system ；

    The windshield is used for presenting a multi-layer image formed by the multi-path light at a different distance from the eye box area on the side of the windshield away from the head-up display to form the multi-layer image The light of any layer of the image can enter the eye box area.
31. The multi-level imaging system according to any one of claims 1-30, further comprising a data acquisition device,

    Wherein, the data collection device is used to obtain environmental information around the vehicle, and send the obtained environmental information to the electronic control unit.
32. The multi-level imaging system according to any one of claims 1-31, wherein the multi-level imaging system is an imaging system for vehicles.
33. A head-up display, including a curved mirror, a first image source and a second image source,

    Wherein, the first image source is used to emit a first path of light incident to the curved mirror, and the second image source is used to emit a second path of light incident to the curved mirror;

    The first path of light has a first propagation path between the first image source and the curved mirror, and the second path of light has a second propagation path between the second image source and the curved mirror path;

    The lengths of the first propagation path and the second propagation path are different, so that the first image displayed by the first path of light by the head-up display and the second image displayed by the second path of light are different The image distance.
34. A head-up display, comprising a curved mirror and at least one image source, wherein,

    The curved mirror is configured to reflect light emitted by the at least one image source, and the at least one image source is close to or located at the focal plane of the curved mirror.
35. A vehicle comprising the multi-level imaging system according to any one of claims 1-32 or the head-up display according to claim 33 or 34.
36. The vehicle according to claim 35, further comprising an imaging window, wherein:

    The head-up display further includes a second sub-head-up display, and the second sub-head-up display includes a projection device and a light control device;

    The projection equipment is used to emit light that can be incident on the light control device;

    The light control device is used to change the exit direction of the light emitted by the projection device, so that the horizontal field angle of the second head-up display is not less than 15 degrees and the vertical field angle is not less than 5 degrees;

    The projection device is installed in the vehicle;

    The light control device is arranged on the surface of the center console of the vehicle opposite to the imaging window.
37. The vehicle according to claim 35 or 36, wherein:

    The electronic control unit is configured as:

    According to the driving information, the second head-up display is controlled to emit light to irradiate the imaging window, so that the image presented by the light from the second head-up display has an imaging area larger than the imaging window at the imaging window. The area of the imaging area of each of the remaining images in the plurality of images at the imaging window.
38. An imaging method of a multi-level imaging system, wherein the multi-level imaging system includes an eye box area and a head-up display, and the imaging method includes:

    Acquiring driving information and sending it to the head-up display;

    According to the driving information, the head-up display is controlled to emit multiple light beams to respectively display multi-layer images to display the driving information, and at least part of the multi-layer images are at different distances from the eye box area.

Images