WO2022266829A1

WO2022266829A1 - Display method and apparatus, device, and vehicle

Info

Publication number: WO2022266829A1
Application number: PCT/CN2021/101446
Authority: WO
Inventors: 朱伟
Original assignee: 华为技术有限公司
Priority date: 2021-06-22
Filing date: 2021-06-22
Publication date: 2022-12-29
Also published as: CN113597617A

Abstract

The present application relates to the technical field of assisted driving, and provides a display method and apparatus, a device, and a vehicle. The method comprises: performing infrared light supplementation on a target region outside a vehicle; obtaining information of an infrared image of the target region in an infrared imaging mode; determining the position of a target object in the infrared image according to the obtained infrared image; and displaying annotation information of the target object in a display region according to the position of the target object in the infrared image. According to the present application, the target object outside the vehicle can be detected in an infrared imaging mode in a driving process, and the annotation information of the target object is displayed, so that a user can still determine the target object outside the vehicle and the position of the target object according to the displayed annotation information when the illumination condition outside the vehicle is poor, and the safety of driving is improved.

Description

A display method and device, equipment and vehicle

technical field

The present application relates to the technical field of driving assistance, in particular to a display method and device, equipment and a vehicle.

Background technique

When driving at night, if you drive in an environment with poor light, you may not be able to see pedestrians, vehicles or other obstacles on the road clearly, which will easily increase the possibility of traffic accidents. In addition, when meeting with an oncoming vehicle at night, because the lights of the oncoming vehicle will be very dazzling, it will often cause instant blindness to the driver, resulting in the inability to see the target objects such as the oncoming vehicle or pedestrians on the road clearly. , will also increase the possibility of traffic accidents.

With the rapid development of head-up display (Head Up Display, HUD) or augmented reality head-up display (Augmented Reality-Head Up Display, AR-HUD) in recent years, existing vehicle manufacturers provide a The solution is to mark and remind pedestrians or obstacles that may exist in front of the glass. However, this solution is still limited by the light conditions of night driving, and cannot solve the problem of not being able to see the road clearly when the light is poor or when vehicles on both sides meet. The problem of pedestrians or obstacles on the road, so its labeling and reminder effects are also easily affected.

Contents of the invention

In view of this, the present application provides a display method, device, equipment, and vehicle, which can detect target objects outside the vehicle through infrared imaging during driving, and display the label information of the target object, so as to improve driving performance. security.

It should be understood that in the solutions provided in this application, the display method may be executed by a display device or some components in the display device, wherein the display device may be an AR-HUD, a HUD or other devices with a display function. Part of the components in the display device may be processing chips, processing circuits, processors, and the like.

The first aspect of the present application provides a display method, including: performing infrared supplementary light on a target area outside the vehicle. Information of an infrared image of the target area is acquired, wherein the target area includes a target object. The location of the target object in the infrared image is determined. According to the position of the target object in the infrared image, the label information of the target object is displayed in the display area.

From the above, this method acquires infrared image information of the target area outside the vehicle by means of infrared supplementary light on the target area outside the vehicle to identify the target object appearing in front of the vehicle, and through the acquired infrared image The position of the target object determines the display position of the label information, so that the generated label information can be displayed at the display position. Through this method, the target object in front of the vehicle can be detected in time during the driving process, especially in the case of poor lighting at night, and the generated annotation information can be displayed on the corresponding position, so that the user can determine Target objects and their positions outside the vehicle improve driving safety.

In a possible implementation manner of the first aspect, determining the position of the target object in the infrared image includes: providing the information of the infrared image to an image recognition model, and the image recognition model determines that the target object is in the infrared image. position in the image.

From the above, according to the acquired infrared image of the target area, the trained image recognition model can be used to identify the target object in the infrared image, and determine the position of the target object in the infrared image. The image recognition model can be trained by neural network or deep learning. According to the recognition requirements of different types of target objects, this method can use different image recognition models to identify the target objects in the infrared image, so as to improve the accuracy of the target object. Object recognition success rate.

In a possible implementation manner of the first aspect, the display position of the annotation information is related to the spatial position of the target object and the position of the user's eyes.

From the above, in order to realize the user's viewing experience, the display position of the annotation information can be determined according to the user's human eye position and the spatial position of the target object, and when the annotation information is displayed according to the determined display position, the annotation that the user sees can be made The information can be fused with the position of the target object, so that the user can still determine the target object and its position outside the vehicle according to the displayed annotation information when the lighting conditions outside the vehicle are not good.

In a possible implementation manner of the first aspect, the display size of the annotation information is related to the display position of the annotation information, the position of the user's eyes, and the size of the target object.

From the above, the annotation information can be in various forms, and the display size of the annotation information can be determined according to the position of the user's eyes, the display position of the annotation information, and the spatial position of the target object. For example, the annotation information can be specifically embodied in the form of a prompt box, and the display size of the prompt box can be calculated by obtaining the user's eye position, the spatial position of the target object, and the display position of the prompt box, so that the user The tooltips seen can be matched with pedestrians. As the vehicle travels, the distance between the target object and the vehicle becomes closer and closer, and the size of the prompt box increases accordingly, so that the user can intuitively perceive the position of the target object.

In a possible implementation of the first aspect, the display area is the display area of an augmented reality head-up display, and the display position of the annotation information is related to the spatial position of the target object and the position of the user's eyes, including: the annotation information The display position of is determined by the user's first line of sight. The first line of sight is the line of sight from the user's eye position to the spatial position of the target object.

In a possible implementation of the first aspect, the display area of the augmented reality head-up display is located on the front windshield of the vehicle, and the display position of the annotation information is determined through the user's first line of sight, including: the annotation The display position of the information is determined by the intersection of the user's first line of sight and the front windshield of the vehicle.

From the above, the display area in this method may specifically be the display area of the augmented reality head-up display, and the augmented reality head-up display may project and display the marked information according to the display position of the marked information. Specifically, the augmented reality head-up display can use the front windshield of the vehicle as a display area, and determine the user's first line of sight according to the position of the user's human eyes and the spatial position of the target object. The intersection point can be determined as the display position of the annotation information, so that the annotation information seen by the user is on the same line of sight as the target object, and the display effect of the annotation information is improved.

In a possible implementation manner of the first aspect, determining the intersection of the first line of sight and the front windshield of the vehicle includes: determining a connection line between the target object and the image acquisition device based on the spatial position of the target object The horizontal direction is parallel to the front windshield of the vehicle at a first included angle with respect to the horizontal direction. Based on the distance between the user's eye position and the image acquisition device, and the first angle, a second angle between the user's eye and the target object relative to the horizontal direction is determined. Based on the distance between the user's eyes and the front windshield of the vehicle, and the second included angle, the display position of the annotation information on the front windshield is determined.

From the above, based on the position of the target object in the infrared image and the horizontal field of view angle of the image acquisition device, the first angle between the line between the target object and the image acquisition device relative to the horizontal direction can be calculated, based on the user's eyes The lateral distance from the image acquisition device and the first angle can be used to calculate the second angle between the user's eyes and the target object relative to the horizontal direction. Based on the second angle and the distance between the user's eyes and the vehicle The distance of the front windshield of the vehicle can be calculated to obtain the display position of the label information on the front windshield. In this method, the calculation amount of the display position is small, and the calculation method is based on simple trigonometric functions, the calculation amount is small, and the accurate display position can be quickly obtained.

In a possible implementation manner of the first aspect, the spatial position of the target object is related to the position of the target object in the infrared image, internal parameters and external parameters of an image acquisition device that acquires the infrared image.

From the above, the spatial position of the target object may specifically be the spatial position of the target object in the vehicle coordinate system. This method can calculate the distance between the target object and the image acquisition device based on the position of the target object in the infrared image, the installation height of the image acquisition device, and the angle between the visible ground and other parameters. Based on the distance and the image acquisition device in the vehicle The spatial position in the vehicle coordinate system can determine the spatial position of the target object in the vehicle coordinate system.

In a possible implementation manner of the first aspect, the target object includes one or more objects among other vehicles, pedestrians, and animals.

From the above, the target object in this method can be moving objects such as other vehicles, pedestrians, animals, etc., and can also be static objects such as road signs and trees. Specifically, the type of target object can be selected according to the user's needs to realize the user's Customized requirements.

In a possible implementation manner of the first aspect, before determining the position of the target object in the infrared image, it further includes: performing one or more of cropping, noise reduction, enhancement, smoothing, and sharpening on the infrared image multiple processing.

From the above, due to the limited range of infrared supplementary light, it may not be able to completely cover the image acquisition range covered by the image acquisition device. Therefore, before the acquired infrared image is recognized, the infrared image can be cropped, denoised, and enhanced. , smoothing, or sharpening, etc., so as to facilitate effective and rapid identification of target objects in infrared images.

A second aspect of the present application provides a display device, including: a supplementary light module, configured to perform infrared supplementary light on a target area outside the vehicle. The obtaining module is used to obtain information of the infrared image of the target area, wherein the target area includes the target object. A processing module, configured to determine the position of the target object in the infrared image. The sending module is configured to display the label information of the target object in the display area according to the position of the target object in the infrared image.

In a possible implementation manner of the second aspect, the display position of the annotation information is related to the spatial position of the target object, the position of the user's eyes, and the size of the target object.

In a possible implementation manner of the second aspect, the display size of the annotation information is related to the display position of the annotation information, the position of the user's eyes, and the size of the target object.

In a possible implementation of the second aspect, the display area is the display area of an augmented reality head-up display, and the display position of the annotation information is related to the spatial position of the target object and the position of the user's eyes, including: the annotation information The display position of is determined by the user's first line of sight. The first line of sight is the line of sight from the user's eye position to the spatial position of the target object.

In a possible implementation manner of the second aspect, the display area of the augmented reality head-up display is located on the front windshield of the vehicle, and the display position of the annotation information is determined by the first line of sight of the user's first line of sight It includes: the display position of the marked information is determined by the intersection point of the user's first line of sight and the front windshield of the vehicle.

In a possible implementation manner of the second aspect, the spatial position of the target object is related to the position of the target object in the infrared image, internal parameters and external parameters of an image acquisition device that acquires the infrared image.

In a possible implementation manner of the second aspect, the target object includes one or more objects among other vehicles, pedestrians, and animals.

In a possible implementation manner of the second aspect, before the processing module is used to determine the position of the target object in the infrared image, it is further configured to: perform cropping, noise reduction, enhancement, smoothing, and one or more of sharpening.

A third aspect of the present application provides a computing device, including: a processor, and a memory, on which program instructions are stored, and when the program instructions are executed by the processor, the processor performs the operations described in the first aspect and the above-mentioned various methods. Display methods in various technical solutions provided by optional implementation manners.

In a possible implementation manner, the computing device is one of AR-HUD and HUD.

In a possible implementation, the computing device is a vehicle.

In a possible implementation manner, the computing device is one of an on-board computer and an on-board computer.

A fourth aspect of the present application provides an electronic device, including: a processor, and an interface circuit, wherein the processor accesses a memory through the interface circuit, and the memory stores program instructions, and when the program instructions are executed by the processor At this time, the processor is made to execute the display methods in the various technical solutions provided in the first aspect and the above-mentioned various optional implementation manners.

In a possible implementation manner, the electronic device is one of AR-HUD and HUD.

In a possible implementation manner, the electronic device is a vehicle.

In a possible implementation manner, the electronic device is one of a car machine and a car computer.

A fifth aspect of the present application provides a display system, including: a vehicle-machine device, and a computing device coupled with the vehicle-machine device as provided in the third aspect and the various optional implementation manners above, Or the electronic device in the various technical solutions provided in the fourth aspect and the above-mentioned various optional implementation manners coupled with the vehicle-machine device.

In a possible implementation manner, the display system is a vehicle.

The sixth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored. When the program instructions are executed by a computer, the computer executes multiple A display method in the technical solution.

The seventh aspect of the present application provides a computer program product, which includes program instructions. When the program instructions are executed by a computer, the computer executes various technical solutions as provided in the first aspect and various optional implementation manners above. display method in .

To sum up, the display method, device, equipment, and vehicle provided by the present application detect and collect real-time infrared images of the target area outside the vehicle through infrared imaging to identify the target object in the infrared image and determine the target object in the infrared image. The position in the image is used to obtain the spatial position of the target object, and the display position of the annotation information is determined according to the spatial position of the target object and the position of the user's eyes, so that the generated annotation information is displayed at the display position. In this way, the annotation information seen by human eyes can be fused with the position of the target object, so as to remind the user to pay attention to the target object outside the vehicle. At the same time, the application can also determine the size of the annotation information according to the user's eye position, the display position of the annotation information, and the size of the target object, so that the size of the annotation information can gradually become larger as the target object approaches, so that To achieve a better display effect. Through the present application, it is possible to timely detect target objects around the vehicle during driving, especially at night when the illumination is poor, and display marked information, thereby improving driving safety.

Description of drawings

FIG. 1 is a structural diagram of an application scenario of a display method provided by an embodiment of the present application;

FIG. 2 is a structural diagram of a vehicle provided in an embodiment of the present application;

FIG. 3A is a schematic side view of a vehicle cockpit provided by an embodiment of the present application;

Fig. 3B is a front schematic diagram of a vehicle cockpit provided by an embodiment of the present application;

FIG. 4 is a flow chart of a display method provided by an embodiment of the present application;

FIG. 5 is a flow chart for determining the display position of annotation information provided by the embodiment of the present application;

FIG. 6 is a position distribution diagram of the front view angles of vehicles and pedestrians according to the embodiment of the present application;

FIG. 7 is a position distribution diagram of side view angles of vehicles and pedestrians according to the embodiment of the present application;

Fig. 8 is a position distribution diagram of the overlooking angles of vehicles and pedestrians according to the embodiment of the present application;

FIG. 9 is a structural diagram of a display device provided by an embodiment of the present application;

FIG. 10 is an architecture diagram of a computing device provided by an embodiment of the present application;

FIG. 11 is a structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 12 is a structural diagram of a display system provided by an embodiment of the present application.

It should be understood that in the above structural schematic diagrams, the dimensions and shapes of each block diagram are for reference only, and should not constitute an exclusive interpretation of the embodiments of the present invention. The relative positions and containment relationships among the block diagrams presented in the structural diagram are only schematic representations of the structural relationships among the block diagrams, rather than limiting the physical connection manners of the embodiments of the present invention.

detailed description

The technical solutions provided by the present application will be further described below in conjunction with the accompanying drawings and examples. It should be understood that the system structure and business scenarios provided in the embodiments of the present application are mainly for illustrating possible implementations of the technical solution of the present application, and should not be interpreted as the only limitation on the technical solution of the present application. Those skilled in the art know that, with the evolution of the system structure and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

It should be understood that the display solutions provided in the embodiments of the present application include display methods and devices, equipment, and vehicles. Because the principles of these technical solutions to solve problems are the same or similar, in the introduction of the following specific embodiments, some repetitions may not be repeated, but it should be considered that these specific embodiments have mutual references and can be combined with each other.

When driving at night, if you drive in a poorly lit environment, you may not be able to see clearly the target objects around the vehicle, such as pedestrians, other vehicles, small animals or other obstacles, which will easily increase the possibility of traffic accidents. In view of this, although the low beam or high beam of the vehicle can be turned on to illuminate the front of the vehicle, the lighting range of the low beam is limited. Although the high beam can properly improve the lighting range in front of the vehicle, the high beam is easy It will interfere with pedestrians or other vehicles on the road. At the same time, the user of the own car is also easily disturbed by the high beams of the oncoming vehicle. It is easy to have blind spots when meeting cars, and there is a risk of accidents.

To this end, the embodiment of the present application provides a display method, device, device, and vehicle, which can detect target objects around the vehicle through infrared imaging during driving, and display label information at the corresponding position of the target object In this way, the annotation information seen by the user is fused with the position of the target object, so as to realize the purpose of reminding the user and improve the safety of driving at night. Among them, the user is usually a driver. The user can also be a copilot passenger or a rear passenger, etc. The application is described in detail below.

Firstly, the application scenarios involved in this embodiment are briefly described. FIG. 1 is an architecture diagram of an application scenario of a display method provided by an embodiment of the present application. As shown in FIG. As shown in FIG. 2 , the application scenario of this embodiment specifically involves a vehicle 100 , which may be a family car or a truck, or a special vehicle such as an ambulance, fire engine, police car or engineering emergency vehicle. The above-mentioned supplementary light device 110, collection device 120, processing device 130, and sending device 140 may be installed on the vehicle, and may be installed outside or inside the vehicle. The specific architecture of the vehicle 100 involved in this application scenario will be described in detail below with reference to FIGS. 3A-3B .

As shown in FIG. 3A , the supplementary light device 110 can be an infrared supplementary light, an infrared emitter, or other equipment or a combination of devices with an infrared emission function, and can be arranged at the front of the vehicle 100, such as a large panel at the front of the vehicle. lights for easy wiring. It can also be arranged on the top of the vehicle or the side of the rearview mirror of the vehicle cockpit facing the outside of the vehicle. It is mainly used to provide infrared supplementary light to the target area around the vehicle when driving the vehicle at night with poor lighting, and the range of supplementary light can cover the maximum field of view angle of the acquisition device 120 . The target area may be the front, side or rear of the vehicle, and the infrared supplementary light is applied to the target area, so that the collection device 120 can obtain a clearer infrared image when detecting and collecting. The target area may contain target objects to be detected and collected, and the target objects may be other vehicles, pedestrians, animals, or other obstacles. When the present embodiment selects the infrared supplementary light, a high-power infrared supplementary light (for example, 30 watts) can be selected for use. Since infrared light is invisible light, a high-power infrared supplementary light will not be harmful to pedestrians or pedestrians on the road. other vehicles are affected. The infrared supplementary light is an example of this embodiment. In addition, other devices or devices capable of emitting infrared rays can also be selected. This embodiment does not specifically limit the type, position and quantity of the supplementary light device 110 .

The acquisition device 120 may include an out-of-vehicle acquisition device and an in-vehicle acquisition device. As shown in Figure 3A, the acquisition device outside the vehicle can specifically use an infrared camera, on-board radar, or other equipment or multiple combined equipment with infrared image acquisition or infrared scanning functions, and can be arranged on the top, head or vehicle cockpit of the vehicle 100 The side of the rearview mirror facing the outside of the vehicle can be installed inside or outside the vehicle. It is mainly used to detect and collect infrared image information of the target area of infrared supplementary light outside the vehicle. The target area may contain target objects to be detected and collected, and the target objects may be other vehicles, pedestrians, animals or other obstacles. The infrared image information may be a single infrared image, or one or more frames of infrared images in the collected video stream. As shown in Figure 3B, the in-vehicle acquisition device can specifically use equipment such as a vehicle-mounted camera, a human eye detector, etc. During the specific implementation process, the in-vehicle acquisition device can be set according to the location according to requirements, for example, it can be installed on the A-pillar of the vehicle cockpit , the B-pillar or the side of the rearview mirror of the vehicle cockpit facing the user, it can also be set on the steering wheel, the area near the center console, and can also be set on the position above the display screen behind the seat. It is mainly used to detect and collect the human eye position information of the user in the vehicle cockpit. There may be one collection device in the vehicle, or there may be multiple collection devices, and the application does not limit its location and quantity.

The processing device 130 can be an electronic device, specifically a processor of a vehicle-mounted processing device such as a car machine or a vehicle-mounted computer, or a conventional processor such as a central processing unit (Central Processing Unit, CPU) or a microprocessor (Micro Control Unit, MCU). The chip processor can also be used as terminal hardware such as mobile phones and tablets. The processing device 130 can be preset with an image recognition model or by acquiring the preset image recognition model in other devices in the vehicle, it can identify the target object in the infrared image according to the received infrared image information, and determine the target object in the infrared image. position in the target object, and generate annotation information corresponding to the target object. The annotation information can be a prompt box, a highlighted sign, an AR image, etc., and can also be a text or a guideline. And it can also determine the spatial position of the target object according to the position of the target object in the infrared image, determine the user's human eye position according to the human eye position information obtained by the acquisition device in the vehicle, and determine the user's human eye position according to the spatial position of the target object and the user's human eye position. eye position, determine the display position of the label information of the target object, and output the determined label information and the display position of the label information to the sending device 140 .

The sending device 140 can be HUD, AR-HUD or other devices with display functions, and can be installed above or inside the center console of the vehicle cockpit, and is mainly used to display the marked information in the display area. The display area of the sending device 140 can be the front windshield of the vehicle, or it can be an independently displayed transparent screen, which is used to reflect the light of the marked information sent by the sending device 140 and then enter the user's eyes, so that the user can see through the front windshield. When the windshield or transparent screen looks out of the car, you can see the fusion of the marked information and the position of the target object outside the vehicle, so as to remind the user of the type or position of the target object that appears outside the vehicle, improve the display effect of the marked information, and improve the driving experience. security.

Among them, the supplementary light device 110, the acquisition device 120, the processing device 130, and the sending device 140 can communicate data or instructions through wired communication (such as interface circuit) or wireless communication (such as Bluetooth, wifi), etc., for example, supplementary light The device 110 may receive a control command from the processing device 130 through Bluetooth communication, and turn on the supplementary light for the target area outside the vehicle. After collecting the infrared image information of the target area, the collecting device 120 may transmit the infrared image information to the processing device 130 through Bluetooth communication. After the acquisition device 120 collects the user's eye position information, it may transmit the user's eye position information to the processing device 130 through Bluetooth communication. The processing device 130 determines the target object and the spatial position of the target object in the infrared image according to the infrared image information, and generates labeling information, and calculates the display position of the labeling information according to the spatial position of the target object and the user's eye position information, and obtains The annotation information and the display position of the annotation information are output to the sending device 140, and the sending device 140 displays the annotation information at the display position of the display area.

Through the above-mentioned structure, the vehicle 100 involved in this embodiment can collect infrared image information on the target area outside the vehicle through infrared imaging to determine the target object in the target area, and can also collect the user's personal information through the in-vehicle acquisition device. Eye position information, and based on the infrared image information and the user's eye position information, determine the label information and its display position, use the sending device to display the label information at the display position, so that the label information seen by the user can be compared with the vehicle In order to achieve the purpose of reminding the user and improve the safety of driving at night.

Fig. 4 shows a flow chart of a display method provided by an embodiment of the present application. The display method can be executed by a display device or some devices in the display device, such as AR-HUD, HUD, car, processor, etc., where The processor may be a processor of a display device, or a processor of a vehicle-mounted processing device such as a vehicle machine or a vehicle-mounted computer. Specifically, the infrared image information of the target area outside the vehicle can be collected by means of infrared imaging, and the label information can be displayed on the corresponding position, so that the user can still use the displayed label information when the lighting conditions outside the vehicle are not good. Improve night driving safety by identifying target objects and their locations outside the vehicle. As shown in Figure 4, the display method includes:

S410: performing infrared supplementary light on the target area outside the vehicle;

In this embodiment, considering that during driving at night or on roads with poor lighting conditions, the infrared imaging effect caused by lack of lighting is not good, it is possible to install a supplementary light device on the vehicle, for example, An infrared supplementary light is arranged on the side of the vehicle top, head or vehicle cockpit rearview mirror facing the outside of the vehicle, wherein the processor can send an instruction to turn on the supplementary light to the infrared supplementary light through the interface circuit to turn on the infrared supplementary light. The infrared supplementary light is used to provide infrared supplementary light to the target area outside the vehicle, and to detect the target area outside the vehicle in real time through the acquisition device, and to obtain infrared image information in real time. For example, the acquisition device may be an infrared camera, and the infrared image information may include information such as resolution, size, dimension, and color.

S420: Acquire information about the infrared image of the target area;

The processor can send an image acquisition instruction to the infrared camera through the interface circuit, so as to control the infrared camera to acquire the infrared image of the target area outside the vehicle. By identifying the target object in the acquired infrared image, the target object can be a target object that can trigger an early warning while the vehicle is driving, specifically pedestrians, vehicles, animals, or other obstacles. The processor can quickly recognize the features of the target object in the infrared image based on the recognition model, thereby determining the target object in the infrared image and the position of the target object in the infrared image. The recognition model can be realized through a neural network model or a deep learning model. Specifically, different recognition models can be used to recognize infrared images based on different forms of target objects. For example, when the target object is a pedestrian, a portrait recognition model can be used to identify infrared images. The image is recognized to determine the pedestrian and the position of the pedestrian in the infrared image. Since the height of the pedestrian is different, in this embodiment, for the convenience of subsequent calculation, the position of the foot of the pedestrian can be used as the position to be determined.

S430: Determine the position of the target object in the infrared image;

In this embodiment, the processor can determine the target object in the infrared image and the position of the target object in the infrared image according to the information of the acquired infrared image. For example, when the target object is a pedestrian, because the height of the pedestrian is different In this embodiment, for the convenience of subsequent calculation, the position of the pedestrian's foot in the infrared image may be used as the position of the target object to be determined in the infrared image. The processor can also generate label information of the target object according to the type of the target object in the infrared image. Wherein, the annotation information can be information with a reminder effect generated based on the target object in the infrared image, for example, it can be a prompt box, a highlighted sign or an arrow mark, etc., it can also be a prompt text or a guide line, etc., or it can be AR images with AR effects, etc.

The processor can determine the spatial position of the target object based on the position of the target object in the acquired infrared image, internal parameters, external parameters, etc. of the infrared camera, and then based on the spatial position of the target object and the spatial position of the user's eyes, The display position of the label information in the display area can be determined. Wherein, the spatial position of the target object and the spatial position of the user's eyes may specifically be their respective spatial positions in the vehicle coordinate system.

In the flow chart of determining the display position of the annotation information shown in FIG. 5 , the display area may be the display area of the AR-HUD, and the display area of the AR-HUD may be located on the front windshield of the vehicle. Then in this embodiment, determining the display position of the annotation information in the display area can be specifically implemented in the following manner:

S431: Determine the spatial position of the target object based on the position of the target object in the infrared image;

As shown in Figure 6, it is a position distribution diagram of the front view angles of vehicles and pedestrians in this embodiment, it can be considered that Figure 6 is equivalent to a kind of equivalent schematic diagram of the infrared image acquired under the infrared camera field of view, the target object in this figure is For the pedestrian in front of the vehicle, the position of the pedestrian's foot is used as the position of the pedestrian in the infrared image, and the horizontal direction is the farthest road surface captured by the infrared camera. The pixel distance from the pedestrian to the horizontal direction in the infrared image is a, and the pixel distance from the front of the vehicle is The distance is b, and the distances from the pixels on both sides of the infrared image are c and d, respectively. Since the parameters of the infrared camera can be determined at the time of installation, the horizontal direction, the front of the vehicle, and the positions on both sides in the infrared image collected by the infrared camera are all fixed parameters. Therefore, based on the two-dimensional position of the pedestrian in the infrared image, The distance of the pedestrian relative to the infrared camera can be calculated.

As shown in Figure 7, it is the position distribution diagram of the side view angles of vehicles and pedestrians in this embodiment, in this Figure 7, according to the position where the infrared camera is installed, it can be determined that the height of the infrared camera installation from the ground is H, and it can also be Determine the angle between the nearest road surface visible to the infrared camera and the farthest road surface as θ. The unit of the angle θ can be radians. Since the installation position of the infrared camera can be fixed at a certain place on the vehicle, the height H and the angle θ can be determined as a known parameter. Then based on the trigonometric function, the calculation method of the horizontal distance L between the pedestrian and the infrared camera can be:

After simplification, we get:

Based on the calculated horizontal distance L between the pedestrian and the infrared camera, and the spatial position of the infrared camera in the vehicle coordinate system, the spatial position of the pedestrian in the vehicle coordinate system, that is, the spatial position of the target object, can be determined.

S432: Obtain the user's eye position information;

The user's eyes are detected by the above-mentioned in-vehicle acquisition device, which may be a camera or an eyeball detector. According to the obtained position information of the user's eyes, and the conversion relationship between the installation position of the in-vehicle acquisition device and the vehicle coordinate system, the spatial position of the user's eyes in the vehicle coordinate system can be obtained.

S433: Based on the spatial position of the user's eyes and the spatial position of the target object, determine the display position of the annotation information;

In this embodiment, when displaying the marked information, specifically, the marked information may be sent to the front windshield of the vehicle for display, so that the user can observe it head-up. Among them, the labeling information can be displayed at a fixed position on the front windshield, and the user can determine whether there are pedestrians, other vehicles, etc. Target objects such as vehicles and animals. The label information can also be displayed on different positions of the front windshield. Specifically, the connection line between the user's eyes and the pedestrian can be determined based on the spatial position of the user's eyes and the spatial position of the pedestrian. The intersection point between the glasses can be determined as the display position of the label information on the front windshield. Wherein, the spatial position of the intersection point can be obtained by coordinate calculation. Specifically, the vehicle coordinate system can be used as the reference coordinate system, and then based on the spatial coordinates of the user's eyes, the spatial coordinates of pedestrians, and the spatial coordinates of the vehicle's front windshield, determine The spatial coordinates of the intersection point on the front windshield.

As shown in Figure 8, it is a position distribution diagram of the overlooking angles of vehicles and pedestrians in this embodiment. In this Figure 8, based on the spatial position of the user's eyes and the spatial position of the infrared camera, the horizontal distance between the user's eyes and the infrared camera can be obtained. The distance is e. The horizontal field of view of the infrared camera is λ, and the angle between the line between the pedestrian and the infrared camera relative to the front windshield is α, and the angle between the line between the pedestrian and the user's eyes is relative to the front windshield is β, wherein, the units of angle α and angle β can also be radians. Then the calculation method of the included angle α is:

After simplification, we get:

Based on the angle α, the calculation method of the angle β is:

Based on the calculated angle β and the distance between the user's eyes and the front windshield, the spatial position of the intersection between the line between the user's eyes and the pedestrian and the front windshield of the vehicle can be calculated, that is, the calculation The display position of the marked information on the front windshield.

The calculated display position is specifically the spatial position in the vehicle coordinate system. Before projection, the spatial position coordinates in the vehicle coordinate system can be further converted into projected coordinates in the AR-HUD coordinate system, and then the label information and The projected coordinates are sent to the AR-HUD.

S440: Display the annotation information of the target object in the display area.

In this embodiment, the processor can send the generated annotation information and its display position to the AR-HUD through the interface circuit, and the AR-HUD projects the annotation information to the calculated display position on the front windshield for display. The annotation information seen by the user is kept in the same line of sight as the pedestrian, so that the user can quickly perceive the target object and its position during driving.

In some embodiments, the display size of the annotation information can be a fixed size, for example, it can be a fixed-size prompt box, an AR image, etc. generated according to the target object, or it can be a fixed-size text and guide line generated according to the target object. Wait. By sending the fixed-size annotation information to the display position for display, the user can be reminded of the existence of the target object outside the vehicle. In some other embodiments, the display size of the annotation information may also be related to the display position of the annotation information, the spatial position of the user's eyes, and the size of the target object. According to the spatial position of the user's eyes, the display position of the annotation information, and the spatial position of the target object, a model similar to a frustum can be formed. Therefore, according to the size of the target object, the display size of the annotation information at the display position can be determined, so that The annotation information seen by the user can match the target object, and the display size of the annotation information can change accordingly with the relative distance between the vehicle and the target object. For example, the annotation information can be specifically embodied in the form of a prompt box, and the display size of the prompt box can be calculated by obtaining the spatial position of the user's eyes, the spatial position of the pedestrian, and the display position of the prompt box, so that The tooltip the user sees can be matched to a pedestrian. Based on this, as the vehicle travels and the distance between the pedestrian and the vehicle becomes closer, the size of the prompt box can also increase accordingly, so as to remind the user to pay attention to the existence of the pedestrian and the change of the distance to the pedestrian.

To sum up, the display method provided by the embodiment of the present application uses infrared imaging to detect the environment outside the vehicle, and determines the display position of the marked information based on the spatial position of the target object and the spatial position of the user's eyes. In this embodiment, infrared supplementary light and infrared imaging are used to improve the imaging capability under poor lighting conditions. The effect of infrared imaging will not be affected by visible light, and infrared supplementary light will not affect oncoming vehicles or pedestrians. influence and avoid traffic hazards. In addition, in this embodiment, based on the spatial position of the user's eyes and the spatial position of the target object, the display position of the annotation information is calculated, which involves a small amount of calculation and reduces the occupation of vehicle processing resources. According to the calculated display position, the label information is displayed, so that when the lighting conditions outside the vehicle are not good, the user can still determine the target object and its position outside the vehicle according to the displayed label information, and improve driving safety.

FIG. 9 is a structural diagram of a display device provided by an embodiment of the present application, and the display device may be used to implement various optional embodiments of the above-mentioned display method. As shown in FIG. 9 , the display device has a supplementary light module 910 , an acquisition module 920 , a processing module 930 , and a sending module 940 .

The supplementary light module 910 is configured to execute step S410 in the above display method and examples therein. The acquiring module 920 is configured to execute step S420 and examples thereof in the above display method. The processing module 930 is configured to execute any one of steps S430, S431-S433 and any optional example thereof in the above display method. The sending module 940 is configured to execute step S440 and examples thereof in the above display method. For details, refer to the detailed description in the method embodiments, and details are not repeated here.

The display device provided in this embodiment acquires the infrared image in front of the vehicle through infrared imaging, determines the display position of the marked information according to the position of the target object in the infrared image and the position of the user's eyes inside the vehicle, and uses the display device to display the The annotation information is displayed at the corresponding display position. Through the display device, the marked information seen by the user can be matched with the target object, which facilitates the user to quickly perceive the target object and its position outside the vehicle, and improves driving safety.

It should be understood that the display device in the embodiment of the present application may be implemented by software, for example, by computer programs or instructions having the above-mentioned functions, and the corresponding computer programs or instructions may be stored in the internal memory of the terminal and read by the processor. The corresponding computer programs or instructions inside the memory realize the above functions. Alternatively, the display device in the embodiment of the present application may also be realized by hardware, for example, the supplementary light module 910 may be realized by a supplementary light device on a vehicle, such as an infrared supplementary light lamp or other equipment capable of realizing infrared supplementary light function. The acquisition module 920 can be realized by an acquisition device outside the vehicle or an acquisition device inside the vehicle. For example, the acquisition device outside the vehicle can be an infrared camera, infrared radar, etc., and the acquisition device inside the vehicle can be a vehicle-mounted camera or an eye tracker, etc., or, The acquisition module 920 may also be implemented by an interface circuit between the processor and the infrared camera or the vehicle camera on the vehicle. The processing module 930 may be realized by a processing device on the vehicle, such as a processor of a vehicle-mounted processing device such as a vehicle machine or a vehicle-mounted computer, or may also be realized by a processor of a HUD or AR-HUD, or the processing module 930 may also be Realized by terminals such as mobile phones or tablets. The sending module 940 can be realized by the display device on the vehicle, for example, it can be realized by HUD or AR-HUD, or it can also be realized by some components of HUD or AR-HUD, or it can also be realized by the processor and HUD or AR-HUD The interface circuit between realizes. Alternatively, the display device in the embodiment of the present application may also be implemented by a combination of a processor and a software module.

It should be understood that for the processing details of the devices or modules in the embodiments of the present application, reference may be made to the relevant descriptions of the embodiments shown in FIGS. 1-8 and related extended embodiments, and the embodiments of the present application will not be repeated here.

In addition, the embodiment of the present application also provides a vehicle with the above-mentioned display device. The vehicle may be a family car or a truck, or may be a special vehicle such as an ambulance, a fire engine, a police car or an engineering emergency vehicle. In addition, the above-mentioned supplementary light device, acquisition device, processing device, sending device, etc. are also installed on the vehicle. Among them, the above-mentioned modules and devices can be arranged in the vehicle system in the form of pre-installation or post-installation, and the data interaction between the modules can rely on the bus or interface circuit of the vehicle, or with the development of wireless technology, each Modules can also use wireless communication for data interaction to eliminate the inconvenience caused by wiring. In addition, the display device of this embodiment can also be combined with the AR-HUD and installed on the vehicle in the form of a vehicle-mounted device, so as to achieve a better AR early warning effect.

FIG. 10 is a structural diagram of a computing device 1000 provided by an embodiment of the present application. The computing device can be used as a display device to execute the optional embodiments of the above display method, and the computing device can be a terminal, or a chip or a chip system inside the terminal. As shown in FIG. 10 , the computing device 1000 includes: a processor 1010 and a memory 1020 .

It should be understood that the computing device 1000 shown in FIG. 10 may further include a communication interface 1030 for communicating with other devices, and may specifically include one or more transceiver circuits or interface circuits.

Wherein, the processor 1010 may be connected to the memory 1020 . The memory 1020 can be used to store the program codes and data. Therefore, the memory 1020 may be an internal storage module of the processor 1010, or an external storage module independent of the processor 1010, or may include an internal storage module of the processor 1010 and an external storage module independent of the processor 1010. part.

Wherein, the computing device 1000 may further include a bus. Wherein, the memory 1020 and the communication interface 1030 may be connected to the processor 1010 through a bus. The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one line is used in FIG. 10 , but it does not mean that there is only one bus or one type of bus.

It should be understood that, in the embodiment of the present application, the processor 1010 may use a central processing unit (central processing unit, CPU). The processor can also be other general-purpose processors, digital signal processors (digital signal processors, DSPs), application specific integrated circuits (application specific integrated circuits, ASICs), off-the-shelf programmable gate arrays (field programmable gate arrays, FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. Alternatively, the processor 1010 adopts one or more integrated circuits for executing related programs, so as to implement the technical solutions provided by the embodiments of the present application.

The memory 1020 may include read-only memory and random-access memory, and provides instructions and data to the processor 1010 . A portion of processor 1010 may also include non-volatile random access memory. For example, processor 1010 may also store device type information.

When the computing device 1000 is running, the processor 1010 executes the computer-executed instructions in the memory 1020 to perform any operation steps of the above-mentioned display method and any optional embodiment thereof, for example, the processor 1010 can execute the memory The computer in 1020 executes instructions to execute the display method in the embodiment corresponding to FIG. 4 . When driving on a road with poor lighting at night, the processor 1010 controls the supplementary light of the vehicle by executing the supplementary light instruction in the memory 1020 The device performs infrared supplementary light on the target area outside the vehicle. The processor 1010 controls the acquisition device of the vehicle to acquire the infrared image information of the target area by executing the acquisition instruction in the memory 1020 . The processor 1010 determines the position of the target object in the infrared image by executing the processing instructions in the memory 1020, and determines the display position of the label information according to the position of the target object in the infrared image. The processor 1010 controls the sending device of the vehicle to display the marking information of the target object in the display area by executing the sending instruction in the memory 1020 .

It should be understood that the computing device 1000 according to the embodiment of the present application may correspond to a corresponding body executing the method according to each embodiment of the present application, and the above-mentioned and other operations and/or functions of each module in the computing device 1000 are for realizing the present invention For the sake of brevity, the corresponding processes of the methods in the embodiments are not repeated here.

Fig. 11 is a structure diagram of an electronic device 1100 provided by an embodiment of the present application. The electronic device 1100 can be used as a display device to execute various optional embodiments of the above-mentioned display method. The electronic device can be a terminal or a terminal A chip or system-on-a-chip inside. As shown in Fig. 11, the electronic device 1100 includes: a processor 1110, and an interface circuit 1120, wherein the processor 1110 accesses the memory through the interface circuit 1120, and the memory stores program instructions, and when the program instructions are executed by the processor, the processor executes the above-mentioned Any operational steps of the method and any alternative embodiments thereof are shown. For example, the processor 1110 can acquire computer-executable instructions in the memory through the interface circuit 1120 to execute the display method in the embodiment corresponding to FIG. Acquire the supplementary light instruction in the memory, and control the supplementary light device of the vehicle to perform infrared supplementary light on the target area outside the vehicle. The processor 1110 acquires the acquisition instruction in the memory through the interface circuit 1120, and controls the acquisition device of the vehicle to acquire the infrared image information of the target area. The processor 1110 acquires processing instructions in the memory through the interface circuit 1120, determines the position of the target object in the infrared image, and determines the display position of the label information according to the position of the target object in the infrared image. The processor 1110 acquires the sending instruction in the memory through the interface circuit 1120, and controls the sending device of the vehicle to display the labeling information of the target object in the display area.

In addition, the electronic device may further include a communication interface, a bus, etc. For details, please refer to the introduction in the embodiment shown in FIG. 10 , and details are not repeated here.

Fig. 12 is a structure diagram of a display system 1200 provided by the embodiment of the present application. The display system 1200 can be used as a display device to execute various optional embodiments of the above display methods. The display system can be a terminal or a terminal A chip or system-on-a-chip inside. As shown in FIG. 12 , the display system 1200 includes: an in-vehicle device 1210 , and an electronic device 1220 coupled with the in-vehicle device 1210 . The electronic device 1220 may be the processing device 130 as shown in FIG. 1 , may also be the processing module 930 as shown in FIG. 9 , may also be the computing device 1000 as shown in FIG. 10 , or may be the The electronic device 1100. The in-vehicle device 1210 may be the supplementary light device 110 on the vehicle shown in FIG. 1 or the supplementary light module 910 shown in FIG. 9 , such as a vehicle headlight, an infrared supplementary light, and the like. The vehicle-machine device 1210 may also be the collection device 120 on the vehicle shown in FIG. 1 or the acquisition module 920 shown in FIG. 9 , such as a vehicle-mounted camera, an infrared camera, or a radar. Alternatively, the vehicle-machine device 1210 may also be the sending device 140 on the vehicle shown in FIG. 1 or the sending module 940 shown in FIG. 9 , such as AR-HUD, HUD, etc., or some devices in the AR-HUD or HUD. In this embodiment, the display system 1200 can execute the display method in the embodiment corresponding to FIG. Infrared images are collected for the target area outside the vehicle. The electronic device 1220 can determine the position of the target object in the infrared image according to the acquired infrared image containing the target object, and determine the display position of the annotation information according to the position of the target object in the infrared image. The in-vehicle device 1210 may send the marked information to a display location for display. Wherein, the in-vehicle device 1210 and the electronic device 1220 can communicate data or instructions through a wired method (such as an interface circuit), and can also perform data or instruction communications through a wireless method (such as bluetooth, wifi).

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing device, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and the program is used to execute a display method when executed by a processor, and the method includes the solutions described in the above-mentioned embodiments at least one.

The computer storage medium in the embodiments of the present application may use any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include: electrical connections with one or more leads, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a data signal in baseband or propagated as part of a carrier wave with computer readable program code embodied therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. .

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for performing the operations of the present application may be written in one or more programming languages or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through the Internet using an Internet service provider). connect).

It should be noted that the embodiments described in this application are only some of the embodiments of this application, not all of them. The components of the embodiments of the application generally described and shown in the drawings may be arranged and designed in a variety of different configurations. Accordingly, the above detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of the present application.

The words "first, second, third, etc." or similar terms such as module A, module B, and module C in the description and claims are only used to distinguish similar objects, and do not represent a specific ordering of objects. It can be understood that Obviously, where permitted, the specific order or sequence can be interchanged such that the embodiments of the application described herein can be practiced in other sequences than those illustrated or described herein.

In the above description, the referenced numbers representing the steps, such as S410, S420... etc., do not mean that this step must be executed, and may also include intermediate steps or be replaced by other steps, where permitted The order of the preceding and following steps can be interchanged, or performed simultaneously.

The term "comprising" used in the description and claims should not be interpreted as being restricted to what is listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of said features, integers, steps or components, but not excluding the presence or addition of one or more other features, integers, steps or components and groups thereof. Therefore, the expression "apparatus comprising means A and B" should not be limited to an apparatus consisting of parts A and B only.

Reference in this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" in various places in this specification do not necessarily all refer to the same embodiment, but may. In addition, in each embodiment of the present application, if there is no special explanation and logical conflict, the terms and/or descriptions between different embodiments are consistent and can be referred to each other, and the technical features in different embodiments are based on their Inherent logical relationships can be combined to form new embodiments.

Note that the above are only preferred embodiments and technical principles used in this application. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present application has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention, all of which belong to protection scope of the present invention.

Claims

A display method, characterized by comprising:

Infrared supplementary light for the target area outside the vehicle;

acquiring information of an infrared image of the target area, wherein the target area includes a target object;

determining the location of the target object in the infrared image;

According to the position of the target object in the infrared image, the label information of the target object is displayed in a display area.
The method according to claim 1, wherein the display position of the annotation information is related to the spatial position of the target object and the position of the user's eyes.
The method according to claim 2, wherein the display size of the annotation information is related to the display position of the annotation information, the position of the user's eyes, and the size of the target object.
The method according to claim 2, wherein the display area is a display area of an augmented reality head-up display, and the display position of the annotation information is related to the spatial position of the target object and the position of the user's eyes, including:

The display position of the annotation information is determined by the user's first line of sight; the first line of sight is the line of sight from the user's eye position to the spatial position of the target object.
The method according to claim 4, wherein the display area of the augmented reality head-up display is located on the front windshield of the vehicle, and the display position of the annotation information is determined by the user's first line of sight include:

The display position of the annotation information is determined by the intersection of the user's first line of sight and the front windshield of the vehicle.
The method according to any one of claims 2 to 5, characterized in that the spatial position of the target object and the position of the target object in the infrared image, the internal reference of the image acquisition device for obtaining the infrared image related to extrinsics.
The method according to claim 1, wherein the target object includes one or more objects among other vehicles, pedestrians, and animals.
The method according to claim 1, wherein before the determining the position of the target object in the infrared image, further comprising:

One or more of cropping, noise reduction, enhancement, smoothing, and sharpening are performed on the infrared image.
A display device, characterized in that it comprises:

The supplementary light module is used to perform infrared supplementary light on the target area outside the vehicle;

An acquisition module, configured to acquire information of an infrared image of the target area, wherein the target area includes a target object;

a processing module, configured to determine the position of the target object in the infrared image;

The sending module is configured to display the labeling information of the target object in the display area according to the position of the target object in the infrared image.
The device according to claim 9, wherein the display position of the annotation information is related to the spatial position of the target object, the position of the user's eyes, and the size of the target object.
The device according to claim 10, wherein the display size of the annotation information is related to the display position of the annotation information, the position of the user's eyes, and the size of the target object.
The device according to claim 10, wherein the display area is a display area of an augmented reality head-up display, and the display position of the annotation information is related to the spatial position of the target object and the position of the user's eyes, including:

The display position of the annotation information is determined by a first line of sight of the user's first line of sight; the first line of sight is a line of sight from the user's eye position to the spatial position of the target object.
The device according to claim 12, wherein the display area of the augmented reality head-up display is located on the front windshield of the vehicle, and the display position of the annotation information is the first line of sight of the user. A line-of-sight determination includes:

The display position of the annotation information is determined by the intersection of the user's first line of sight and the front windshield of the vehicle.
The device according to any one of claims 10 to 13, characterized in that the spatial position of the target object and the position of the target object in the infrared image, and the internal parameters of the image acquisition device for obtaining the infrared image related to extrinsics.
The device according to claim 9, wherein the target object includes one or more objects among other vehicles, pedestrians, and animals.
The device according to claim 9, wherein before the processing module is used to determine the position of the target object in the infrared image, it is further used to:

One or more of cropping, noise reduction, enhancement, smoothing, and sharpening are performed on the infrared image.
A computing device, characterized in that, comprising

processor, and

A memory, on which program instructions are stored, and the program instructions, when executed by the processor, cause the processor to execute the display method according to any one of claims 1 to 8.
An electronic device, characterized in that it comprises:

A processor, and an interface circuit, wherein the processor accesses a memory through the interface circuit, the memory storing program instructions that, when executed by the processor, cause the processor to perform claim 1 The display method described in any one of to 8.
A display system, characterized in that it comprises:

A vehicle-machine device, and the computing device described in claim 17 coupled with the vehicle-machine device, or the electronic device described in claim 18 coupled with the vehicle-machine device.
A computer-readable storage medium, characterized in that program instructions are stored thereon, and when the program instructions are executed by a computer, the computer executes the display method described in any one of claims 1 to 8.
A computer program product, characterized in that it includes program instructions, and when the program instructions are executed by a computer, the computer executes the display method described in any one of claims 1 to 8.