WO2024001884A1

WO2024001884A1 - Road condition prompting method, and electronic device and computer-readable medium

Info

Publication number: WO2024001884A1
Application number: PCT/CN2023/101492
Authority: WO
Inventors: 曹宇宁
Original assignee: 深圳市中兴微电子技术有限公司
Priority date: 2022-06-29
Filing date: 2023-06-20
Publication date: 2024-01-04
Also published as: CN117351754A

Abstract

A road condition prompting method, which comprises: generating a sound field signal according to road condition information, wherein the sound field signal is used for simulating a road condition, which is represented by the road condition information (S1); and generating a virtual sound field according to the sound field signal, and giving a road condition prompt in an audio-based augmented reality manner (S2). Further provided are an electronic device and a computer-readable medium.

Description

Traffic condition prompting method, electronic device, computer readable medium

Cross-references to related applications

This application claims priority from Patent Application No. 202210749766.4 submitted to the China Patent Office on June 29, 2022, the entire content of which is incorporated herein by reference.

Technical field

The present disclosure relates to, but is not limited to, the field of augmented reality technology.

Background technique

With the trend of intelligent and connected cars, the digital transformation of the automobile industry continues to deepen, and the relationship between people, cars, and the environment is also constantly evolving. Human-computer interaction design has become a core element of the development and innovation of smart cars. In the future, smart cars will serve as platforms integrating a large number of functions, and human-computer interaction will play a key role in car safety and operating efficiency. The working efficiency, safety, and convenience of the human-computer interaction system will directly affect the operating status and safety performance of the car. Common automotive human-computer interaction systems include instrument panels, touch screens, and heads-up displays (HUD, Head Up Display). Drivers mainly rely on vision when using the above-mentioned human-computer interaction systems and have to often move their eyes away from the front, giving the driver It poses safety risks, and frequent head and eye movements can also cause driver fatigue and affect the driving experience.

Automobile human-computer interaction systems can also be used to provide road condition prompts. However, common road condition prompts are implemented through traffic lights, displays, radar charts, etc., which not only cause safety hazards due to the driver's movement of sight, but also the content of the traffic conditions that can be prompted. Also very limited.

Contents of the invention

The present disclosure provides a road condition prompting method based on audio augmented reality, an electronic device, and a computer-readable medium.

In a first aspect, the present disclosure provides a road condition prompting method, which includes: generating a sound field signal according to the road condition information, and the sound field signal is used to perform traffic conditions represented by the road condition information. Simulation; generate a virtual sound field based on the sound field signal, and provide traffic prompts in the form of audio augmented reality.

In a second aspect, the present disclosure provides an electronic device, including: one or more processors; and a memory on which one or more programs are stored. When the one or more programs are processed by the one or more processors, Execution causes the one or more processors to implement the traffic prompt method described in the first aspect of this disclosure.

In a third aspect, the disclosure provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the road condition prompting method described in the first aspect of the disclosure is implemented.

Description of drawings

Figure 1 is a flow chart of a road condition prompting method in the present disclosure;

Figure 2 is a flow chart of some steps in another road condition prompting method in the present disclosure;

Figure 3 is a flow chart of some steps in yet another road condition prompting method in the present disclosure;

Figure 4 is a flow chart of some steps in yet another road condition prompting method in the present disclosure;

Figure 5 is a flow chart of some steps in yet another road condition prompting method in the present disclosure;

Figure 6 is a flow chart of some steps in yet another road condition prompting method in the present disclosure;

Figure 7 is a flow chart of some steps in yet another road condition prompting method in the present disclosure;

Figure 8 is a flow chart of some steps in yet another road condition prompting method in the present disclosure;

Figure 9 is a block diagram of an electronic device in the present disclosure;

Figure 10 is a block diagram of a computer-readable medium in the present disclosure;

Figure 11 is a schematic diagram of the system architecture of a traffic prompt in the present disclosure.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the audio augmented reality (AR, Augmented Reality)-based road condition prompting method, electronic equipment, and computer-readable media provided by the present disclosure will be described in detail below in conjunction with the accompanying drawings. .

Example embodiments will be described more fully below with reference to the accompanying drawings, which may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. comprehensive and will enable those skilled in the art to fully understand the scope of the present disclosure.

The various embodiments and features in the embodiments of the present disclosure may be combined with each other without conflict.

As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used herein is used to describe particular embodiments only and is not intended to limit the disclosure. As used herein, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and/or "made of" are used in this specification, the presence of stated features, integers, steps, operations, elements and/or components is specified but does not exclude the presence or Add one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be construed to have meanings consistent with their meanings in the context of the relevant art and the present disclosure, and will not be construed as having idealized or excessive formal meanings, Unless expressly so limited herein.

In the first aspect, referring to FIG. 1 , an embodiment of the present disclosure provides a road condition prompting method, including S1 and S2.

In S1, a sound field signal is generated according to the road condition information, and the sound field signal is used to simulate the road condition represented by the road condition information.

In S2, a virtual sound field is generated according to the sound field signal, and traffic conditions are prompted in the form of audio augmented reality.

The road condition prompting method provided by this disclosure can be executed in the centrally controlled Advanced Driving Assistance System (ADAS) of the car, or in the smart wearable device (such as an AR head-mounted display device) worn by the driver of the car. implement. This disclosure does not impose special limitations on this.

In the present disclosure, road condition information is information that can reflect real-time road conditions and is obtained by monitoring the road conditions around the car. In some implementations, the car monitors the road conditions around the car and obtains road condition information through sensors such as millimeter wave radar and cameras.

In this disclosure, generating a virtual sound field based on a sound field signal refers to generating a virtual sound field based on a sound field signal. Surround sound technology creates a virtual sound field, and audio augmented reality is used to simulate road conditions, thereby achieving the purpose of reminding the driver of road conditions.

It should be noted that creating a virtual sound field based on sound field signals is based on auditory psychology and simulates a three-dimensional sound field to make the driver feel like he is in a three-dimensional space. The driver can perceive the road conditions based on the sounds in the virtual sound field. For example, the driver can determine the location of the sound source based on the sound in the virtual sound field, and the location of the sound source in the virtual sound field represents the relative positional relationship between the car and other target vehicles on the road. Therefore, the driver can determine the location of the sound source based on the virtual sound field. The sound in the vehicle determines the relative positional relationship between the car and other target vehicles on the road, that is, it obtains road condition prompts.

In the present disclosure, the process of generating a sound field signal based on road condition information is a process of constructing a three-dimensional sound field to simulate the road conditions, and the sound field signal is a data description of the constructed three-dimensional sound field.

In the road condition prompting method provided by the present disclosure, a sound field signal can be generated based on the road condition information, the road conditions represented by the road condition information can be simulated, and then a virtual sound field can be created based on the sound field signal, so that the driver can perceive the road conditions using the sounds in the virtual sound field, thereby realizing It uses audio augmented reality to provide drivers with road condition prompts. This disclosure provides road condition prompts based on the driver's hearing, so the driver does not need to move his or her eyes, thus eliminating potential safety hazards, and the audio augmented reality-based prompts for rich road condition content are conducive to improving the driving experience.

This disclosure does not place special limitations on the traffic content prompted in the form of audio augmented reality. In some implementations, the morphological characteristics of the target vehicle around the car, such as the size, shape, type, etc. of the target vehicle, can be prompted in an audio augmented reality manner. In some embodiments, audio augmented reality can be used to prompt the movement status of the target vehicle around the car relative to the car, such as the position and speed of the target vehicle relative to the car. In some embodiments, the audio augmented reality method can not only prompt the morphological characteristics of the target vehicles around the car, but also prompt the motion status of the target vehicles around the car relative to the car.

This disclosure does not place any special limitations on how to generate sound field signals based on road condition information.

It should be noted that in the present disclosure, the sound field signal generated according to the road condition information determines the traffic condition content prompted in the form of audio augmented reality. For example, if the generated sound field signal contains information that simulates the morphological characteristics of the target vehicle, then the virtual image generated based on the sound field signal will The onomatopoeic field contains sound features that can reflect the morphological characteristics of the target vehicle; the generated sound field signal contains information that simulates the motion state of the target vehicle, and the virtual sound field generated based on the sound field signal contains information that can reflect the motion state of the target vehicle. sound characteristics. This disclosure does not impose special limitations on this.

Correspondingly, in some implementations, referring to FIG. 2 , a sound field signal is generated according to the road condition information, including S11 and S12.

In S11, a virtual sound image is determined based on the morphological information characterizing the morphological characteristics of the target vehicle in the road condition information, where the virtual sound image is used to simulate the morphological characteristics of the target vehicle through sound.

In S12, according to the motion information characterizing the motion state of the target vehicle in the road condition information, a sound field effect is added to the virtual sound image to simulate the motion state of the target vehicle to obtain the sound field signal.

In the present disclosure, the virtual sound image is a virtual sound source corresponding to the target vehicle, and the virtual sound image is used to simulate the sound emitted by the target vehicle. For example, the virtual sound image is used to simulate a sound that is a mixture of at least one of tire noise, wind noise, engine sound, etc. when the target vehicle is driving.

This disclosure does not place any special limitations on the morphological characteristics of the vehicle. For example, the morphological characteristics of the vehicle may include the type of the vehicle, such as a car, a truck, an off-road vehicle, etc.; the morphological characteristics of the vehicle may also include the size of the vehicle; and the morphological characteristics of the vehicle may also include the weight of the vehicle.

It should be noted that vehicles of different forms emit different sounds when driving. In the present disclosure, the sounds emitted by vehicles of different forms while driving can be simulated through different virtual sound images, so that the driver can respond to different sounds based on the virtual sound images. Differentiation of vehicles.

It should also be noted that in the present disclosure, a sound field effect is added to the virtual sound image so that the driver can perceive the position and movement state of the virtual sound source in the virtual sound field, and the position and movement state of the virtual sound source in the virtual sound field are The status, etc. are consistent with the position and motion state of the target vehicle relative to the current vehicle, thereby enabling the driver to perceive the position and motion state of the target vehicle relative to the current vehicle.

In some implementations, a sound retrieval system (SRS) is used to process the virtual sound image according to auditory psychological effects, thereby adding a sound field effect to the virtual sound image and creating a virtual sound field.

In the present disclosure, virtual sound and images are used to simulate the morphological characteristics of the target vehicle, and sound field effects are added to the virtual sound and images, which can prompt the driver with road condition information such as the morphological characteristics and motion status of the target vehicle, enriching the content of the road condition prompts.

This disclosure does not specifically limit how to determine the virtual sound and image based on the morphological information of the target vehicle.

In some embodiments, referring to FIG. 3 , determining the virtual sound image according to the morphological information characterizing the morphological characteristics of the target vehicle in the road condition information includes S111 to S113.

In S111, the tonnage level of the target vehicle is determined based on the morphological information.

In S112, at least one voiceprint feature is determined according to the tonnage level of the target vehicle, where different voiceprint features correspond to different tonnage levels.

In S113, the virtual sound image is generated according to the at least one voiceprint feature.

In this disclosure, vehicles are divided into multiple tonnage levels, and voiceprint features are recorded or extracted in advance corresponding to each tonnage level. The voiceprint features include tire noise, wind noise, engine sound, etc. when the vehicle of this tonnage level is driving. sound characteristics.

In the present disclosure, a virtual sound image is generated based on at least one voiceprint feature, that is, the voiceprint feature corresponding to at least one sound such as tire noise, wind noise, and engine sound when a vehicle of a corresponding tonnage level is driving is modulated to generate a virtual sound image. picture.

It should be noted that in this disclosure, it is considered that vehicles of different tonnage have different impacts on the driving safety of the current vehicle. For example, a vehicle with a larger tonnage has a greater impact on the driving safety of the current vehicle. Therefore, audio augmented reality is used This method prompts the driver with the tonnage information of the target vehicle, which is helpful for the driver to perform corresponding driving actions based on the tonnage of the target vehicle, thereby improving driving safety.

This disclosure does not place any special restrictions on how to determine the tonnage level of the target vehicle.

In some embodiments, the morphological information includes the type and size of the target vehicle; determining the tonnage level of the target vehicle based on the morphological information includes: determining the target vehicle based on the type and size of the target vehicle. The tonnage rating of the vehicle.

In the present disclosure, the current vehicle can obtain the size of the target vehicle through sensors such as millimeter wave radar and cameras around the vehicle, and can identify the type of the target vehicle, such as cars, trucks, off-road vehicles, etc. Based on the size and type of the target vehicle, the tonnage class of the target vehicle can be estimated.

In some embodiments, the cross-sectional area of the target vehicle on a vertical plane represents the size of the target vehicle.

This disclosure does not place special limitations on how to obtain the cross-sectional area of the target vehicle on the vertical plane.

In some implementations, the current vehicle uses millimeter wave radar to detect the distance between the target vehicle and the current vehicle, uses a camera to capture the image of the target vehicle, and then determines the position of the target vehicle in the image through a target detection algorithm, and extracts the image block of the target vehicle. , measure the number of pixels corresponding to the length and width of the image block; then calculate the actual width and height of the target vehicle based on the distance of the target vehicle obtained by the millimeter-wave radar and the focal length of the camera lens, and then calculate the distance of the target vehicle on the vertical plane Sectional area.

In some embodiments, referring to Figure 4, the motion information includes position information and speed information of the target vehicle relative to the current vehicle; according to the motion information characterizing the motion state of the target vehicle in the road condition information, the motion information is Adding a sound field effect to the virtual sound image includes: S121. Adding a sound field effect to the virtual sound image according to the position information and speed information of the target vehicle relative to the current vehicle.

In the present disclosure, a sound field effect is added to the virtual sound image, so that the driver can perceive the position and moving speed of the virtual sound source in the virtual sound field, and the position and moving speed of the virtual sound source in the virtual sound field are consistent with the target vehicle relative to the current vehicle. The position and speed of the target vehicle are consistent, allowing the driver to perceive the position and speed of the target vehicle relative to the current vehicle.

This disclosure does not impose special limitations on the sound field effect.

In some embodiments, the sound field effect includes at least one of binaural volume difference, binaural delay, pinna effect, reflection effect, absorption effect, and Doppler effect.

In some embodiments, the virtual sound image is processed based on the auditory psychological effect based on the Sound Retrieval System (SRS), adding sound field effects to the virtual sound image, simulating binaural volume difference, binaural delay, and pinna effect. , reflection effect, absorption effect, and Doppler effect, thereby creating a virtual sound field.

In some embodiments, referring to FIG. 5 , before generating the sound field signal according to the road condition information, the road condition prompting method further includes: S3. Obtaining the road condition information.

In some implementations, road condition information is obtained through sensors such as millimeter wave radars and cameras around the vehicle.

In the present disclosure, road condition prompts are provided to the driver only when an event requiring prompts occurs.

Correspondingly, in some embodiments, referring to Figure 6, obtaining the traffic information includes: S31. Monitoring the road conditions to determine whether there is an event to be prompted; S32. When there is an event to be prompted, obtaining the traffic information.

This disclosure does not place special restrictions on prompt events. For example, the event to be prompted can be that the target vehicle enters the blind spot of the current vehicle's mirror/rearview mirror; the event to be prompted can also be that the distance between the target vehicle and the current vehicle is less than a specific threshold; the event to be prompted can also be that the target vehicle is overtaking.

In the present disclosure, road condition prompts are only provided to the driver when an event that requires prompting occurs, which is helpful to avoid unnecessary prompt information from interfering with the driver and improve the driving experience.

This disclosure does not place special restrictions on obtaining traffic information.

In some embodiments, referring to Figure 7, when there is an event to be prompted, obtaining the road condition information includes: S321, obtaining the position information and speed information of the target vehicle relative to the current vehicle; S322, obtaining the video data of the target vehicle ; S323. Determine the type and size of the target vehicle based on the position information of the target vehicle relative to the current vehicle and the video data.

In some embodiments, types of vehicles include cars, trucks, SUVs, and the like.

In some embodiments, referring to Figure 8, a virtual sound field is generated according to the sound field signal, and traffic prompts are provided in an audio augmented reality manner, including: S21. Using the audio playback device Be prepared to play stereo audio based on the sound field signal and provide traffic prompts.

In the present disclosure, the audio playback device plays stereo audio according to the sound field signal, that is, a virtual sound field is created around the driver, thereby realizing road condition prompts in an audio augmented reality manner.

This disclosure does not impose special limitations on audio playback devices. For example, the audio playback device can be the sound system of the vehicle; the audio playback device can also be the sound playback system of a smart wearable device worn by the driver (such as an AR head-mounted display device); the audio playback device can also be connected to the central control system of the vehicle. Or the driver wears headphones connected to a smart wearable device.

In some embodiments, the audio playback device is a bone conduction earphone.

In the present disclosure, bone conduction headphones are used to create a virtual sound field to remind the driver of road conditions. The bone conduction headphones can conduct sound information to the driver's auditory nerve through the auditory ossicles. The driver can also receive other sounds through the ears, such as hearing Music, navigation, etc. can help improve the driving experience.

In the second aspect, referring to FIG. 9 , the present disclosure provides an electronic device, which includes: one or more processors 101; a memory 102 on which one or more programs are stored. When one or more programs are processed by one or more A processor is executed, so that one or more processors implement the traffic prompt method described in the first aspect of this disclosure; one or more I/O interfaces 103 are connected between the processor and the memory, and are configured to implement the processor and the memory. Memory information exchange.

Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically Such as SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I/O interface (read-write interface) 103 is connected between the processor 101 and the memory 102 , can realize information interaction between the processor 101 and the memory 102, which includes but is not limited to a data bus (Bus), etc.

In some implementations, processor 101, memory 102, and I/O interface 103 are connected to each other and, in turn, to other components of the computing device via bus 104.

In some implementations, the electronic device is a head-mounted display device. For example, augmented reality (AR, Augmented Reality) head-mounted display devices.

In some embodiments, the electronic device includes an audio playback device. For example, sound The video playback equipment is bone conduction headphones.

In a third aspect, referring to FIG. 10 , the present disclosure provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the road condition prompting method described in the first aspect of the present disclosure is implemented.

In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure are described in detail below through exemplary embodiments:

The system architecture of traffic prompts based on audio reality augmentation in this exemplary embodiment is shown in Figure 11. The system architecture is centered on an augmented display (AR) head-mounted display device, including multiple automotive sensors and a car central control system. The AR head-mounted display device includes a sound restoration system (SRS) processing engine and bone conduction headphones.

Bone conduction headphones are configured to alert drivers to road conditions in an audio-augmented reality manner.

The SRS processing engine is configured to simulate a three-dimensional sound field based on auditory psychology, making the driver feel like they are in a three-dimensional sound field. In the three-dimensional sound field, sound can come from anywhere in the spherical space surrounding the driver, and the driver can accurately determine the location of the sound source. The SRS processing engine psychologically and subjectively restores the sound wave state (direct sound, reflected sound, reverberation sound) caused by the real sound source at the ears, and reproduces the orientation and spatial distribution of the real sound source, making it People have the feeling of being there.

In addition, the system architecture also includes: a communication module. The communication module can choose to support wireless communication protocols such as Bluetooth and wifi or wired communication protocols such as USB. It is configured to receive information from the car’s central control system, instruments, ADAS and physical buttons, and Transmits information to the AR head-mounted display device; the battery and power management module is configured to power the AR head-mounted display device; the operating system is responsible for the software and hardware management of the system.

In this exemplary embodiment, the process of traffic prompts based on audio reality augmentation includes the following three stages.

1. The car central control system monitors road conditions in real time through sensors installed in all directions of the current vehicle. When an event that needs to be prompted is found and the driver needs to be reminded, the road condition information is extracted and transmitted to the AR head-mounted display device through the communication module. For example, if a millimeter-wave radar that illuminates the side and rear finds that a target vehicle has invaded the blind spot of the rearview mirror of the current vehicle, the relative position and speed information of the detected target vehicle will be transmitted to the central control system of the current vehicle. The vehicle central control system selects and retrieves the video data of the side and rear cameras according to the message type, and identifies the target vehicle in the video, such as distinguishing cars, off-road vehicles, trucks, etc., and detects the position of the intruding target vehicle on the vertical plane. cross-sectional area, and then transmit this information to the AR head-mounted display device in digital form in real time. The cross-sectional area can be obtained in the following way: the current vehicle uses millimeter wave radar to detect the distance between the target vehicle and the current vehicle, uses a camera to capture the image of the target vehicle, and then determines the position of the target vehicle in the image through the target detection algorithm, and extracts the image of the target vehicle. Block, measure the number of pixels corresponding to the length and width of the image block; then calculate the actual width and height of the target vehicle based on the distance of the target vehicle obtained by the millimeter wave radar and the focal length of the camera lens, and then calculate the vertical plane position of the target vehicle cross-sectional area on.

2. After the AR head-mounted display device receives the road condition information from the car's central control system, it performs secondary processing on the road condition information: estimating the tonnage of the target vehicle based on the type of the target vehicle and the cross-sectional area of the target vehicle on the vertical plane. ; The tonnage of the vehicle is pre-divided into multiple tonnage levels. Each tonnage level corresponds to a voiceprint feature. The voiceprint feature is pre-recorded and extracted, which is roughly equivalent to the tire noise, wind noise and noise of a car of that tonnage level when driving. The mixture of at least one of the engine sound and the like uses the voiceprint characteristics corresponding to the target vehicle to modulate a virtual sound image to simulate the target vehicle.

The SRS processing engine creates a virtual sound field based on the auditory psychological effect, and then uses the position and moving speed of the virtual sound image in the sound field to simulate the position and relative speed of the vehicle behind, thereby achieving the purpose of reminding the driver. The principle is as follows:

Since there is a certain distance between the two ears of a person, as long as the sound does not come from the midline connecting the two ears, the distance from the sound source to the ear is different. Sound takes time to propagate in the air. Different distances will cause different propagation times. At the same time, because the sound will attenuate as the distance increases, the loudness of the sound reaching the two ears is also different. These two factors allow us to tell the horizontal direction of a sound. As sound propagates in the medium, its amplitude will decrease as the distance increases, and generally the attenuation of high-frequency components is greater than that of low-frequency components. Our brain can roughly calculate the distance between the sound source and us based on such characteristics. But since we have no way of knowing what the sound source should look like, we can only infer the distance of the sound based on experience, but cannot make an accurate judgment. Only when the spectral distribution and amplitude of the sound source and the attenuation characteristics of the medium with respect to frequency can be known exactly, Only in this way can the distance of the sound source be accurately calculated. But when the sound source is on the vertical bisector of the two ears, the sound received by the two ears has neither phase difference nor loudness difference. How do we judge the position of the sound at this time? Our ears have a very special shape. The complex folds and bulges allow the original sound to be superimposed with the sound reflected by the auricle after it reaches the auricle. Due to the existence of phase difference, some frequency components are enhanced and some frequency components are attenuated. In this way, we can more specifically distinguish the direction of the sound through the sound changed by the pinna. For the auricle, we can establish a function that describes its impact on the frequency characteristics of sound. This function is the head-related transfer function (HRTF). HRTF is a function of sound location. Sounds from different places will have different frequency corresponding characteristics after passing through HRTF, so that we can distinguish sounds from different places.

The HRTF synthesis simulation circuit can be divided into two main parts: digital filter and delay.

The application first sends the optimized composite coordinates to the HRTF to find the appropriate parameters, and then it returns the parameters to the digital filter. The filters of the left and right channels calculate the frequency curve that is easy for people to hear - the volume difference between the two ears (IAD) and the pinna effect of this frequency.

After simulating the 3D characteristics of the frequency band, the only thing left is the interaural time delay difference (ITD). Synthetic ITD only needs to reconstruct the approximate estimated delay according to the digital delay circuit.

Delay of reflections, wall absorption effects, field effects and Doppler effects are implemented via the API.

3. The central processor of the AR head-mounted display device transmits the processed modulation signal to the bone conduction headset in real time. The headset restores the sound field signal through stereo technology and reminds the driver of road conditions.

Those of ordinary skill in the art can understand that all or some steps, systems, and functional modules/units in the devices disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may consist of several physical components. Components execute cooperatively. Some or all of the physical components may be implemented by a processor, such as Software executed by a central processing unit, digital signal processor, or microprocessor is implemented either as hardware or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. removable, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical disk storage, magnetic cassettes, tapes, disk storage or other magnetic storage devices, or may Any other medium used to store desired information and that can be accessed by a computer. Additionally, it is known to those of ordinary skill in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted in a general illustrative sense only and not for purpose of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics and/or elements described in connection with a particular embodiment may be used alone, or may be used in conjunction with other embodiments, unless expressly stated otherwise. Features and/or components are used in combination. Accordingly, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

A road condition prompting method includes:

Generate a sound field signal according to the road condition information, and the sound field signal is used to simulate the road condition represented by the road condition information;

A virtual sound field is generated according to the sound field signal, and traffic prompts are provided in the form of audio augmented reality.
The road condition prompting method according to claim 1, wherein generating the sound field signal according to the road condition information includes:

Determine a virtual sound image according to the morphological information characterizing the morphological characteristics of the target vehicle in the road condition information, wherein the virtual sound image is used to simulate the morphological characteristics of the target vehicle through sound;

According to the motion information representing the motion state of the target vehicle in the road condition information, a sound field effect is added to the virtual sound image to simulate the motion state of the target vehicle to obtain the sound field signal.
The road condition prompting method according to claim 2, wherein determining the virtual sound and image according to the morphological information characterizing the morphological characteristics of the target vehicle in the road condition information includes:

Determine the tonnage level of the target vehicle according to the morphological information;

Determine at least one voiceprint feature according to the tonnage level of the target vehicle, wherein different voiceprint features correspond to different tonnage levels;

The virtual sound image is generated based on the at least one voiceprint feature.
The road condition prompting method according to claim 3, wherein the morphological information includes the type and size of the target vehicle; determining the tonnage level of the target vehicle according to the morphological information includes:

The tonnage class of the target vehicle is determined based on the type and size of the target vehicle.
The road condition prompting method according to claim 2, wherein the motion information includes position information and speed information of the target vehicle relative to the current vehicle; according to the motion information characterizing the motion state of the target vehicle in the road condition information, Add sound field effects to the virtual sound image, including:

According to the position information and speed information of the target vehicle relative to the current vehicle, a sound field effect is added to the virtual sound image.
The road condition prompting method according to claim 5, wherein the sound field effect includes at least one of binaural volume difference, binaural delay, auricle effect, reflection effect, absorption effect, and Doppler effect.
The road condition prompting method according to any one of claims 1 to 6, wherein before generating the sound field signal according to the road condition information, the road condition prompting method further includes:

Get the traffic information.
The traffic prompting method according to claim 7, wherein obtaining the traffic information includes:

Monitor road conditions to determine whether there are events to be prompted;

When there is an event to be prompted, the traffic information is obtained.
The traffic prompting method according to claim 8, wherein when there is an event to be prompted, obtaining the traffic information includes:

Obtain the position information and speed information of the target vehicle relative to the current vehicle;

Obtain video data of the target vehicle;

The type and size of the target vehicle are determined based on the position information of the target vehicle relative to the current vehicle and the video data.
The traffic prompting method according to any one of claims 1 to 6, wherein a virtual sound field is generated according to the sound field signal, and traffic prompting is performed in an audio augmented reality manner, including:

The audio playback device plays stereo audio according to the sound field signal to provide traffic prompts.
The road condition prompting method according to claim 10, wherein the audio playback device is a bone conduction earphone.
An electronic device including:

one or more processors;

A memory having one or more programs stored thereon. When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11. The road condition prompting method described in the item.
A computer-readable medium on which a computer program is stored. When the program is executed by a processor, the road condition prompting method according to any one of claims 1 to 11 is implemented.