WO2021258671A1 - Assisted driving interaction method and apparatus based on vehicle-mounted digital human, and storage medium - Google Patents

Abstract

An assisted driving interaction method and apparatus based on a vehicle-mounted digital human, and a storage medium. The method comprises: obtaining perception information of an environment outside a vehicle (101); and according to the perception information of the environment outside the vehicle, generating and displaying an animation of interaction information which is made by a digital human, used for assisting driving, and displayed on a vehicle-mounted display device provided in a vehicle cabin (102).

Description

Assistant driving interaction method and device based on vehicle-mounted digital human, and storage medium

Cross-references to related applications

This disclosure requires the priority of a Chinese patent application filed with the Chinese Patent Office on June 24, 2020, the application number is CN202010592118.3, and the invention title is "assisted driving interaction method and device based on vehicle-mounted digital human, and storage medium". The entire content is incorporated into this disclosure by reference.

Technical field

The present disclosure relates to the field of augmented reality, and in particular to an assisted driving interaction method and device based on a vehicle-mounted digital human, and a storage medium.

Background technique

With the rapid development of intelligent driving technology, autonomous driving technology and assisted driving technology are becoming more mature. At present, the perception results of most driving assistance systems are text display and/or voice broadcast through on-board equipment, which will give the people in the car a sense of rigidity in interacting with the machine, and the interaction mode between the on-board equipment and the people in the car is relatively fixed. , Lack of humanization.

Summary of the invention

The present disclosure provides an assisted driving interaction method and device based on a vehicle-mounted digital human, and a storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided an assisted driving interaction method based on a vehicle-mounted digital human. The method includes: acquiring external environment perception information; according to the external environment perception information, generating and displaying the The digital person displayed on the on-board display device in the cabin makes an animation of interactive information used to assist driving.

In some embodiments, the generating and displaying an animation of interactive information used to assist driving by a digital person displayed on a vehicle-mounted display device installed in the vehicle cabin according to the perception information of the outside environment of the vehicle includes: determining and Action information matched with the perception information of the outside environment for driving assistance; generated according to the action information and displayed on the vehicle-mounted display device in the cabin an animation of the digital person performing the corresponding action.

In some embodiments, the generating and displaying an animation of a digital person performing a corresponding action on a vehicle-mounted display device in the cabin according to the action information includes: determining a voice that matches the perception information of the outside environment Information; Obtain the corresponding voice according to the voice information, the voice includes a timestamp; while playing the voice, generate and display the digital person at the time corresponding to the timestamp while playing the voice Describe the animation of the action.

In some embodiments, the action includes a plurality of sub-actions, and each sub-action matches one phoneme in the voice, and the time stamp includes a time stamp of each phoneme; Displaying the animation of the digital person performing the action at the time corresponding to the time stamp includes: determining the execution time of the sub-action matching each phoneme according to the time stamp of each phoneme; and according to the action Information, generating and displaying the animation of the digital person performing the sub-action matching the phoneme at the timestamp of each phoneme.

In some embodiments, the generating and displaying the animation of the digital person performing the corresponding action on the on-board display device in the cabin according to the action information includes: calling the action model library corresponding to the interaction information At least one frame of motion slices of the digital person; the motion slices of each frame of the at least one frame of digital human motion slices are sequentially displayed on the display device.

In some embodiments, the generating and displaying an animation of interactive information used to assist driving by a digital human displayed on a vehicle-mounted display device installed in the vehicle cabin according to the external environment perception information includes: The perception information of the outside environment of the vehicle is processed by a first predetermined task to obtain a first predetermined task processing result; in response to the first predetermined task processing result meeting a preset first safe driving warning condition, it is generated and displayed in the cabin The digital human displayed on the vehicle-mounted display device makes an animation of interactive information for the first safe driving warning.

In some embodiments, the method further includes: acquiring vehicle control information; the generating and displaying the digital person displayed on the vehicle-mounted display device arranged in the vehicle cabin according to the external environment perception information of the vehicle is used to assist The animation of driving interactive information includes: generating and displaying the interactive information used to assist driving by the digital human displayed on the on-board display device installed in the vehicle cabin according to the external environment perception information and the vehicle control information Animation.

In some embodiments, according to the external environment perception information and the vehicle control information, the generated and displayed digital person displayed on the vehicle-mounted display device arranged in the vehicle cabin makes interactive information for assisting driving The animation includes: performing a second predetermined task processing according to the external environment perception information and the vehicle control information to obtain a second predetermined task processing result; responding to the second task processing result meeting a preset second safe driving The pre-warning condition, generating and displaying the animation of the interactive information of the second safe driving pre-warning made by the digital person displayed on the on-board display device installed in the cabin.

In some embodiments, the method further includes: acquiring status analysis information of the driver in the vehicle; said generating and displaying the digital human status displayed on the vehicle-mounted display device installed in the vehicle cabin according to the perception information of the outside environment of the vehicle. The animation of interactive information used to assist driving includes: generating and displaying the digital human images displayed on the on-board display device installed in the cabin according to the external environment perception information and the in-vehicle driver status analysis information. An animation of interactive information used to assist driving.

In some embodiments, according to the perception information of the external environment and the status analysis information of the driver in the vehicle, the digital person displayed on the vehicle-mounted display device installed in the vehicle cabin is generated and displayed to assist driving The interactive information animation includes: performing a third predetermined task processing according to the external environment perception information and the in-vehicle driver status analysis information to obtain a third predetermined task processing result; responding to the third task processing result Meet the preset third safe driving warning condition, generate and display the interactive information animation of the digital person displayed on the on-board display device set in the cabin to make the third safe driving warning.

In some embodiments, the generating and displaying an animation of interactive information used to assist driving by a digital human displayed on a vehicle-mounted display device installed in the vehicle cabin according to the external environment perception information includes: Describes the perception information of the environment outside the vehicle, the acquired vehicle control information, and the acquired status analysis information of the driver in the vehicle, and generates and displays the digital human displayed on the on-board display device installed in the vehicle cabin to make interactions for assisting driving Information animation.

In some embodiments, the method further includes: acquiring map information including the environment outside the vehicle; and generating and displaying numbers displayed on a vehicle display device installed in the cabin according to the perception information of the outside environment. The animation of interactive information used by people to assist driving includes: generating first navigation information according to the perception information of the outside environment and the map information; generating and displaying the first navigation information according to the first navigation information. The digital human displayed on the on-board display device inside makes an animation of navigation interactive information.

In some embodiments, the method further includes: acquiring map information including the environment outside the vehicle and traffic control information; and generating and displaying the vehicle display device installed in the vehicle cabin according to the perception information of the environment outside the vehicle The animation of interactive information for assisting driving made by the digital human displayed on the above includes: generating second navigation information according to the external environment perception information, the map information, and the traffic control information; and according to the second The navigation information generates and displays the animation of the digital person making the navigation interaction information displayed on the on-board display device installed in the cabin.

According to a second aspect of the embodiments of the present disclosure, there is provided an interactive device for assisting driving, the device including: a first acquisition module for acquiring perception information of an outside environment; an interaction module for perception information of an outside environment , To generate and display the interactive information animation of the digital person displayed on the on-board display device installed in the cabin to assist driving.

According to a third aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when a processor executes the computer program, the processor is configured to execute the above-mentioned first The driving assistance interaction method based on a vehicle-mounted digital human according to any one of the embodiments of the aspect.

According to a fourth aspect of the embodiments of the present disclosure, there is provided an assisted driving interaction device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to call The stored executable instructions implement the driving assistance interaction method based on a vehicle-mounted digital human according to any one of the embodiments in the first aspect.

According to a fifth aspect of the embodiments of the present disclosure, a vehicle is provided, including: a camera for acquiring images in the vehicle and/or images outside the vehicle; a vehicle-mounted display device for displaying a digital person and the digital person An animation of interactive information for driving assistance; and the driving assistance interaction device of any one of the second aspect or the fourth aspect.

According to a sixth aspect of the embodiments of the present disclosure, there is provided a computer program product, including computer-readable code, and when the computer-readable code runs on a processor, the processor is configured to execute any one of the implementations in the first aspect The interactive method for assisted driving based on the digital human in the vehicle described in the example.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

In the embodiments of the present disclosure, based on the acquired perception information of the environment outside the vehicle, the digital human displayed on the vehicle-mounted display device installed in the cabin is generated and displayed to make an animation of interactive information used to assist driving, thereby enabling human-computer interaction The method is more in line with human interaction habits, and the interaction process is more natural, allowing people to feel the warmth of human-computer interaction, improving driving pleasure, comfort and sense of accompaniment, and helping to reduce driving safety risks.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments consistent with the disclosure, and are used together with the specification to explain the principle of the disclosure.

FIG. 1 is a flowchart of a method for assisted driving interaction based on a vehicle-mounted digital human shown in an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of an interaction scene between on-vehicle devices according to an embodiment of the present disclosure;

[Corrected according to Rule 26 30.12.2020]
FIG. 3 is a flowchart of an implementation manner of step 102 shown in an embodiment of the present disclosure;

FIG. 4 is a flowchart of an implementation manner of step 202 shown in an embodiment of the present disclosure;

FIG. 5 is a flowchart of an implementation manner of step 303 shown in an embodiment of the present disclosure;

FIG. 6 is a flowchart of an implementation manner of step 202 shown in an embodiment of the present disclosure;

FIG. 7 is a flowchart of an implementation manner of step 102 shown in an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a scene for generating and displaying an animation according to an embodiment of the present disclosure;

FIG. 9 is a flowchart of a driving assistance interaction method based on a vehicle-mounted digital human shown in an embodiment of the present disclosure;

FIG. 10 is a flowchart of an implementation manner of step 1021 shown in an embodiment of the present disclosure;

FIG. 11 is a flowchart of a method for assisted driving interaction based on a vehicle-mounted digital human shown in an embodiment of the present disclosure;

FIG. 12 is a flowchart of an implementation manner of step 1022 shown in an embodiment of the present disclosure;

FIG. 13 is a schematic diagram of a scene for generating and displaying an animation according to an embodiment of the present disclosure;

FIG. 14 is a flowchart of a driving assistance interaction method based on a vehicle-mounted digital human shown in an embodiment of the present disclosure;

FIG. 15 is a flowchart of step 1023 shown in an embodiment of the present disclosure;

FIG. 16 is a flowchart of an assisted driving interaction method based on a vehicle-mounted digital human shown in an embodiment of the present disclosure;

FIG. 17 is a flowchart of an implementation manner of step 1024 shown in an embodiment of the present disclosure;

FIG. 18 is a block diagram of an interactive device for assisting driving according to an embodiment of the present disclosure;

FIG. 19 is a schematic diagram of the hardware structure of a driving assistance interactive device shown in an embodiment of the present disclosure;

Fig. 20 is a schematic diagram showing a hardware structure of a vehicle according to an embodiment of the present disclosure.

detailed description

Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

The terminology used in the present disclosure is only for the purpose of describing specific embodiments, and is not intended to limit the present disclosure. The singular forms of "a", "said" and "the" used in the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and/or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to certainty."

The embodiments of the present disclosure provide an assisted driving interaction method based on a vehicle-mounted digital human, which can be used for drivable machinery and equipment, such as smart vehicles, smart cabins that simulate vehicle driving, and the like. As shown in Fig. 1, Fig. 1 shows a method for assisted driving interaction based on a vehicle-mounted digital human according to an exemplary embodiment, which includes the following steps:

In step 101, the perception information of the environment outside the vehicle is acquired.

In the embodiments of the present disclosure, the external environment perception information includes but is not limited to at least one of the following: road line detection information, road line attribute detection information, traffic light detection information, traffic sign detection information, drivable area detection information, obstacles Detection information. Wherein, road lines include but are not limited to at least one of the following: lane lines, stop lines, turning lines, U-turn lines, solid lines, dashed lines, single lines, and double lines.

In some embodiments, after real-time acquisition of the external environment perception information by the vehicle-mounted external driving assistance device, the external environment perception information may be sent to the on-board central control device. Among them, the auxiliary driving equipment outside the vehicle may include, but is not limited to, Advanced Driving Assistance System (ADAS).

In one example, the vehicle-mounted central control device may obtain the external environment perception information sent by the vehicle-mounted vehicle-mounted auxiliary driving device through an in-vehicle communication bus, such as a Controller Area Network (CAN) bus. In another example, the vehicle-mounted central control device may acquire the external environment perception information sent by the vehicle-mounted vehicle-mounted auxiliary driving device in a vehicle-to-vehicle communication mode.

In step 102, according to the external environment perception information, an animation of interactive information for assisting driving made by the digital person displayed on the on-board display device installed in the cabin is generated and displayed.

In the embodiments of the present disclosure, the digital person is a virtual image generated by software, which can be displayed on a vehicle-mounted display device in a cabin, where the vehicle-mounted display device may include, but is not limited to, a central control display or a vehicle-mounted tablet Equipment, etc. The interactive information includes but is not limited to one of the following: facial expression information, action information, and voice information.

The architecture between the vehicle central control device and other vehicle equipment is shown in Figure 2. The vehicle central control device can interact with the vehicle’s external auxiliary driving equipment, vehicle camera, vehicle display device and other vehicle equipment (such as air conditioners, radios, car windows). Etc.) to communicate, of course, the in-vehicle central control device can also be connected to other in-vehicle devices, such as mobile phones carried by people in the vehicle, through the network. The vehicle-mounted central control device can also obtain at least one of the voice input and the action input of the person entering the cabin through the vehicle-mounted audio collection device and the vehicle-mounted camera, and subsequently can control the digital person to perform corresponding interactive feedback.

In the embodiment of the present disclosure, after the vehicle-mounted central control device obtains the external environment perception information sent by the external auxiliary driving device, it can generate the digital human set displayed on the vehicle-mounted display device according to the external environment perception information. An animation of interactive information used to assist driving, and the animation is displayed by the on-board display device.

In the above-mentioned embodiment, it is possible to generate and display an animation of interactive information used to assist driving by the digital human displayed on the vehicle-mounted display device installed in the vehicle cabin according to the acquired perception information of the outside environment of the vehicle, thereby enabling human-computer interaction The method is more in line with human interaction habits, and the interaction process is more natural, allowing people to feel the warmth of human-computer interaction, improving driving pleasure, comfort and sense of accompaniment, and helping to reduce driving safety risks.

In some embodiments, such as shown in FIG. 3, step 102 may include:

In step 201, the action information for driving assistance that matches the perception information of the outside environment is determined.

In one example, it is possible to determine the degree of matching between each type of action information used to assist driving and the perception information of the environment outside the vehicle, and determine the action information with the highest degree of matching as the action information that matches the perception information of the outside environment.

In the case that the perception information of the outside environment includes multiple types of information, the degree of matching between the same action information and the perception information of the outside environment can be determined respectively, and then the degree of matching between the action information and the perception information corresponding to the various information can be determined. The degree of matching of the perception information of the environment outside the vehicle. For example, the matching degree corresponding to various information is weighted and averaged.

In another example, the mapping relationship between different external environment perception information and the matching action information can be established in advance, and the matching action information can be determined according to the mapping relationship.

In step 202, an animation of a digital person performing a corresponding action is generated and displayed on the vehicle-mounted display device in the cabin according to the action information.

In the above-mentioned embodiment, when the interactive information includes motion information, combining the motion information matched with the perception information of the outside environment allows the digital human to make animations of corresponding actions that assist driving, reducing the safety risk of driving, and also making the digital human Interaction with people is more natural, making people feel the warmth of human-computer interaction.

In some embodiments, in addition to generating and displaying corresponding animations in combination with action information, it is also possible to generate and display corresponding animations in combination with voice information. For example, as shown in FIG. 4, step 202 may include:

In step 301, the voice information that matches the perception information of the outside environment of the vehicle is determined.

In one example, a trained deep learning neural network may be used to determine the voice information that matches the perception information of the outside environment.

In another example, the mapping relationship between different perception information of the outside environment and the matching voice information may be established in advance, and the matching voice information may be determined according to the mapping relationship.

In step 302, a corresponding voice is acquired according to the voice information, and the voice includes a timestamp.

In the embodiment of the present disclosure, the voice can be pulled from the voice database according to the voice information, and the pulled voice carries the timestamp of the voice, which is used to make the digital person in the animation perform the corresponding action time and time. Said voice synchronization. The timbre of the pulled voice may be a preset timbre, or a timbre set according to the user's preference. For example, the preset timbre is a child's timbre, and the voice corresponding to the child's timbre is pulled when the voice is pulled.

In step 303, while the voice is being played, an animation of the digital person performing the action at the time corresponding to the time stamp is generated and displayed according to the action information.

In the embodiments of the present disclosure, it is possible to generate and display an animation for the digital person to perform a corresponding action at the time corresponding to different timestamps in the voice while the voice is being played.

In the above-mentioned embodiment, combining the action information and voice information that matches the perception information of the outside environment of the vehicle, the digital person can not only play the voice, but also allow the digital person to perform corresponding actions to assist driving at the time corresponding to the timestamp in the voice. While reducing the safety risk of driving, the interaction between digital humans and humans is richer and more natural, making people feel the warmth of human-computer interaction.

In some embodiments, a piece of speech often includes multiple phonemes. A phoneme is the smallest phonetic unit divided according to the natural attributes of the speech. It is analyzed according to the pronunciation actions in the syllable, and one pronunciation action constitutes a phoneme. For example, "hello" includes the two phonemes of "you" and "good". In the case where multiple phonemes are included in the speech, the timestamp may include the timestamp of each phoneme. An action generally includes multiple sub-actions. For example, a beckoning action may include a sub-action of swinging an arm to the left and a sub-action of swinging an arm to the right. In order to make the displayed digital person more vivid, each sub-action can be matched with a phoneme in the voice.

For example, as shown in FIG. 5, step 303 may include:

In step 401, according to the timestamp of each phoneme, the execution time of the sub-action matching each phoneme is determined.

In step 402, according to the action information, an animation in which the digital person performs a sub-action matching the phoneme at the timestamp of each phoneme is generated and displayed.

For example, when the phoneme "you" is played, the mouth movement that matches "you" is displayed, and the beckoning movement of the digital person's arm swinging to the left is displayed. Matching mouth movements, and show the beckoning movement of the digital human arm swinging to the right.

In the above embodiment, combining the action information, voice information and phonemes in the voice that matches the perception information of the outside environment of the car, the digital person is allowed to perform sub-actions matching the phoneme at the time stamp of each phoneme, thereby reducing the safety risk of driving. At the same time, it makes the interaction between digital people and humans richer and more natural, and makes people feel the warmth of human-computer interaction.

In some embodiments, such as shown in FIG. 6, step 202 may include:

In step 501, at least one frame of motion slice of the digital person corresponding to the motion information is called from the motion model library.

The action corresponding to the action information can be called from the action model library. Specifically, at least one frame of motion slices of the digital person corresponding to the motion information can be called from the motion model library.

In step 502, the motion slices of each of the at least one frame of digital human motion slices are sequentially displayed on the display device.

In the embodiments of the present disclosure, at least one of the digital human's body movements, facial expression movements, mouth movements, eye movements, etc. corresponding to different action slices is different. By calling and sequentially displaying the corresponding action slices, Therefore, an animation of the digital person performing the corresponding action can be displayed on the vehicle-mounted display device.

In the foregoing embodiment, a manner of sequentially displaying action slices called from the action model library can be used to display corresponding animations, and the action model library can be updated as needed, so that the interaction between digital humans and humans is richer and more natural.

In some embodiments, such as shown in FIG. 7, step 102 may include:

In step 601, a first predetermined task processing is performed according to the perception information of the outside environment of the vehicle, and a processing result of the first predetermined task is obtained.

In the embodiment of the present disclosure, after the perception information of the environment outside the vehicle is acquired, the first predetermined task processing may be performed based on the perception information of the environment outside the vehicle, so as to obtain the corresponding processing result. Among them, the first predetermined task processing includes at least one of the following: lane departure detection processing, speed limit detection processing, traffic light violation detection processing, traffic sign violation detection processing, drivable area detection processing, collision detection processing, and vehicle distance detection processing .

The lane departure detection process indicates whether the vehicle has deviated from the lane corresponding to the current driving route. The speed limit detection process indicates whether the current speed of the vehicle exceeds the maximum speed indicated by the traffic signs appearing on the current route. Violation of the traffic light indication detection process indicates whether the vehicle has violated the traffic light indication. The drivable area detection processing indicates that the vehicle does not appear in the drivable area. The collision detection processing includes but is not limited to at least one of the following: forward collision detection processing, pedestrian collision detection processing, and urban forward collision detection processing. Forward collision detection processing is used to detect whether the distance between the vehicle and other vehicles is likely to collide, and the pedestrian collision detection processing is used to detect whether the distance between the vehicle and the pedestrian is likely to collide, and the urban forward collision detection processing It is used to indicate whether the distance between the vehicle and other vehicles is likely to collide in a congested environment in the city. The distance detection process is used to indicate the distance between the vehicle and other vehicles.

In step 602, in response to the first predetermined task processing result meeting the preset first safe driving warning condition, the digital person displayed on the on-board display device set in the cabin is generated and displayed to make the first safe driving warning Animation of interactive information.

In the embodiment of the present disclosure, the first predetermined task processing result that satisfies the preset first safe driving warning condition includes but is not limited to at least one of the following: lane departure detection processing indicates lane departure, speed limit detection processing indicates vehicle speeding, violation The traffic light indicator detection process indicates that the vehicle has violated the traffic light instruction, the traffic sign detection process indicates that the vehicle has violated the content corresponding to the traffic sign, the drivable area detection process indicates that the vehicle is not in the drivable area, the collision detection process indicates that the vehicle is about to collide, The distance detection process indicates that the distance between the vehicle and other vehicles is less than the threshold.

For example, if the distance detection process indicates that the distance between the vehicle and other vehicles is less than the threshold, then an animation of the interactive information of the first safe driving warning set by the digital person displayed on the vehicle display device can be generated and displayed. The content includes, but is not limited to: playing the voice of "Be careful, you are too close to other vehicles", the digital person makes a "careful" expression, and the digital person makes actions such as "shaking his head and waving his hand", as shown in Figure 8.

In the above-mentioned embodiment, the digital human can make the animation of the first safe driving warning based on the perception information of the outside environment of the car. While reducing the safety risk of driving, the way of human-computer interaction is more in line with human interaction habits and interaction process. It is more natural and makes people feel the warmth of human-computer interaction, which enhances driving pleasure, comfort and sense of accompany.

In some embodiments, such as shown in FIG. 9, the foregoing method may further include:

In step 103, vehicle control information is acquired.

In the embodiment of the present disclosure, the execution order of step 101 and step 103 is not limited. The vehicle control information includes but is not limited to at least one of the following: vehicle turn signal control information, steering wheel control information, accelerator control information, brake control information, and clutch control information. The driver's control information on the turn signal, steering wheel, accelerator, brake, clutch, etc. can be obtained through the on-board central control device.

Correspondingly, step 102 may include step 1021:

Step 1021, according to the external environment perception information and the vehicle control information, generate and display an animation of interactive information for assisting driving made by the digital person displayed on the on-board display device installed in the vehicle cabin.

In the embodiments of the present disclosure, the external environment perception information and the acquired vehicle control information can be combined to jointly generate and display the interactive information animation of the digital human displayed on the vehicle display device for assisting driving.

In some embodiments, such as shown in FIG. 10, step 1021 may include:

In step 701, a second predetermined task is processed according to the external environment perception information and the vehicle control information to obtain a second predetermined task processing result.

In the embodiment of the present disclosure, the second predetermined task processing includes but is not limited to at least one of the following: vehicle turn signal control matching detection processing, steering wheel control matching detection processing, throttle control matching detection processing, brake control matching detection processing, clutch control matching Detection and processing.

The second predetermined task processing result includes whether the acquired vehicle control information matches the vehicle control information corresponding to the external environment perception information.

For example, the external environment perception information includes the traffic light being a red light. At this time, the vehicle control information that matches the external environment perception information should be brake control information. If the actual vehicle control information obtained is brake control information, it is determined The processing result of the second predetermined task is brake control matching, otherwise it is brake control mismatch.

In step 702, in response to the second task processing result meeting the preset second safe driving warning condition, the digital person displayed on the on-board display device set in the cabin is generated and displayed to give the second safe driving warning. Animation of interactive information.

In the embodiment of the present disclosure, if the second predetermined task processing result is that the acquired vehicle control information does not match the perception information of the outside environment, it is determined that the preset second safe driving warning condition is satisfied, and accordingly, it can be generated and displayed The digital person displayed on the on-board display device installed in the cabin makes an animation of the interactive information of the second safe driving warning.

For example, the processing result of the second predetermined task is that the brake control does not match, and the generated animation content includes but is not limited to at least one of the following: playing the voice "It's time to brake", making the expression "this is not right", and making "brake" Actions. In the above-mentioned embodiment, the perception information of the environment outside the vehicle can be combined with the acquired vehicle control information to generate and display the interactive information of the digital human displayed on the vehicle-mounted display device installed in the vehicle cabin to make the second safe driving warning. Animation. In addition to reducing the safety risks of driving, it can also make safe driving warnings more accurate, and enhance driving pleasure, comfort and sense of accompaniment.

In some embodiments, such as shown in FIG. 11, the foregoing method may further include:

In step 104, the status analysis information of the driver in the vehicle is obtained.

In the embodiment of the present disclosure, the execution order of step 104 and step 101 is also not limited. The status analysis information of the driver in the vehicle includes but is not limited to at least one of the following: human body status analysis information, emotional status analysis information, fatigue status analysis information, distraction status analysis information, dangerous action analysis information, seat belt wearing analysis information, driver Off-job analysis information.

Correspondingly, step 102 may include step 1022:

Step 1022: Based on the external environment perception information and the driver status analysis information in the vehicle, generate and display an animation of interactive information used to assist driving by the digital person displayed on the on-board display device installed in the vehicle cabin .

In the embodiments of the present disclosure, the external environment perception information and the acquired status analysis information of the driver in the vehicle can be combined to jointly generate and display the interactive information animation of the digital human displayed on the vehicle display device for assisting driving.

In some embodiments, such as shown in FIG. 12, step 1022 may include:

In step 801, a third predetermined task processing is performed according to the external environment perception information and the in-vehicle driver status analysis information to obtain a third predetermined task processing result.

In the embodiment of the present disclosure, the third task processing includes but is not limited to at least one of the following: human body state detection processing, emotional state detection processing, fatigue state detection processing, distraction state detection processing, dangerous action detection processing, seat belt wearing detection processing Processing, driver's departure detection processing.

In step 802, in response to the third task processing result satisfying the preset third safe driving warning condition, the digital person displayed on the on-board display device installed in the cabin is generated and displayed to make the third safe driving warning. Animation of interactive information.

In the embodiment of the present disclosure, firstly, the status analysis information of the driver in the vehicle satisfies but is not limited to at least one of the following conditions: the human body state detection processing indicates that the sitting posture of the human body is not suitable for driving, and the emotional state detection result indicates that the person in the vehicle is negative The detection result of emotion and fatigue state indicates that the person in the car is in a state of fatigue, and the detection result of the distracted state indicates that the person in the car is distracted, and the detection result of dangerous actions indicates that the person in the car has dangerous actions and seat belt wearing detection processing The result indicates that the driver is not wearing a seat belt, and the driver's departure detection processing indicates that the driver is not on duty. Further, the first predetermined task processing is performed according to the external environment perception information, and the processing result of the first predetermined task meets the preset first safe driving warning condition. At this time, it is determined that the processing result of the third task meets the preset third Warning conditions for safe driving. Correspondingly, it is possible to generate and display the animation of the interactive information of the digital person making the third safe driving warning displayed on the on-board display device installed in the vehicle cabin.

For example, if the processing result of the third predetermined task includes that the people in the vehicle are in a state of fatigue and a lane departure occurs, then an animation of the interactive information of the digital person making the third safe driving warning set on the vehicle display device can be generated and displayed. The content of the animation includes, but is not limited to: playing the voice of "The lane has deviated, is it because you are tired, take a break and set off", the digital person makes a "don't be like this" expression, and the digital person is playing "The lane has deviated" Make a hand waving action when the voice is played, and make a rest action when the voice "Take a break and then start" is played, for example, as shown in Figure 13.

In the above embodiment, the perception information of the environment outside the vehicle can be combined with the acquired analysis information of the driver's status in the vehicle to generate and display the digital person displayed on the on-board display device installed in the cabin to make the third safe driving warning. Animation of interactive information. When the driver in the car is in a state that requires driving assistance, the corresponding animation is generated and displayed based on the perception information of the external environment. While reducing the safety risk of driving, it can also make the safe driving warning more accurate and avoid unnecessary driving assistance reminders. , To enhance driving pleasure, comfort and sense of companionship.

In some embodiments, the perception information of the environment outside the vehicle may also be combined with the acquired vehicle control information and the acquired status analysis information of the driver in the vehicle, so as to generate a corresponding animation.

In the embodiment of the present disclosure, when the driver status analysis information in the vehicle satisfies the above-mentioned conditions, it is possible to generate and display the digital human status displayed on the vehicle-mounted display device installed in the vehicle cabin according to steps 701 to 702. Animation of interactive information used to assist driving.

In the above-mentioned embodiment, the perception information of the outside environment of the vehicle can be combined with various acquired information to jointly determine the final generated and displayed animation content. While reducing the safety risk of driving, it can also make the safe driving warning more accurate. Avoid unnecessary reminders of assisted driving to enhance driving pleasure, comfort and sense of accompany.

In some embodiments, such as shown in FIG. 14, the foregoing method may further include:

In step 105, map information including the environment outside the vehicle is acquired.

In the embodiment of the present disclosure, the execution order of step 105 and step 101 is not limited. The map information of the environment outside the vehicle can be obtained from the high-precision map or navigation application through the vehicle network or other communication methods. The map information includes but is not limited to at least one of the following: current location information of the vehicle, at least one planned driving route information, traffic sign information in front of the vehicle, speed limit information in front of the vehicle, obstacle information in front of the vehicle, and lane ahead information.

Correspondingly, step 102 may include step 1023: step 1023, according to the external environment perception information and the map information, generate and display the digital person displayed on the on-board display device installed in the cabin to assist driving Animation of interactive information.

For example, as shown in Figure 15, step 1023 may include:

In step 901, first navigation information is generated according to the external environment perception information and the map information.

In step 902, according to the first navigation information, an animation of the digital person making navigation interaction information displayed on the on-board display device installed in the cabin is generated and displayed.

For example, the first navigation information includes a left turn 50 meters ahead and the need to enter a left-turn lane. The animation content may include but is not limited to at least one of the following: playing a voice saying "turn left 50 meters ahead and need to enter the left-turn lane", The digital person makes an expression of "concentration", and the digital person makes an action of "waves his left hand".

In the above-mentioned embodiment, the digital person displayed on the on-board display device can make an animation of the navigation interaction information based on the perception information of the external environment and the map information, and the navigation process is completed by the digital person in the anthropomorphic image. The navigation process is more vivid, making people feel the warmth of human-computer interaction, and enhancing driving pleasure, comfort and sense of accompany.

In some embodiments, such as shown in FIG. 16, the foregoing method may further include:

In step 106, map information including the environment outside the vehicle and traffic control information are acquired.

In the embodiment of the present disclosure, similarly, the execution order of step 106 and step 101 is not limited. In addition to obtaining map information including the environment outside the car, you can also obtain traffic control information. Among them, traffic control information includes but is limited to: real-time traffic control information, long-term or short-term traffic control information in a certain place. Real-time traffic control information can be obtained through real-time road condition detection, etc., and long-term or short-term traffic control information in a certain place can be obtained through interaction with the Internet.

Correspondingly, step 102 may include step 1024: step 1024, according to the external environment perception information, the map information, and the traffic control information, generate and display the digital human figure displayed on the vehicle-mounted display device installed in the vehicle cabin. Animation of interactive information used to assist driving.

For example, as shown in FIG. 17, step 1024 may include:

In step 1001, second navigation information is generated according to the external environment perception information, the map information, and the traffic control information.

In step 1002, according to the second navigation information, an animation of the digital person making navigation interaction information displayed on the on-board display device installed in the cabin is generated and displayed.

For example, the second navigation information includes temporary traffic control caused by road construction ahead and 500 meters ahead, and the animation content may include but not limited to at least one of the following: playing "go straight ahead" and "road construction ahead 500 meters ahead" Temporary traffic control will slow down traffic. Don’t be impatient” voices, digital humans make “attention” expressions, and digital humans make “wave hands” while playing the “don’t worry” voice.

In the foregoing embodiment, the digital person displayed on the vehicle display device can make an animation of navigation interaction information based on the perception information of the external environment, the map information, and the traffic control information, so that the navigation process is more accurate and allows people Feel the warmth of human-computer interaction, enhance driving pleasure, comfort and sense of accompany.

As shown in FIG. 18, FIG. 18 is a block diagram of an assisted driving interaction device according to an exemplary embodiment of the present disclosure. The device includes: a first acquisition module 1110 for acquiring perception information of the environment outside the vehicle; an interaction module 1120 for According to the perception information of the outside environment of the vehicle, an animation of interactive information for assisting driving made by a digital person displayed on an on-board display device installed in the vehicle cabin is generated and displayed.

In some embodiments, the interaction module includes: a first determination sub-module, configured to determine action information matching the perception information of the outside environment for driving assistance; a first interaction sub-module, configured according to the The action information is generated and displayed on the vehicle-mounted display device in the cabin of the digital person to perform the corresponding action animation.

In some embodiments, the first interaction submodule includes: a first determining unit, configured to determine voice information that matches the perception information of the outside environment; an acquiring unit, configured to obtain corresponding voice information according to the voice information Voice, the voice includes a timestamp; the first interaction unit is used to generate and display the digital person performing the action at the time corresponding to the timestamp while playing the voice, according to the action information Animation.

In some embodiments, the action includes a plurality of sub-actions, each sub-action matches one phoneme in the voice, and the time stamp includes the time stamp of each phoneme; the first interaction submodule includes: The second determining unit is used to determine the execution time of the sub-action matching each phoneme according to the time stamp of each phoneme; the second interaction unit is used to generate and display the number according to the action information The person executes the animation of the sub-action matching the phoneme at the timestamp of each phoneme.

In some embodiments, the first interaction sub-module includes: a calling unit for calling at least one frame of motion slices of a digital person corresponding to the interaction information from an action model library; a display unit for displaying the Each of the at least one frame of motion slices of the digital person is sequentially displayed on the display device.

In some embodiments, the interaction module includes: a first processing sub-module, configured to process a first predetermined task according to the perception information of the outside environment to obtain the processing result of the first predetermined task; a second interactive sub-module, In response to the first task processing result satisfying the preset first safe driving warning condition, an animation of interactive information of the first safe driving warning made by the digital person displayed on the on-board display device installed in the cabin is generated and displayed .

In some embodiments, the device further includes: a second acquisition module, configured to acquire vehicle control information; and the interaction module includes: a third interaction sub-module, configured according to the perception information of the outside environment and the vehicle Control information, generate and display an animation of interactive information for assisting driving made by the digital person displayed on the on-board display device installed in the cabin.

In some embodiments, the third interaction submodule includes: a first processing unit, configured to perform a second predetermined task processing according to the external environment perception information and the vehicle control information to obtain a second predetermined task processing result ; The third interaction unit, in response to the second task processing results to meet the preset second safe driving early warning conditions, generate and display the digital person displayed on the vehicle-mounted display device set in the cabin to make the second safe Animation of interactive information for driving warning.

In some embodiments, the device further includes: a third acquisition module, configured to acquire status analysis information of the driver in the vehicle; the interaction module includes: a fourth interaction sub-module, configured to perceive information based on the environment outside the vehicle Analyze the information with the state of the driver in the vehicle, and generate and display an animation of interactive information for assisting driving made by the digital person displayed on the on-board display device installed in the vehicle cabin.

In some embodiments, the fourth interaction submodule includes: a second processing unit, configured to perform a third predetermined task processing according to the external environment perception information and the in-vehicle driver status analysis information to obtain a third The predetermined task processing result; the fourth interaction unit, in response to the third task processing result meets the preset third safe driving warning condition, generates and displays the digital person displayed on the vehicle-mounted display device installed in the cabin An animation of the interactive information of the third safe driving warning is presented.

In some embodiments, the interaction module includes: a fifth interaction sub-module for generating and displaying information based on the external environment perception information, the acquired vehicle control information, and the acquired status analysis information of the in-vehicle driver The digital human displayed on the on-board display device installed in the cabin makes an animation of interactive information used to assist driving.

In some embodiments, the device further includes: a fourth obtaining module, configured to obtain map information including the environment outside the vehicle; the interaction module includes: a first generation sub-module, configured according to the environment outside the vehicle Perceived information and the map information to generate first navigation information; the fifth interaction sub-module is used to generate and display the digital person displayed on the on-board display device in the cabin according to the first navigation information to make navigation Animation of interactive information.

In some embodiments, the device further includes: a fifth acquisition module for acquiring map information and traffic control information including the environment outside the vehicle; the interaction module includes: a second generation sub-module for The external environment perception information, the map information, and the traffic control information generate second navigation information; the sixth interaction sub-module is used to generate and display the vehicle mounted in the cabin according to the second navigation information The digital person displayed on the display device makes an animation of navigation and interactive information.

As for the device embodiment, since it basically corresponds to the method embodiment, the relevant part can refer to the part of the description of the method embodiment. The device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place. , Or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the present disclosure. Those of ordinary skill in the art can understand and implement without creative work.

The embodiment of the present disclosure also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the processor reads the computer program, the processor is used to execute the An interactive method for assisted driving based on on-board digital humans.

In some embodiments, the embodiments of the present disclosure provide a computer program product, including computer-readable code. When the computer-readable code is run on a device, the processor in the device executes to achieve the same as that provided in any of the above embodiments. The instruction of the interactive method of assisted driving based on the on-board digital human.

The computer program product can be specifically implemented by hardware, software, or a combination thereof. In one embodiment, the computer program product is specifically embodied as a computer storage medium. In another embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and so on.

The embodiment of the present disclosure also provides a driving assistance interaction device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to call the executable instructions stored in the memory to implement the foregoing The driving assistance interaction method based on a vehicle-mounted digital human described in any one of the embodiments.

FIG. 19 is a schematic diagram of the hardware structure of a driving assistance interaction device provided by an embodiment of the application. The driving assistance interactive device 1210 includes a processor 1211, and may also include an input device 1212, an output device 1213, and a memory 1214. The input device 1212, the output device 1213, the memory 1214, and the processor 1211 are connected to each other through a bus. The memory 1214 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable Read-only memory (compact disc read-only memory, CD-ROM), the memory 1214 is used to store related instructions and data.

The input device 1212 is used to input data and/or signals, and the output device 1213 is used to output data and/or signals. The output device 1213 and the input device 1212 may be independent devices or a whole device. In the embodiment of the present disclosure, the input device 1212 includes a camera 01 and other input devices 02, and the other input devices 02 include but are not limited to audio capture devices. The output device 1213 includes a display device 03 and other output devices 04. The other output devices 04 may include but are not limited to audio output devices. The digital human is displayed through the display device 03, and the animation of the digital human performing the corresponding steering action is also through the display device. 03 to show.

The processor 1211 may include one or more processors, such as one or more central processing units (CPU). In the case where the processor 1211 is a CPU, the CPU may be a single-core CPU or It is a multi-core CPU. The memory 1214 is used to store program codes and data of the network device. The processor 1211 is configured to call the program code and data in the memory 1214 to execute the steps in the foregoing method embodiment. For details, please refer to the description in the method embodiment, which will not be repeated here.

It is understandable that Fig. 19 only shows a simplified design of a driving assistance interactive device. In practical applications, the driving assistance interaction device may also include other necessary components, including but not limited to any number of input/output devices, processors, controllers, memories, etc., and all of them can implement the driving assistance in the embodiments of the present application. All interactive devices are within the protection scope of this application.

In some embodiments, the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. For specific implementation, refer to the description of the above method embodiments. For brevity, here No longer.

The embodiment of the present disclosure also provides a vehicle, including: a camera for acquiring images in the vehicle and/or images outside the vehicle; a vehicle-mounted display device for displaying a digital person and the digital person's interaction for assisting driving Information animation; and the driving assistance interactive device described in any of the above embodiments.

For example, as shown in FIG. 20, an embodiment of the present disclosure further provides a vehicle 1300, which includes a vehicle-mounted camera 1310, a vehicle-mounted display device 1320, and a driving assistance interaction device 1330.

Among them, the vehicle-mounted camera 1310 can be used to obtain images inside and/or outside the vehicle, and the vehicle-mounted display device 1320 can be used to display the digital person and the animation of the digital person’s interactive information for assisting driving, assisting the driving interactive device 1330 can use the driving assistance interactive device described in any of the above embodiments to generate and display the digital person displayed on the vehicle-mounted display device set in the cabin according to the acquired perception information of the external environment to assist driving. Animation of interactive information.

Those skilled in the art will easily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field that are not disclosed in the present disclosure. . The description and the embodiments are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are pointed out by the following claims.

The above are only some embodiments of the present disclosure, and are not used to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection of the present disclosure. Within the range.

Claims (18)

1. An assisted driving interaction method based on a vehicle-mounted digital human, including:

   Obtain the perception information of the environment outside the vehicle;

   According to the perception information of the outside environment of the vehicle, an animation of interactive information for assisting driving made by the digital person displayed on the on-board display device installed in the vehicle cabin is generated and displayed.
2. The method according to claim 1, wherein said generating and displaying an animation of interactive information for assisting driving made by a digital person displayed on a vehicle-mounted display device installed in the vehicle cabin based on the perception information of the outside environment of the vehicle ,include:

   Determining the action information used to assist driving that matches the perception information of the outside environment of the vehicle;

   The animation of the digital person performing the corresponding action is generated and displayed on the vehicle-mounted display device in the cabin according to the action information.
3. The method according to claim 2, wherein the generating and displaying an animation of a digital person performing a corresponding action on a vehicle-mounted display device in the cabin according to the action information comprises:

   Determining voice information that matches the perception information of the outside environment of the vehicle;

   Acquiring a corresponding voice according to the voice information, where the voice includes a timestamp;

   While playing the voice, an animation of the digital person performing the action at the time corresponding to the time stamp is generated and displayed according to the action information.
4. The method according to claim 3, wherein the action includes a plurality of sub-actions, each sub-action matches one phoneme in the speech, and the time stamp includes the time stamp of each phoneme;

   The generating and displaying the animation of the digital person performing the action at the time corresponding to the time stamp according to the action information includes:

   Determine the execution time of the sub-action matching each phoneme according to the time stamp of each phoneme;

   According to the action information, an animation in which the digital person performs a sub-action matching the phoneme at the timestamp of each phoneme is generated and displayed.
5. The method according to any one of claims 2-4, wherein the generating and displaying an animation of a digital person performing a corresponding action on a vehicle-mounted display device in the cabin according to the action information comprises:

   Calling at least one frame of motion slices of the digital person corresponding to the motion information from the motion model library;

   The motion slices of each frame of the digital human in the at least one frame of motion slices of the digital human are sequentially displayed on the display device.
6. The method according to any one of claims 1-5, wherein said generating and displaying the digital person displayed on the vehicle-mounted display device installed in the vehicle cabin according to the perception information of the external environment is used to assist driving Animation of interactive information, including:

   Performing the first predetermined task processing according to the perception information of the outside environment of the vehicle to obtain the first predetermined task processing result;

   In response to the first predetermined task processing result satisfying the preset first safe driving warning condition, an animation of interactive information of the first safe driving warning made by the digital person displayed on the on-board display device installed in the cabin is generated and displayed .
7. The method according to any one of claims 1-6, wherein the method further comprises:

   Obtain vehicle control information;

   The generating and displaying the interactive information animation used to assist driving by the digital person displayed on the on-board display device installed in the vehicle cabin according to the perception information of the outside environment of the vehicle includes:

   According to the external environment perception information and the vehicle control information, an animation of interactive information for assisting driving made by a digital person displayed on a vehicle-mounted display device installed in the vehicle cabin is generated and displayed.
8. The method according to claim 7, wherein said generating and displaying a digital person displayed on an on-board display device installed in the cabin according to the perception information of the external environment and the vehicle control information is used to assist Animation of interactive driving information, including:

   Performing a second predetermined task processing according to the external environment perception information and the vehicle control information to obtain a second predetermined task processing result;

   In response to the second task processing result satisfying the preset second safe driving warning condition, an animation of interactive information of the digital person making the second safe driving warning displayed on the on-board display device arranged in the cabin is generated and displayed.
9. 8. The method according to any one of claims 1-8, wherein the method further comprises: obtaining status analysis information of the driver in the vehicle;

   The generating and displaying the interactive information animation used to assist driving by the digital person displayed on the on-board display device installed in the vehicle cabin according to the perception information of the outside environment of the vehicle includes:

   According to the external environment perception information and the in-vehicle driver status analysis information, an animation of interactive information for assisting driving made by a digital person displayed on an on-board display device installed in the vehicle cabin is generated and displayed.
10. The method according to claim 9, wherein said generating and displaying a digital person displayed on a vehicle-mounted display device installed in the vehicle cabin based on the perception information of the external environment and the status analysis information of the driver in the vehicle Animation of interactive information used to assist driving, including:

    Performing a third predetermined task processing according to the external environment perception information and the in-vehicle driver status analysis information to obtain a third predetermined task processing result;

    In response to the third task processing result meeting the preset third safe driving early warning condition, an animation of interactive information of the third safe driving early warning made by the digital person displayed on the on-board display device installed in the cabin is generated and displayed.
11. The method according to any one of claims 1-10, wherein the generating and displaying the digital person displayed on the vehicle-mounted display device arranged in the cabin according to the perception information of the external environment is used to assist driving Animation of interactive information, including:

    According to the external environment perception information, the acquired vehicle control information, and the acquired status analysis information of the driver in the vehicle, the digital person displayed on the on-board display device installed in the cabin is generated and displayed to assist driving Animation of interactive information.
12. The method according to any one of claims 1-11, wherein the method further comprises: obtaining map information including the environment outside the vehicle;

    The generating and displaying the interactive information animation used to assist driving by the digital person displayed on the on-board display device installed in the vehicle cabin according to the perception information of the outside environment of the vehicle includes:

    Generating first navigation information according to the perception information of the outside environment of the vehicle and the map information;

    According to the first navigation information, an animation of the digital person making navigation interaction information displayed on the on-board display device installed in the cabin is generated and displayed.
13. The method according to any one of claims 1-12, wherein the method further comprises: obtaining map information and traffic control information including the environment outside the vehicle;

    The generating and displaying the interactive information animation used to assist driving by the digital person displayed on the on-board display device installed in the vehicle cabin according to the perception information of the outside environment of the vehicle includes:

    Generating second navigation information according to the external environment perception information, the map information, and the traffic control information;

    According to the second navigation information, an animation of the digital person making navigation interaction information displayed on the on-board display device installed in the cabin is generated and displayed.
14. A driving assistance interactive device, including:

    The first acquisition module is used to acquire the perception information of the environment outside the vehicle;

    The interaction module is used to generate and display the interactive information animation of the digital human displayed on the vehicle-mounted display device installed in the cabin to assist driving according to the perception information of the external environment.
15. A computer-readable storage medium, the computer-readable storage medium stores a computer program, and when a processor executes the computer program, the processor is used to execute the vehicle-based digital Human-assisted driving interaction method.
16. A driving assistance interactive device, including:

    processor;

    A memory for storing executable instructions of the processor;

    Wherein, the processor is configured to implement the assisted driving interaction method based on a vehicle-mounted digital human according to any one of claims 1-13 when calling the executable instructions stored in the memory.
17. A vehicle including:

    Camera, used to obtain images inside the car and/or outside the car;

    A vehicle-mounted display device for displaying a digital person and an animation of interactive information made by the digital person for assisting driving; and

    The driving assistance interactive device according to claim 14 or 16.
18. A computer program product, comprising computer readable code, and when the computer readable code runs on a processor, the processor is used to execute the vehicle-mounted digital human-based assistant according to any one of claims 1-13 Driving interactive method.

Images