JP7204804B2 - Smart rearview mirror interaction method, device, electronic device and storage medium - Google Patents

Description

TECHNICAL FIELD The present application relates to the field of artificial intelligence, and more particularly to smart rearview mirror interaction methods, devices, electronic devices and storage media.

With the development of vehicle intelligence, the number of in-vehicle electronic devices has gradually increased. The rearview mirror also evolved from the original rearview function to a smart rearview mirror with auxiliary reversing function. To realize many functions such as map navigation.

In the prior art, the user can speak a wake word to wake up the smart rearview mirror, and the smart rearview mirror displays a voice assistant interface or emits a voice after recognizing the wake word spoken by the user. The manner in which it is played reminds the user to proceed to the next voice interaction. During the voice interaction process between the user and the smart rearview mirror, the smart rearview mirror can recognize the semantics of the user's voice and perform actions such as music playback or map navigation.

However, in some application scenarios, for example, when the vehicle is in a noisy environment, the user's voice is covered by the surrounding sounds, the smart rearview mirror cannot recognize the user's voice, and the accurate response operation is difficult. become unable.

The present application provides an interaction method, device, electronic device and storage medium for a smart rearview mirror, which can interact with the user through a portable interaction method, increase the interaction success rate, and further improve the user experience. .

In a first aspect, the present application provides a method for interacting with a smart rearview mirror, determining a current usage state of the smart rearview mirror after waking up the smart rearview mirror; and controlling the smart rearview mirror to interact with the user based on the usage status of the smart rearview mirror.

In this embodiment, on the premise of increasing the interaction mode with the user, the conventional voice interaction can also solve the problem that the user's voice cannot be recognized in certain scenes, based on the current use state of the smart rearview mirror. , use a portable interaction method to interact with the user, increase the interaction success rate, and further improve the user experience.

In a second aspect, the present application provides a smart rearview mirror interaction device, wherein after waking up the smart rearview mirror, a first processing module for determining the current usage status of said smart rearview mirror; a second processing module for controlling the smart rearview mirror to interact with a user based on the current usage of the smart rearview mirror.

For the beneficial effects of the smart rearview mirror interaction device according to the second aspect above, reference can be made to the beneficial effects according to the first aspect above and will not be repeated here.

In a third aspect, the present application provides an electronic apparatus, comprising at least one processor, memory, a camera and an audio output device, said memory storing computer executable instructions, said at least one processor storing in said memory executing the computer-executable instructions described above to cause the electronic device to perform the smart rearview mirror interaction method of the first aspect above, wherein the camera collects user movements under the control of the processor; The audio output device outputs audio under the control of the processor to realize voice interaction between the electronic device and the user. is.

In a fourth aspect, the present application provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed by a processor, the above first step is provided. A method for interacting with a smart rearview mirror according to the aspect of .

In a fifth aspect, the present application provides a computer program stored on a computer-readable storage medium, instructions in said computer program, when executed by a processor, instructing a computer to perform the smart rearview mirror of the above first aspect. Execute the interaction method.

The smart rearview mirror interaction method, device, electronic device and storage medium according to the present application relate to the field of big data. In this application, based on the front camera in the smart rearview mirror, it is possible to realize the movement interaction with the user, and on the premise of increasing the interaction method with the user, the conventional voice interaction can recognize the user's voice in a specific scene. You can also solve the problem that you can't. Moreover, in the present application, the smart rearview mirror can also be controlled to interact with the user based on the current usage of the smart rearview mirror, so that the smart rearview mirror can control the current usage of the smart rearview mirror. Based on the state, a more portable interaction scheme can be used to interact with the user, increasing the interaction success rate and further improving the user experience.

It should be understood that the content described in this section is not intended to mark key or critical features of embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easier to understand throughout the following specification.

The drawings are used for a better understanding of the solution and do not constitute a limitation to the present application.
1 is a schematic diagram of a scene applying the interaction method of a smart rearview mirror according to the present application; FIG. 1 is a flow chart 1 of an embodiment of a method for interacting with a smart rearview mirror according to the present application; Fig. 2 is a flow chart 2 of an embodiment of a smart rearview mirror interaction method according to the present application; 1 is a schematic diagram 1 of the interaction between a smart rearview mirror and a user according to the present application; FIG. FIG. 2 is a schematic diagram 2 of interaction between a smart rearview mirror and a user according to the present application; Fig. 3 is a schematic diagram 3 of the interaction between a smart rearview mirror according to the present application and a user; FIG. 5 is a schematic diagram 5 of interaction between a smart rearview mirror according to the present application and a user; 1 is a structural schematic diagram of an interaction device for a smart rearview mirror according to the present application; FIG. 1 is a structural schematic diagram of an electronic device according to the present application; FIG.

DETAILED DESCRIPTION OF THE INVENTION Illustrative embodiments of the present application will now be described with reference to the drawings, wherein various details of the embodiments of the present application are included for ease of understanding and are considered to be exemplary only. should. Accordingly, those skilled in the art should appreciate that various changes and modifications can be made to the examples described herein without departing from the scope and spirit of the present application. Similarly, in the following description, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

With the development of vehicle intelligence, the number of in-vehicle electronic devices has gradually increased. Users can realize functions such as entertainment and navigation by interacting with in-vehicle electronic devices such as in-vehicle speakers and smart rearview mirrors installed in the vehicle. In the prior art, the interaction method between the user and the in-vehicle electronic device may be a voice interaction, for example, the user speaks a wake word to wake up the in-vehicle electronic device, and the in-vehicle electronic device wakes up the user's voice. , parse the user's semantics, and perform corresponding feedback actions. Exemplarily, when the user says "play song A", the on-vehicle electronic device recognizes the semantics of the user's voice and then plays song A.

In the vehicle-mounted field in the prior art, there is only one method of interaction with the user, namely voice interaction. When the vehicle is in a noisy environment, the method of voice interaction cannot accurately recognize the user's voice, and furthermore, cannot make accurate response movements, requiring the user to repeat and explain, and the user You may have a bad experience. Based on the conventional vehicle, if the method of interacting with the user is increased, such as movement interaction, the vehicle must be equipped with a camera, and the camera can collect the user's movement, and To be able to perform a corresponding response motion based on the user's motion. In order to mount a camera on a vehicle, it is necessary to install a specific model number of camera based on the vehicle model, the model number of the vehicle's central control motor, the operation system, etc., so the application of motion interaction is limited. It should be noted that some vehicles may not support the mounting of cameras, resulting in the inability to achieve motion interaction with the user.

At present, the development of smart rearview mirrors is in full swing, and their functions are also evolving from the original rearview function to auxiliary reversing function and driving record function. It should be noted that the smart rearview mirror in this application is a smart rearview mirror installed in the vehicle, and the smart rearview mirror is equipped with a rear camera to easily record the driving data of the vehicle. to In order to solve the above problems, a front camera can be installed in the smart rearview mirror, and the front camera can be used universally, regardless of the vehicle model, the model number of the central control motor of the vehicle, the operation system, etc. highly sexual. It should be understood that the front camera of the smart rearview mirror is a camera that captures the interior of the vehicle, and the rear camera of the smart rearview mirror is a camera that captures the exterior of the vehicle and the driving screen of the vehicle.

Therefore, in order to solve the above technical problems, the present application provides a smart rearview mirror interaction method, which controls the front camera installed in the smart rearview mirror to collect the user's movement, thereby eliminating the conventional Based on the voice interaction between the smart rearview mirror and the user, the movement interaction between the smart rearview mirror and the user is increased, and the technical solution of the present application increases the interaction mode between the vehicle electronic device and the user. On that premise, it can also solve the problem that the traditional voice interaction cannot recognize the user's voice in a noisy environment, thus improving the user experience.

FIG. 1 is a schematic diagram of a scene for applying the smart rearview mirror interaction method according to the present application, where the scene includes a user in a vehicle 10 and a smart rearview mirror 11, as shown in FIG. The smart rearview mirror integrates a front camera to capture the user's movement inside the vehicle and realize the motion interaction between the user and the smart rearview mirror. It should be understood that FIG. 1 shows the front camera as a circle on the smart rearview mirror.

It should be understood that the execution body of the smart rearview mirror interaction method according to the present application may be a smart rearview mirror interaction device, and the smart rearview mirror interaction device is a smart rearview mirror or a smart rearview mirror. It may be a processor, chip, etc. in the mirror. The smart rearview mirror has an integrated front camera.

Hereinafter, the smart rearview mirror interaction method according to the present application will be described with reference to specific embodiments. It should be noted that some of the examples below can be combined with each other. FIG. 2 is a flow chart 1 of an embodiment of a smart rearview mirror interaction method according to the present application. As shown in FIG. 2, the smart rearview mirror interaction method according to the present embodiment may include the following steps.

S201, after waking up the smart rearview mirror, determine the current use status of the smart rearview mirror.

S202, controlling the smart rearview mirror to interact with the user according to the current usage status of the smart rearview mirror;

After waking up the smart rearview mirror in S201 above, the current usage status of the smart rearview mirror can be determined. After waking up the smart rearview mirror can also be expressed as the smart rearview mirror being in a wakeup state.

It should be noted that in this embodiment, the manner in which the user wakes up the smart rearview mirror is not limited. Selectably, the user can wake up the smart rearview mirror by way of voice interaction, for example, the user can wake up the smart rearview mirror by a wake word. Illustratively, for example, entering a standby state after turning on the smart rearview mirror enters a wake-up state if it detects that a wake word is included in the user's voice. Optionally, the user can also wake up the smart rearview mirror by way of motion interaction. Illustratively, when turning on the smart rearview mirror, the front camera can be activated and controlled to collect the user's movement, and the movement collected by the camera is determined to be the wake-up movement. When it does, it enters the wake-up state. The wake-up motion may be a hand caress, a continuous wink for a preset number of times, or the like. It should be appreciated that the wake word or motion that wakes up the smart rearview mirror can be customized by the user.

In this embodiment, the current working status of the smart rearview mirror can be detected to obtain the current usage status of the smart rearview mirror. Alternatively, the current usage of the smart rearview mirror can be obtained by interacting with a sensor installed in the vehicle, and the sensor can detect the current usage of the smart rearview mirror.

The current usage status of the smart rearview mirror in this embodiment may be the current operating status of the smart rearview mirror, such as the operating time of the smart rearview mirror, the application program in operation, and so on. Alternatively, the current usage state of the smart rearview mirror may be the state of the vehicle with the smart rearview mirror, for example, the vehicle is in a driving state or a stationary state, and the driving state is divided into a high speed driving state and a low speed driving state. be done. It should be understood that depending on the current usage of the smart rearview mirror, the corresponding smart rearview mirror and the user's suitable interaction method will be different.

After determining the current usage status of the smart rearview mirror in S202 above, the smart rearview mirror can be controlled to interact with the user based on the current usage status of the smart rearview mirror. It should be understood that in this embodiment, the manner of interaction between the smart rearview mirror and the user includes, but is not limited to, voice interaction and motion interaction. Correspondingly, controlling the smart rearview mirror and the user to perform voice interaction is to collect the user's voice and perform corresponding operations based on the user's voice. Controlling the motion interaction between the smart rearview mirror and the user, that is, controlling the camera of the smart rearview mirror to collect the user's motion and performing corresponding operations based on the user's motion. It is possible.

For different usage states of the smart rearview mirror, the corresponding interaction mode between the smart rearview mirror and the user, such as voice interaction or motion interaction, can be preset. Further, in this embodiment, after determining the current use state of the smart rearview mirror, it is determined that the smart rearview mirror and the user perform voice interaction or motion interaction according to the preset correspondence relationship. can be done.

For example, if the current use state of the smart rearview mirror is the state of the environment in which the smart rearview mirror exists, the corresponding relationship between different states of the environment in which the smart rearview mirror exists and the interaction method can be preset; For example, in order to ensure the user's driving safety, when a vehicle with a smart rearview mirror is in a high-speed driving state, the interaction mode between the smart rearview mirror and the user is set to be the motion interaction mode, and the vehicle is running at low speed. When in driving state, the interaction mode between the smart rearview mirror and the user can be set to be the voice interaction mode.

In the interaction method of the smart rearview mirror according to the present embodiment, the front camera installed in the smart rearview mirror can realize the movement interaction with the user. Interaction can also solve the problem of not being able to recognize the user's voice in certain scenes, improving the user experience. Moreover, in this embodiment, the smart rearview mirror can also be controlled to interact with the user based on the current usage status of the smart rearview mirror, and based on the current usage status of the smart rearview mirror, a more portable The interaction method is used to interact with the user to increase the interaction success rate and further improve the user experience.

Based on the above examples, in this example, the current usage state of the smart rearview mirror may be the state of the environment in which the smart rearview mirror is located, the type of interaction application currently provided to the user by the smart rearview mirror, or the smart rearview mirror. It may be the interaction content that the rearview mirror currently has a voice interaction with the user. Below, the current usage status of the smart rearview mirror is the state of the environment where the smart rearview mirror is located, the type of interaction application currently provided to the user by the smart rearview mirror, or the interaction that the smart rearview mirror currently performs voice interaction with the user. The interaction method of the smart rearview mirror according to the present embodiment will be described respectively from the three scene angles of content.

The state of the environment in which the smart rearview mirror is located may be a noisy state or a non-noisy state, and the type of interaction applications currently provided to users by the smart rearview mirror may be display service type, audio or video service type or storage. It may be a service type or the like, and the interaction content that the smart rearview mirror currently performs voice interaction with the user may be voice interaction content, display interaction content, or the like. In this embodiment, the smart rearview mirror acquires the state of the environment by interacting with sensors in the vehicle. The state of the environment in which the smart rearview mirror resides can be obtained by interacting with a decibel meter in the vehicle or other sensor that detects environmental noise. To obtain the type of interaction application currently provided to the user by the smart rearview mirror in this embodiment and the interaction content that the smart rearview mirror currently performs voice interaction with the user, the content that the smart rearview mirror is currently activated is It can be obtained by detecting For example, detecting the application currently displayed on the smart rearview mirror and making the type of the application the type of interaction application currently being provided to the user by the smart rearview mirror; The content that is present or being played by the smart rearview mirror can be detected and made into the interaction content that the smart rearview mirror currently has a voice interaction with the user.

In the first type of scene, the current use state of the smart rearview mirror is the state of the environment in which the smart rearview mirror is located.

The smart rearview mirror obtains the noise decibel value of the current environment, and based on the noise decibel value, the smart rearview mirror can determine whether the environment is in a noisy state. When the environment is currently noisy, the smart rearview mirror determines that the user's voice cannot be recognized, determines that it is better to use motion interaction, and controls the smart rearview mirror and the user to perform motion interaction. The smart rearview mirror determines that the user's voice can be accurately recognized when the current environment is in a non-noisy state, and the smart rearview mirror and the user perform voice interaction to ensure the user's driving safety. can be controlled.

Optionally, the vehicle in this example is equipped with a decibel meter or other device that detects decibels, and interacts with the decibel meter or other device that detects decibels in the vehicle to create a smart A rearview mirror can obtain the decibel value of the noise in the current environment. Optionally, on the smart rearview mirror, a decibel meter or other decibel detecting device function may also be integrated, for the smart rearview mirror to obtain the decibel value of the noise in the current environment. used.

In this embodiment, a preset decibel threshold can be preset, the preset decibel threshold can be a decibel value at which the user's voice cannot be recognized, and the preset decibel threshold can be obtained through simulation experiments. Alternatively, the preset decibel threshold can be set customized by the user. When the smart rearview mirror determines that the current environmental noise decibel value is greater than or equal to the preset decibel threshold, the smart rearview mirror can determine that the environment is in a noisy state, and the smart rearview mirror detects the current environmental noise decibel value. When the value is less than the preset decibel threshold, the smart rearview mirror can determine that the environment is in a non-noisy state. Further, in the present embodiment, the smart rearview mirror determines that the environment is currently in a noisy state based on the current environmental noise decibel value, and furthermore, the smart rearview mirror determines an accurate interaction between the smart rearview mirror and the user. Lay a foundation for establishing an interaction strategy.

In such a scene, ensure that the user can interact with the correct interaction method, and in a noisy environment, the user's voice cannot be accurately recognized, and the corresponding operation must be performed accurately. It can avoid the problem of not being able to do it, increase the success rate of interaction, and improve the user experience.

In the second type of scene, the current usage of smart rearview mirrors is the type of interaction applications currently provided to users by smart rearview mirrors.

The types of interaction applications currently offered to users by smart rearview mirrors can include display service types, audio and video service types, storage service types, and the like. Display service-type interaction applications can include weather query applications, shopping applications, social applications, navigation applications, vehicle parameter control applications, etc., and audio and video service-type interaction applications include music applications, video applications, and broadcast applications. etc., and storage service type interaction applications include driving record applications, image recording applications, and the like. It should be understood that the vehicle parameter control application can be used to regulate vehicle temperature, purify vehicle interior air, and the like.

In this embodiment, the smart rearview mirror and the user can be controlled to perform voice interaction or motion interaction based on the type of interaction application currently provided to the user by the smart rearview mirror. Since different types of interaction applications have their own attributes, in this embodiment, based on the type of interaction application, a more suitable interaction method for the current interaction application can be determined to increase the interaction success rate. can.

It should be appreciated that audio and video service type interaction applications play audio constantly during operation, and the played audio can lead to an inaccurate recognition of the user's voice.

In one possible implementation, in this embodiment, the audio or video service type can be set as the first type, that is, the first type interaction application is an interaction application that provides audio services to users. be.

In this embodiment, when detecting that the type of interaction application currently provided to the user by the smart rearview mirror is the first type, the smart rearview mirror and the user are controlled to perform motion interaction. When determining that the type of interaction application currently provided to the user by the smart rearview mirror is not the first type, controlling the smart rearview mirror and the user to perform voice interaction. Motion recognition is not recommended for audio service type interaction applications because audio service type interaction applications play audio constantly while running, and the played audio may interfere with accurate recognition of the user's voice. Using Speech Recognition can be used for non-audio service type interaction applications, increasing the interaction success rate.

In order to ensure that individual users have various options for interaction methods, in this embodiment, according to the user's interaction habits, the smart rearview mirror and the user are controlled to perform motion interaction or voice interaction. By determining , it is possible to interact with the user using an interaction scheme that is more suitable for the user's habits, thereby improving the user experience.

In one possible implementation, in this example, we obtain the user's preferred interactions for various types of interaction applications on the smart rearview mirror, and also obtain the user's preferred interaction for various types of interaction applications on the smart rearview mirror, and also After determining the type, the smart rearview mirror and the user can be controlled to have the preferred interaction of that type of interaction application. It should be understood that the preferred interaction is the interaction method used by the user's habits or preferences, and the preferred interaction is voice interaction or motion interaction.

In this embodiment, based on the historical interaction data between the user and various types of interaction applications on the smart rearview mirror, the user's preferred interactions for various types of interaction applications can be determined. Optionally, the historical interaction data may be interaction data within a preset period of time before the current smart rearview mirror was woken up. It should be understood that instead of using historical interaction data from a long time ago, using interaction data within a preset time period before the present smart rearview mirror was woken up as historical interaction data is based on the user's This is to more accurately determine the recent favorite interaction, improve the accuracy of the determined user's favorite interaction, and further improve the accuracy of controlling the interaction method between the smart rearview mirror and the user.

In this embodiment, the method of determining the user's preferred interaction for different types of interaction applications based on historical interaction data is based on the historical interaction data, the rate of motion interaction between the user and different types of interaction applications, and Determine the rate of voice interaction. It should be understood that in this embodiment, the number of motion interactions and the number of voice interactions of the user's preferred interaction with the interaction application are statistically calculated, and the ratio of the number of motion interactions to the total number of interactions is called the motion interaction ratio. and the ratio of the number of voice interactions to the total number of interactions can be taken as the rate of voice interactions. Each movement in the interaction process between the user and the interaction application can be regarded as one movement interaction, and each speech spoken in the interaction process between the user and the interaction application can be regarded as one voice interaction.

If it is determined that the proportion of motion interaction for the second type interaction application is greater than or equal to the proportion of voice interaction, then determining that the user's preferred interaction for the second type interaction application is motion interaction. If it is determined that the proportion of motion interactions for the second type of interaction application is less than the proportion of voice interactions, then it is determined that the user's preferred interaction for the second type of interaction application is voice interaction. The second type is any type among interaction application types in the smart rearview mirror, for example, the second type may be audio or video service type, display service type or storage service type, etc. .

Illustratively, based on historical interaction data for an interaction application of type audio or video service, the proportion of motion interaction between the user and that type of interaction application is 80%, and the proportion of voice interaction between the user and that type of interaction application. is 20%, then we determine that the user's preferred interaction for audio and video service type interaction applications is motion interaction. Based on the historical interaction data of interaction applications of the display service type, determining that the percentage of motion interactions between the user and interaction applications of that type is 30% and the percentage of voice interactions between users and interaction applications of that type is 70%. If so, determine that the user's preferred interaction for the display service type interaction application is voice interaction.

Further, after determining the user's preferred interaction for various types of interaction applications on the smart rearview mirror, the smart rearview mirror can provide the user with the current interaction based on the type of interaction application currently provided to the user by the smart rearview mirror. The user's preferred interaction with the provided interaction application can be determined, and the user's preferred interaction with the smart rearview mirror can be controlled. In this embodiment, the user's preferred interactions for different types of interaction applications are determined based on historical interaction data of different types of interaction applications on the smart rearview mirror, thereby determining the user's preferred interactions. Establish and then lay the groundwork to establish the correct interaction scheme.

In such a scene, based on the type of interaction applications currently offered to the user by the smart rearview mirror, or the user's preferred interaction with different types of interaction applications, the smart rearview mirror can provide more accurate and user It can be determined to interact with the user using an interaction scheme that is more suited to their habits to improve the user experience.

In the third type of scene, the current usage status of the smart rearview mirror is the interaction content that the smart rearview mirror currently performs voice interaction with the user.

In this embodiment, the smart rearview mirror and the user can be controlled to perform voice interaction or motion interaction based on the current interaction content of the smart rearview mirror with the user. The interaction content that the smart rearview mirror currently performs voice interaction with the user may be the audio content that is currently being played on the smart rearview mirror. It should be understood that the audio currently playing in the smart rearview mirror may be audio played in response to the user's voice or movement, such as "Would you like to play this song?" good. Alternatively, the audio currently being played by the smart rearview mirror may also be the audio played by itself after turning on the smart rearview mirror, e.g. There may be. It should be appreciated that the self-played audio may be predefined or set by the user. In this design, the user may respond in different manners according to the interaction content, and in this embodiment, the interaction manner more suitable for the user can be used according to the interaction content, and the user experience can be improved.

In one possible implementation, in this embodiment, the smart rearview mirror and the user are controlled to perform voice interaction or motion interaction according to the semantics of the interaction content that the smart rearview mirror currently performs voice interaction with the user. can be determined. Specifically, by determining whether the user instructs to confirm, select, or turn a page according to the content of the interaction, the smart rearview mirror and the user can perform voice interaction or motion interaction. can be determined to control If the interaction content prompts the user to confirm, select or turn a page, control the smart rearview mirror and the user to interact with each other, otherwise, the smart rearview mirror and the user will hear voice Control to interact.

For example, the content of the interaction that the smart rearview mirror currently performs voice interaction with the user is "Do you want to plan the route to your house now?" , the smart rearview mirror and the user can be controlled to interact with each other. The confirmation operation performed by the user may be a head movement, specifically nodding or shaking the head.

Exemplarily, the interaction content that the smart rearview mirror currently performs voice interaction with the user is "I found two songs, the first is song A and the second is song B", and the interaction content allows the user to The smart rearview mirror and the user can be controlled to move and interact when instructed to perform a selection operation. The selection operation performed by the user may be a hand movement. Specifically, the song selected is represented by the number of fingers extended. For example, extending one finger represents selection of song A. , indicates that song B is selected by extending two fingers.

For example, the interaction content that the smart rearview mirror currently performs voice interaction with the user is "Please enter your phone number on the next page", and the interaction content prompts the user to turn the page. , the smart rearview mirror and the user can be controlled to interact with each other. The user operation to turn a page may be a hand movement, and specifically, it can represent turning a page by sliding the palm from right to left.

In one possible implementation, each interaction content that the smart rearview mirror in this embodiment currently performs voice interaction with the user is marked with a label, and the label indicates the user's interaction method based on the interaction content. . For example, the interaction content that the smart rearview mirror currently performs voice interaction with the user is "Enter your phone number on the next page", and the label of the interaction content is "Voice Interaction, Motion Interaction". if there is, the label indicates that the user's interaction mode based on the interaction content includes voice interaction and motion interaction, that is, the label indicates that the interaction mode for the user to respond to the interaction content is voice interaction and Indicates the inclusion of motion interaction.

In this embodiment, based on the interaction content and the label of the interaction content, the smart rearview mirror and the user can be controlled to perform voice interaction or motion interaction. As long as the interaction content label has motion interaction, the smart rearview mirror and the user can be controlled to perform motion interaction. In other words, if the interaction content label includes motion interaction, the smart rearview mirror and the user are controlled to perform motion interaction, and if the interaction content label does not include motion interaction, the smart rearview mirror and Control voice interactions with users.

In such a scene, according to the meaning or label of the interaction content with which the smart rearview mirror currently performs voice interaction with the user, the smart rearview mirror may interact with the user using an interaction method that is more accurate and more suitable for the user's habits. can be determined and the user experience can be improved.

Based on the above embodiments, the smart rearview mirror interaction method according to the present application will now be described in more detail with reference to FIG. FIG. 3 is a flow chart 2 of an embodiment of a smart rearview mirror interaction method according to the present application. As shown in FIG. 3, the smart rearview mirror interaction method according to the present embodiment may include the following steps.

S301, after waking up the smart rearview mirror, determine the current use status of the smart rearview mirror.

S302, if it is determined to control the movement interaction between the smart rearview mirror and the user according to the current use state of the smart rearview mirror, control the camera of the smart rearview mirror to open, and the camera will move to the position of the user; Control to collect movements.

S303, perform a corresponding operation according to the user's movement;

S304, if it is determined to control the smart rearview mirror and the user to have voice interaction according to the current usage status of the smart rearview mirror, collecting the user's voice and obtaining the meaning of the voice;

S305, perform a corresponding operation according to the meaning of the voice;

For the embodiment in S301 above, reference can be made to the relevant description in the above embodiment S201, without repeating here, in S302, controlling the smart rearview mirror and the user to perform motion interaction. and determining in S304 to control the smart rearview mirror and the user to have a voice interaction, reference can be made to the relevant description in the above embodiment S202, here again: No explanation. It should be understood that either of the above S302-S303 and S304-S305 is selected.

In S302 above, if it is determined to control the smart rearview mirror and the user to perform motion interaction according to the current use state of the smart rearview mirror, the smart rearview mirror and the user perform motion interaction; Specifically, the camera of the smart rearview mirror can be controlled to open, and the camera can be controlled to collect the user's movements. It should be understood that in this embodiment, when activating the smart rearview mirror, the camera of the smart rearview mirror can be controlled to open. Among them, after the camera is opened, if the smart rearview mirror and the user have a voice interaction within a preset period, the camera can be controlled to enter the sleep state. In this embodiment, when it is determined that the smart rearview mirror and the user make a motion interaction, the camera is controlled to be opened, thereby controlling the camera to collect the user's movement, and the power consumption of the camera can be reduced. It should be understood that when the camera is controlled to collect user movements, the user movements that need to be collected can be sent to the camera. User movements may include head movements, gesture movements, body movements, or the like. Head movements can include movements such as nodding, shaking the head, blinking the eyes, making funny faces, and the like. Gesture movements can include movements such as a few finger extensions, a palm slide, a hand OK, a hand affection, and the like. Body movements may be crossing hands, fighting poses, and the like.

In order to ensure the user's driving safety, it can be determined to set the user's movement according to the driving state of the vehicle. Optionally, when the vehicle is in motion, the user's movements are head movements to ensure that the user's hands do not leave the steering wheel. When the vehicle is in a non-running state, the user's movements include head movements and/or hand movements.

In S303 above, the camera can be controlled to collect the user's movement, and the user's movement marks that need to be collected, ie the preset movement marks, can be sent to the camera. The preset motion mark is for representing the preset motion. Each camera can store the correspondence between the preset motion marks and the preset motion marks. The camera can determine the user motion that needs to be collected based on the marks of the preset motion, and after collecting the user motion, the camera determines the collected motion and the motion that needs to be collected (i.e., the preset motion ) to determine whether it is the preset motion.

After determining that the preset movement has been collected, the successfully collected information and the mark of the user's movement (i.e., the mark of the preset movement) can be sent to the smart rearview mirror interaction device, and the smart rearview mirror interaction device , determine that the user's motion is the preset motion, and perform corresponding operations according to the user's motion mark. It should be understood that the interaction device of the smart rearview mirror can memorize the correspondence between the preset movement mark and the command, and when it receives the successful collection response sent by the camera, the preset movement According to the corresponding relationship between the mark and the instruction, the instruction corresponding to the movement can be obtained, and the corresponding operation can be performed according to the instruction.

Exemplarily, Table 1 shows the correspondence between preset movement marks and commands stored in the interaction device of the smart rearview mirror,

When the smart rearview mirror plays the audio "Can you plan your route to home now?" and the audio instructs the user to perform a confirmation operation, the camera of the smart rearview mirror Movement can be controlled to collect. The preset motion that the user confirms may be a head motion, specifically nodding or shaking the head. One or two user movement marks that need to be collected can be sent to the camera. If the camera collects the user's nodding motion, it can compare it with preset motion 1 or 2, and when it determines that the user's motion is preset motion 1, smart mark 1 of the preset motion and successfully collect information. can be sent to the rearview mirror interaction device, and the smart rearview mirror interaction device determines that the user's motion is the preset motion 1, and according to the instructions corresponding to the preset motion 1, performs corresponding operations. can be executed.

FIG. 4 is a schematic diagram 1 of the interaction between a smart rearview mirror and a user according to the present application. As shown at a in FIG. 4 , the smart rearview mirror exemplarily plays the audio "Would you like to plan your route to home now?" , the smart rearview mirror's camera can be controlled to collect the user's nodding or shaking motion. For example, as shown in FIG. 4a, when the camera collects the user's nodding motion, the mark of the user's nodding motion can be sent to the interaction device of the smart rearview mirror, and the interaction device of the smart rearview mirror can: determining that the user's nodding motion is a preset motion, and determining a command that agrees with the suggestion corresponding to the user's nodding motion based on the correspondence relationship between the preset motion mark and the command; Therefore, the planned route can be displayed on the interface of the smart rearview mirror, as shown at b in FIG.

Selectably, when performing the corresponding operation corresponding to the preset motion, an audio can be played, and the audio indicates that the corresponding operation is being performed. In such a situation, replaying the commensurate operation being performed can draw the user's attention to the movement being performed, without the need for the user to lift his head to look at the smart rearview mirror, and the user's Driving safety can be ensured. By way of example, as shown at b in FIG. 4, for example, "I have planned the route to my house. I will be leaving soon, so please drive to the road XX." can be played.

FIG. 5 is a schematic diagram 2 of the interaction between the smart rearview mirror and the user according to the present application. Illustratively, as shown at a in FIG. 5, for example, the smart rearview mirror plays the audio "I found two songs. The first is song A and the second is song B. Which would you like?" , if the audio dictates a user-selected operation, the camera of the smart rearview mirror can be controlled to collect the user's movements. Among them, the preset motion for the user to select may be a hand motion, specifically, a finger stretching motion, and the number of fingers represents the user's selection. As shown in FIG. 5a, when the camera collects the movement of the user stretching one finger, the camera can send the mark of the movement of the user stretching one finger to the interaction device of the smart rearview mirror, and the smart The interaction device of the rearview mirror enables the user to determine that the movement of extending one finger is a preset movement, and according to the correspondence between the marks of the preset movement and the command, the user can extend one finger. The command corresponding to the stretch motion can determine that the user's selection is 1, such that the command plays song A in time with the audio played by the smart rearview mirror, as shown at b in FIG. Since it is confirmed that the instruction is given to , the song A can be reproduced. Alternatively, in this embodiment, as shown in FIG. 5b, it is also possible to play, for example, "Song A will be played", so as to indicate that a corresponding operation is being performed. become.

FIG. 6 is a schematic diagram 3 of the interaction between the smart rearview mirror and the user according to the present application. As shown in a in FIG. 6, for example, the smart rearview mirror plays an audio "Please enter your phone number on the next page", and the audio prompts the user to turn pages. The smart rearview mirror's camera can be controlled to collect the user's movements if instructed to do so. The preset motion for the user to turn the page may be the hand motion, specifically, the left and right slide of the palm, and the slide of the palm from right to left represents turning the page to the right. and sliding the palm from left to right represents turning the page to the left. As shown in FIG. 6a, when the camera collects the right-to-left slide motion of the user's palm, the camera sends the mark of the right-to-left slide motion of the palm to the interaction device of the smart rearview mirror. and the interaction device of the smart rearview mirror determines that the movement of the palm sliding from right to left is a preset movement, and according to the correspondence between the preset movement marks and the instructions, the user's palm moves to the right. Since it can be determined that the command corresponding to the movement of sliding left from is the command to turn the page to the right, the smart rearview mirror can perform the operation of turning the page to the right, as shown in b in FIG. As such, the interface of smart rearview mirrors has changed to an interface for inputting phone numbers by users. Alternatively, in this embodiment, as shown in FIG. to tell it to run.

It should be noted that when the smart rearview mirror's camera collects the user's movements and compares them with the preset movements, if it determines that the user's movements are not the preset movements, it sends the failed collection information to the smart rearview mirror's interaction device. After receiving the collection failure information, it can be determined that the user's motion is not the preset motion, controlling the smart rearview mirror and the user to perform voice interaction.

FIG. 7 is a schematic diagram 4 of the interaction between the smart rearview mirror and the user according to the present application. As shown in a in FIG. 7, for example, the smart rearview mirror plays an audio "Please enter your phone number on the next page", and the audio prompts the user to turn pages. The smart rearview mirror's camera can be controlled to collect the user's movements if instructed to do so. If the user did not perform the preset motion, or the motion that the user performed is not a palm sliding operation, the camera can send the failed collection information to the interaction device of the smart rearview mirror, at this time , the smart rearview mirror and the user can be controlled to have a voice interaction. As shown in b in FIG. 7, the user can say "turn to the next page" and further perform the operation of turning pages based on the user's voice, and b in FIG. The mirror's interface is the user's phone number entry interface.

In the above S304, if, in the present embodiment, it is determined to control the smart rearview mirror and the user to perform voice interaction based on the current usage status of the smart rearview mirror, the smart rearview mirror and the user can communicate with voice The interaction can be controlled, specifically, the user's voice can be collected and the meaning of the voice can be obtained.

In the above S305, according to the semantics of the voice, corresponding operations are performed.

In this embodiment, the method of obtaining the meaning of the user's voice and performing corresponding operations can specifically refer to the voice interaction method in the prior art. Illustratively, as shown in FIG. 7, when controlling the smart rearview mirror and the user to have a voice interaction, the user's voice, for example, "turn to the next page" can be collected, and FIG. If the meaning of the voice is to instruct to turn a page, perform a corresponding operation to turn the page, as shown in b in FIG.

It should be understood that FIGS. 4-7 show that the smart rearview mirror is in an open state by a white ring when the smart rearview mirror and the user are moving and interacting with each other. and when the smart rearview mirror and the user have a voice interaction, the smart rearview mirror is a black ring to indicate that the smart rearview mirror is in a closed state.

In this embodiment, when controlling the smart rearview mirror and the user to perform motion interaction, if the collected user's motion is the preset motion, a corresponding operation can be performed, and the collected user's If the movement is not a preset movement, the smart rearview mirror and the user can be controlled to have a voice interaction, in this embodiment, based on the current usage status of the smart rearview mirror and the user's movement, the smart rearview It can be determined that the mirror performs corresponding operations or conducts voice interaction with the user, and uses an interaction mode that is easier and more suitable for the user's behavior to interact with the user, increasing the interaction mode with the user. On the premise, it can also increase the interaction success rate.

FIG. 8 is a structural schematic diagram of the smart rearview mirror interaction device according to the present application. As shown in FIG. 8 , the smart rearview mirror interaction device 800 includes a first processing module 801 and a second processing module 802 .

The first processing module 801 is used to determine the current usage status of the smart rearview mirror after waking up the smart rearview mirror.

A second processing module 802 is used to control the smart rearview mirror to interact with the user based on the current usage of the smart rearview mirror.

In one possible design, the first processing module 801 is specifically used to determine whether the smart rearview mirror is currently in a noisy environment.

Correspondingly, the second processing module 802 specifically controls the movement and interaction between the smart rearview mirror and the user when the smart rearview mirror is currently in a noisy environment; Mirrors are currently used to control smart rearview mirrors and user voice interaction when the environment is in a non-noisy state.

In one possible design, the first processing module 801 also obtains the noise decibel value of the environment where the smart rearview mirror is currently located, and if the noise decibel value is greater than the preset decibel threshold, the smart rearview mirror is now Used to determine that the environment is in a noisy state.

In one possible design, the first processing module 801 is specifically used to determine the type of interaction application currently offered to the user by the smart rearview mirror.

Correspondingly, the second processing module 802 performs voice interaction or motion interaction between the smart rearview mirror and the user, specifically based on the type of interaction application currently provided to the user by the smart rearview mirror. used to control the

In one possible design, interaction applications include music applications, navigation applications or vehicle parameter control applications.

In one possible design, the second processing module 802 specifically determines that the type of interaction application currently provided to the user by the smart rearview mirror is the first type. to perform movement interaction, the first type of interaction application is an interaction application that provides audio services to the user, and the type of interaction application currently provided to the user by the smart rearview mirror is If it determines that it is not the first type, it is used to control the smart rearview mirror and the user to have a voice interaction.

In one possible design, the second processing module 802 specifically identifies the user's preferences for different types of interaction applications based on historical interaction data between the user and different types of interaction applications on the smart rearview mirror. and that the preferred interaction is a voice interaction or a motion interaction, the user's preferred interactions for different types of interaction applications, and the number of interaction applications currently offered to the user by the smart rearview mirror. Based on the type, it is used to determine the user's preferred interaction for the interaction application currently provided to the user by the smart rearview mirror, and to control the smart rearview mirror and the user to make the preferred interaction. .

In one possible design, the second processing module 802 specifically determines the percentage of motion interaction and the percentage of voice interaction between the user and various types of interaction applications based on historical interaction data; is determined to be greater than or equal to the voice interaction percentage of the interaction application of type , then determine that the user's preferred interaction for the second type of interaction application is motion interaction, and the second type is , any one type among the interaction application types in the smart rearview mirror, and the second type if the motion interaction percentage of the interaction application of the second type is determined to be less than the voice interaction percentage. interaction application to determine that the user's preferred interaction for the application is voice interaction.

In one possible design, the first processing module 801 is specifically used to determine the interaction content for which the smart rearview mirror currently performs voice interaction with the user.

Correspondingly, the second processing module 802 specifically causes the smart rearview mirror and the user to perform voice interaction or motion interaction according to the interaction content that the smart rearview mirror currently performs voice interaction with the user. used to control.

In one possible design, the second processing module 802 specifically determines whether the interaction content prompts the user to confirm, select, or turn pages; , to control the motion interaction between the smart rearview mirror and the user, and otherwise control the voice interaction between the smart rearview mirror and the user.

In one possible design, the second processing module 802 specifically controls the smart rearview mirror and the user to perform voice interaction or motion interaction based on the interaction content and the label of the interaction content; Labels are used to direct the user's interaction strategy based on interaction content.

In one possible design, the second processing module 802 specifically controls the smart rearview mirror and the user to perform motion interaction when the interaction content label includes motion interaction; If the interaction content label does not include motion interaction, it is used to control the voice interaction between the smart rearview mirror and the user.

In one possible design, the second processing module 802 specifically controls the camera of the smart rearview mirror to open, controls the camera to collect the user's movement, and based on the user's movement , is used to perform the corresponding operations.

In one possible design, the second processing module 802 specifically acquires instructions corresponding to the movement when the movement is determined to be a preset movement, and performs corresponding operations according to the instruction. used for

In one possible design, the second processing module 802 is also used to control voice interaction between the smart rearview mirror and the user if it determines that the motion is not a preset motion.

In one possible design, the second processing module 802 is specifically used to collect the user's speech, obtain the speech semantics, and perform corresponding operations based on the speech semantics. be.

In one possible design, the second processing module 802 is further used to play audio when performing corresponding operations, and to indicate via audio that corresponding operations are being performed.

In one possible design, when the vehicle is in a driving state, the user's movements are head movements, and when the vehicle is in a non-running state, the user's movements include head movements and/or hand movements.

The implementation principle and technical effects of the smart rearview mirror interaction device according to the present embodiment are similar to the smart rearview mirror interaction method described above, so the description thereof will not be repeated here.

According to embodiments of the present application, the present application further provides an electronic device and a readable storage medium. FIG. 9 is a structural schematic diagram of an electronic device according to the present application. The electronic device may be a smart rearview mirror. The components, their connections and relationships, and their functionality illustrated herein are exemplary only and are not intended to limit the implementation of the application described and/or sought herein. .

As shown in FIG. 9, the electronic device comprises one or more processors 901, memory 902, and interfaces for connecting components comprising high speed and low speed interfaces. Each component is interconnected by a different bus and may be installed on a common motherboard or in some other manner as desired. The processor translates instructions executed by the electronic device, including instructions stored in or on the memory, so that the graphical information of the GUI can be displayed on an external input/output device (such as a display device coupled to the interface). can be processed. In other embodiments, multiple processors and/or multiple buses can be used along with multiple memories, if desired. Similarly, multiple electronic devices can be connected, each providing some necessary operating. In FIG. 9, one processor 901 is taken as an example.

Memory 902 is a non-transitory computer-readable storage medium according to the present application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the smart rearview mirror interaction method according to the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, and the computer instructions are used to cause a computer to perform the smart rearview mirror interaction method of the present application.

The memory 902 is a non-transitory computer-readable storage medium for storing non-transitory software programs, such as program instructions/modules corresponding to the smart rearview mirror interaction method of the present embodiment, non-transitory computer-executable It may be used to store programs and modules. The processor 901 performs the various functional applications and data processing of the server by executing non-transitory software programs, instructions and modules stored in the memory 902, i.e. the smart back in the above method embodiments. Realize the mirror interaction method.

The memory 902 may include a stored program area and a stored data area, where the stored program area may store an operating system, application programs required for at least one function, and the stored data area may store a smart rearview mirror. can be stored, such as data created using the electronic device to perform the interaction method of . Memory 902 can also include high speed random access memory and also includes non-transitory memory such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. can also In some embodiments, the memory 902 can optionally include memory remotely located with respect to the processor 901, which remote memory implements the smart rearview mirror interaction method over a network. It can be connected to an electronic device for Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

The electronics of the smart rearview mirror interaction method can further include an input device 903 , an audio output device 904 , a camera 905 and a display device 906 . The camera 905 may be the front camera of a smart rearview mirror, and the camera 905 collects user movements under the control of the processor 901 and is used to implement the camera movements in the above examples. The audio output device 904 outputs audio under the control of the processor 901 and is used to achieve interaction with the user, and the display device 906 is an electronic device interface, such as the is used to display the interface of the smart rearview mirror shown in FIGS.

The processor 901, the memory 902, the input device 903, the audio output device 904, the camera 905 and the display device 906 can be connected by a bus or other methods, and the connection by the bus is taken as an example in FIG.

The input device 903 can receive input numeric or character information and generate key signal input related to user settings and function control of the electronic device to perform the interaction method of the smart rearview mirror; Input devices include touch screens, keypads, mice, trackboards, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, and the like. The display device 906 can include a display device, a supplemental lighting device (such as an LED), a tactile feedback device (such as a vibration motor), and the like. The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, or a plasma display. In some embodiments, the display device may be a touch screen.

Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuit systems, integrated circuit systems, application specific integrated circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof. realizable. These various example embodiments may include the following, embodied in one or more computer programs that run on a programmable system that includes at least one programmable processor and/or or interpreted, the programmable processor receives data and instructions from the storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, at least one input device, and It may be a dedicated or general purpose programmable processor capable of sending to at least one output device.

These computer programs (also called programs, software, software applications, or code) contain machine instructions for programmable processors and utilize high-level process and/or object oriented programming languages and/or assembly/machine language. These computer programs can be implemented. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus used to provide machine instructions and/or data to a programmable processor. , and/or devices (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)), including machine-readable media for receiving machine instructions that are machine-readable signals. The term "" refers to any signal used to provide machine instructions and/or data to a programmable processor.

A computer may implement the systems and techniques described herein to provide interaction with a user, and the computer may include a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor), and a keyboard and pointing device (such as a mouse or trackball) through which a user can provide input to the computer. Other types of devices are also used to provide interaction with the user, e.g., the feedback provided to the user can be any form of sensory feedback (e.g., visual, auditory, or tactile feedback). well, it can receive input from the user in any form (including sound, speech, or tactile input).

The systems and techniques described herein may be computer systems that include back-end components (e.g., data servers), computer systems that include middleware components (e.g., application servers), or computer systems that include front-end components (e.g., graphical user interface or a user's computer equipped with a web browser, such that the user can interact with the system and technology embodiments with such a graphical user interface or web browser), or any such back-end, middleware, or front-end components. Also implemented in combination, the components of the system can be interconnected through any form or medium of digital data communication, including a communication network. Examples of communication networks include local area networks (LAN), wide area networks (WAN), and the Internet.

The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. A client-server relationship is created by executing computer programs on corresponding computers that have a client-server relationship to each other.

According to one embodiment of the present application, the present application further provides a computer program stored on a computer-readable storage medium, wherein instructions in the computer program, when executed by a processor, cause a computer to execute the smart back of the present embodiment. Execute the mirror interaction method.

It should be appreciated that the various forms of the process shown above can be used to sort, add and remove steps. For example, each step described in the present application may be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and is not limited to the text here.

The specific embodiments described above are not intended to limit the patent scope of this application. It should be understood by those skilled in the art that various modifications, combinations, subcombinations, and substitutions can be made depending on design requirements and other factors. All amendments, equivalent substitutions, improvements, etc. made without departing from the spirit and principles of the present application shall be included within the patent scope of the present application.

Claims (20)

An interaction method for a smart rearview mirror, comprising:
determining the current usage status of the smart rearview mirror after waking up the smart rearview mirror;
controlling the smart rearview mirror to interact with a user based on the current usage of the smart rearview mirror ;
The step of determining the current usage status of the smart rearview mirror includes:
determining a type of interaction application currently being offered to a user by said smart rearview mirror;
Correspondingly, controlling the smart rearview mirror to interact with a user based on the current usage of the smart rearview mirror comprises:
controlling the smart rearview mirror and a user to engage in voice interaction or motion interaction based on the type of interaction application currently offered to the user by the smart rearview mirror;
controlling the smart rearview mirror and a user to perform voice interaction or motion interaction based on the type of interaction application currently provided to the user by the smart rearview mirror;
determining the user's preferred interactions for different types of interaction applications based on historical interaction data between the user and different types of interaction applications on the smart rearview mirror, wherein the preferred interactions are , a voice interaction, or a step that is a motion interaction;
Based on the user's preferred interactions for various types of interaction applications and the types of interaction applications currently offered to the user by the smart rearview mirror, the interaction applications currently offered to the user by the smart rearview mirror. determining a user's preferred interaction;
and C. controlling the smart rearview mirror and a user to perform a preferred interaction of an interaction application currently provided to the user by the smart rearview mirror. The step of determining the current usage status of the smart rearview mirror includes:
further comprising determining whether the smart rearview mirror is currently in a noisy environment;
Correspondingly, controlling the smart rearview mirror to interact with a user based on the current usage of the smart rearview mirror comprises:
if the smart rearview mirror is currently in a noisy environment, controlling the smart rearview mirror and a user to interact in motion;
2. The method of claim 1, further comprising controlling a voice interaction between the smart rearview mirror and a user when the smart rearview mirror is not currently in a noisy environment. Determining whether the smart rearview mirror is currently in a noisy environment includes:
obtaining a noise decibel value of the environment in which the smart rearview mirror is currently located;
3. The method of claim 2, comprising determining that the smart rearview mirror is currently in a noisy environment if the decibel value of the noise is greater than a preset decibel threshold. The interaction application includes:
2. The method of claim 1 , comprising a music application, a navigation application or a vehicle parameter control application. controlling the smart rearview mirror and a user to perform voice interaction or motion interaction based on the type of interaction application currently provided to the user by the smart rearview mirror;
If determining that the type of interaction application currently provided to the user by the smart rearview mirror is the first type, controlling the smart rearview mirror and the user to perform motion interaction, wherein: the first type of interaction application is an interaction application that provides audio services to the user;
when determining that the type of interaction application currently provided to the user by the smart rearview mirror is not the first type, controlling the smart rearview mirror and the user to perform voice interaction. 5. A method according to claim 1 or claim 4 , characterized in that: determining the user's preferred interactions for different types of interaction applications based on historical interaction data between the user and different types of interaction applications on the smart rearview mirror;
determining a motion interaction rate and a voice interaction rate between the user and various types of interaction applications based on the historical interaction data;
determining that the user's preferred interaction for the second type interaction application is a motion interaction if determining that the motion interaction percentage of the interaction application of the second type is greater than or equal to the voice interaction percentage; Type 2 is any one type among interaction application types in the smart rearview mirror;
determining that the user's preferred interaction for a second type of interaction application is a voice interaction if determining that the percentage of motion interactions for the second type of interaction application is less than the percentage of voice interactions. 2. The method of claim 1 , comprising: The step of determining the current usage status of the smart rearview mirror includes:
further comprising determining an interaction content in which the smart rearview mirror currently performs voice interaction with the user;
Correspondingly, controlling the smart rearview mirror to interact with a user based on the current usage of the smart rearview mirror comprises:
2. The method of claim 1, further comprising: controlling the smart rearview mirror and the user to perform voice interaction or motion interaction based on an interaction content in which the smart rearview mirror is currently performing voice interaction with the user. The method described in . The step of controlling the smart rearview mirror and the user to perform voice interaction or motion interaction based on an interaction content in which the smart rearview mirror is currently performing voice interaction with the user,
a step of determining whether the user instructs to confirm, select, or turn a page according to the content of the interaction;
if so, controlling the smart rearview mirror to interact with a user;
otherwise, controlling the smart rearview mirror to have voice interaction with a user. The step of controlling the smart rearview mirror and the user to perform voice interaction or motion interaction based on an interaction content in which the smart rearview mirror is currently performing voice interaction with the user,
Based on the interaction content and the label of the interaction content, the smart rearview mirror and the user are controlled to perform voice interaction or motion interaction, and the label allows the user to select an interaction method based on the interaction content. 8. The method of claim 7 , comprising the step of indicating. controlling the smart rearview mirror and a user to perform voice interaction or motion interaction based on the interaction content and the interaction content label;
if the interaction content label includes motion interaction, controlling the smart rearview mirror and the user to perform motion interaction;
10. The method of claim 9 , comprising controlling the smart rearview mirror and a user to have voice interaction if the interaction content label does not include motion interaction. controlling the smart rearview mirror and the user to interact with each other in motion,
controlling the camera of the smart rearview mirror to open and controlling the camera to collect the movement of the user;
and performing corresponding operations based on the user's movements. performing a corresponding operation based on the user's movement,
if the motion is determined to be a preset motion, obtaining instructions corresponding to the motion;
and c. performing the corresponding operation based on the instruction. The method also includes:
13. A method according to claim 11 or 12 , comprising controlling a voice interaction between the smart rearview mirror and a user when determining that the movement is not a preset movement. The step of controlling the smart rearview mirror and the user to have a voice interaction comprises:
collecting the user's speech and obtaining a meaning of the speech;
and performing corresponding operations based on the semantics of the speech. The method further comprises:
15. The method of any one of claims 11 to 14 , comprising, when performing the commensurate operation, playing an audio indicating that the commensurate operation is being performed . Method. when the vehicle is in a running state, the movement of the user is head movement;
A method according to any one of claims 11 to 14 , characterized in that when the vehicle is in a stationary state, the user's movements comprise head movements and/or hand movements. A smart rearview mirror interaction device, comprising:
a first processing module used to determine the current usage status of the smart rearview mirror after waking up the smart rearview mirror;
a second processing module used to control the smart rearview mirror to interact with a user based on the current usage of the smart rearview mirror ;
The first processing module includes:
also used to determine the type of interaction application currently offered to the user by said smart rearview mirror;
Correspondingly, the second processing module comprises:
also used for voice interaction or motion interaction between the smart rearview mirror and a user based on the type of interaction applications currently offered to the user by the smart rearview mirror;
The second processing module includes:
determining the user's preferred interactions for different types of interaction applications based on historical interaction data between the user and different types of interaction applications on the smart rearview mirror, wherein the preferred interactions are , voice interaction, or motion interaction;
Based on the user's preferred interactions for various types of interaction applications and the types of interaction applications currently offered to the user by the smart rearview mirror, the interaction applications currently offered to the user by the smart rearview mirror. determining a user's preferred interaction; and
and controlling the smart rearview mirror and a user to perform a preferred interaction of an interaction application currently provided to the user by the smart rearview mirror. interaction device. an electronic device,
at least one processor;
a memory communicatively coupled with the at least one processor;
including a camera and an audio output device;
Instructions executable by the at least one processor are stored in the memory, and when the instructions are executed by the at least one processor, the at least one processor performs any of claims 1 to 16 . Capable of executing the method of any one of claims 1 to 3, wherein the camera collects user's movements under control of the processor to realize movement interaction between the electronic device and the user. and wherein said audio output device is for outputting audio under the control of said processor to realize voice interaction between said electronic device and a user. A non-transitory computer-readable storage medium having computer instructions stored thereon, said computer instructions for causing a computer to perform the method of any one of claims 1 to 16 . A non-transitory computer-readable storage medium characterized by: A computer program, characterized in that, when said computer program is executed by a processor, it implements the method of any one of claims 1 to 16 .

Images