JP2021103580A - Method for interaction of smart back mirror, device, electronic apparatus, and storage medium - Google Patents

Abstract

To provide a method for an interaction of a smart back mirror, a device, an electronic apparatus, and a storage medium that can interact with a user by using a portable interaction system, increases the success rate of an interaction, and can improve a user experience.SOLUTION: The method includes the step of waking up a smart back mirror and then confirming the current usage state of the smart back mirror (S201). Detecting the current operational state of the smart back mirror allows acquisition of the current usage state of the smart back mirror. The method further includes the step of exercising control so that the smart back mirror can interact with a user on the basis of the current usage state of the smart back mirror (S202). It is possible to set, in advance, a voice interaction between a corresponding smart back mirror and a user or a working interaction for different usage states of the smart back mirror.SELECTED DRAWING: Figure 2

Description

The present application relates to the field of artificial intelligence, and more particularly to smart rearview mirror interaction methods, devices, electronic devices and storage media.

With the development of vehicle intelligence, the number of in-vehicle electronic devices has gradually increased. The rear-view mirror has also evolved from the initial rear-view function to a smart rear-view mirror with an auxiliary reversal function, and the conventional smart rear-view mirror can perform voice interaction with the user, play music on the smart rear-view mirror, and To realize many functions such as map navigation.

In the prior art, the user can speak a wake word to wake up the smart rearview mirror, which either displays a voice assistant interface or speaks after recognizing the wake word spoken by the user. The playback method reminds the user to proceed to the next voice interaction. In the process of voice interaction between the user and the smart rearview mirror, the smart rearview mirror can recognize the meaning of the user's voice and perform movements such as music playback or map navigation.

However, in some application scenes, for example, when the vehicle is in a noisy environment, the user's voice is covered with the sound of the environment, the smart rearview mirror cannot recognize the user's voice, and moreover, accurate response operation is performed. become unable.

The present application provides smart rearview mirror interaction methods, devices, electronic devices and storage media that allow users to interact with the user using a portable interaction method, increasing the interaction success rate and further improving the user experience. ..

In the first aspect, the present application provides a method of interacting with the smart rear-view mirror, a step of determining the current usage state of the smart rear-view mirror after wake-up of the smart rear-view mirror, and the present state of the smart rear-view mirror. Includes a step of controlling the smart rearview mirror to interact with the user based on the usage status of the.

In this embodiment, on the premise that the interaction method with the user is increased, the problem that the user's voice cannot be recognized in a specific scene can be solved by the conventional voice interaction, and based on the current usage state of the smart rearview mirror. , Interact with users using a portable interaction method to increase the interaction success rate and further improve the user experience.

In a second aspect, the present application provides a smart rear-view mirror interaction device, a first processing module for determining the current usage state of the smart rear-view mirror after wake-up of the smart rear-view mirror. It includes a second processing module for controlling the smart rearview mirror to interact with the user based on the current usage state of the smart rearview mirror.

Regarding the beneficial effects of the smart rear-view mirror interaction device according to the second aspect, the beneficial effects of the first aspect can be referred to and will not be repeated here.

In a third aspect, the present application provides electronic devices, including at least one processor, memory, camera and audio output device, said memory storing computer execution instructions and said at least one processor storing in said memory. The computer execution instruction is executed so that the electronic device executes the interaction method of the smart back mirror according to the first aspect, and the camera collects the user's movement under the control of the processor and electronically. The purpose is to realize motion interaction between the device and the user, and the audio output device is for outputting audio under the control of the processor and realizing voice interaction between the electronic device and the user. Is.

In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, the first aspect described above. The smart back mirror interaction method of the above aspect is realized.

In a fifth aspect, the present application provides a computer program stored in a computer-readable storage medium, and when an instruction in the computer program is executed by a processor, the computer is equipped with the smart back mirror of the first aspect described above. Execute the interaction method.

The smart rear-view mirror interaction method, device, electronic device and storage medium according to the present application relate to the big data field. In the present application, it is possible to realize motion interaction with the user based on the front camera in the smart rear-view mirror, and on the premise that the interaction method with the user is increased, the user's voice is recognized in a specific scene by the conventional voice interaction. It can also solve the problem of not being able to. Moreover, in the present application, the smart rear-view mirror can also be controlled to interact with the user based on the current usage state of the smart rear-view mirror, whereby the smart rear-view mirror can be controlled to interact with the user. Based on the state, it is possible to interact with the user using a more portable interaction method, increasing the interaction success rate and further improving the user experience.

It should be understood that the content described in this section is not intended to mark the essential or important features of the embodiments of the present application and is not used to limit the scope of the present application. Other features of the present application will be facilitated through the following specification.

The drawings are used to better understand the present solution and do not constitute a limitation to the present application.
It is the schematic of the scene which applies the interaction method of the smart rearview mirror which concerns on this application. FIG. 1 is a flowchart 1 of an embodiment of an interaction method for a smart rear-view mirror according to the present application. FIG. 2 is a flowchart 2 of an embodiment of an interaction method for a smart rear-view mirror according to the present application. FIG. 1 is a schematic view 1 of an interaction between a smart rear-view mirror and a user according to the present application. FIG. 2 is a schematic view 2 of an interaction between a smart rear-view mirror and a user according to the present application. FIG. 3 is a schematic view of the interaction between the smart rear-view mirror and the user according to the present application. FIG. 5 is a schematic view of the interaction between the smart rear-view mirror and the user according to the present application. It is a structural schematic diagram of the interaction device of the smart rearview mirror which concerns on this application. It is a structural schematic diagram of the electronic device which concerns on this application.

Hereinafter, exemplary embodiments of the present application will be described with reference to the drawings, but various details of the embodiments of the present application are included for ease of understanding and are considered to be merely exemplary. Should be. Therefore, one of ordinary skill in the art should be aware that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, in the following description, well-known functions and structures will be omitted for clarity and brevity.

With the development of vehicle intelligence, the number of in-vehicle electronic devices has gradually increased. The user can realize functions such as entertainment and navigation by interacting with in-vehicle electronic devices such as an in-vehicle speaker and a smart rear-view mirror installed in the vehicle. In the prior art, the interaction method between the user and the in-vehicle electronic device may be a voice interaction, for example, the user speaks a wake word to wake up the in-vehicle electronic device, and further, the user's voice is used by the in-vehicle electronic device. To recognize, analyze the user's meaning, and execute the corresponding feedback movement. Illustratively, when the user speaks "play song A", the in-vehicle electronic device recognizes the meaning of the user's voice and further plays song A.

In the in-vehicle field in the conventional technology, the interaction method with the user is single, and there is only a voice interaction method. When the vehicle is in a noisy environment, using the voice interaction method makes it impossible to accurately recognize the user's voice and further makes it impossible to perform an accurate response movement, which requires the user to explain repeatedly, and the user needs to explain repeatedly. The experience may be bad. Based on the traditional vehicle, the method of interacting with the user, for example, if the motion interaction is increased, the vehicle must be equipped with a camera, and the camera can collect the user's movements, and Allows the corresponding response movement to be performed based on the user's movement. In order to attach a camera to a vehicle, it is necessary to attach a camera of a specific model number based on the vehicle model, the model number of the vehicle's central control motor, the operation system, etc., so that the application of motion interaction is limited. It should be noted that some vehicles may not support camera mounting, resulting in inability to achieve motion interaction with the user.

Currently, the development of smart rear-view mirrors is in full swing, and their functions are also developing from the original rear view function to the auxiliary reversal function and the operation recording function. It should be noted that the smart rear-view mirror in the present application is a smart rear-view mirror installed in the vehicle, and the smart rear-view mirror is equipped with a rear camera so that the driving data of the vehicle can be easily recorded. To. In order to solve the above problem, a front camera can be installed in the smart rearview mirror, and the front camera does not need to consider the vehicle model, the model number of the central control motor of the vehicle, the operation system, etc., and is universal. Highly sexual. It should be understood that the front camera of the smart rearview mirror is a camera that captures the inside of the vehicle, and the rear camera of the smart rearview mirror is a camera that captures the outside of the vehicle and the running screen of the vehicle.

Therefore, in order to solve the above technical problems, the present application provides an interaction method of a smart rear-view mirror, and controls a front camera installed in the smart rear-view mirror to collect user's movements. Increase the motion interaction between the smart rearview mirror and the user based on the voice interaction between the smart rearview mirror and the user, and increase the interaction method between the in-vehicle electronic device and the user in the technical solution of the present application. It is also possible to solve the problem of not being able to recognize the user's voice in a noisy environment by the conventional voice interaction, and improve the user experience.

FIG. 1 is a schematic view of a scene to which the smart rear-view mirror interaction method according to the present application is applied, and as shown in FIG. 1, the scene includes a user in a vehicle 10 and a smart rear-view mirror 11. In the smart rear-view mirror, a front camera for collecting the movement of the user in the vehicle and realizing the movement interaction between the user and the smart rear-view mirror is integrated. As should be understood, FIG. 1 shows a circular front camera on a smart rearview mirror.

It should be understood that the executing body of the smart rear-view mirror interaction method according to the present application may be a smart rear-view mirror interaction device, and the smart rear-view mirror interaction device is a smart rear-view mirror or a smart rear-view mirror. It may be a processor, a chip, or the like in a mirror. The front camera is integrated in the smart rear-view mirror.

Hereinafter, the smart rear-view mirror interaction method according to the present application will be described with reference to specific examples. It should be noted that some of the following examples can be combined with each other. FIG. 2 is a flowchart 1 of an embodiment of the smart rear-view mirror interaction method according to the present application. As shown in FIG. 2, the smart rear-view mirror interaction method according to the present embodiment can include the following steps.

S201, After waking up the smart rear-view mirror, the current usage state of the smart rear-view mirror is determined.

S202, Controls the smart rearview mirror to interact with the user based on the current usage state of the smart rearview mirror.

In S201 described above, after the smart rear-view mirror is woken up, the current usage state of the smart rear-view mirror can be determined. After wake-up of the smart rear-view mirror, it can be said that the smart rear-view mirror is in the wake-up state.

It should be noted that in this embodiment, the method in which the user wakes up the smart rearview mirror is not limited. Optionally, the user can wake up the smart rearview mirror by a method of voice interaction, for example, the user can wake up the smart rearview mirror by a wake word. Illustratively, for example, when the smart rearview mirror is turned on and then the standby state is entered, when it is detected that the user's voice contains a wake word, the wakeup state is entered. Optionally, the user can also wake up the smart rearview mirror by means of motion interaction. Illustratively, when turning on the smart rearview mirror, the front camera can be activated and controlled so that the front camera collects the user's movements, and the movements collected by the camera are determined to be wakeup movements. When you do, you will enter a wake-up state. The wake-up movement may be a love by hand, a continuous wink of a preset number of times, or the like. It should be understood that the wake word or wake-up movement that wakes up the smart rearview mirror can be customized by the user.

In this embodiment, the current operating status of the smart rear-view mirror can be detected, and the current usage status of the smart rear-view mirror is acquired. Alternatively, the current usage status of the smart rear-view mirror can be obtained by interacting with the sensor installed in the vehicle, and the sensor can detect the current usage status of the smart rear-view mirror.

The current usage state of the smart rear-view mirror in this embodiment may be the current operating state of the smart rear-view mirror, for example, the operating time of the smart rear-view mirror, the operating application program, and the like. Alternatively, the current usage state of the smart rear-view mirror may be a state of a vehicle having a smart rear-view mirror. For example, the vehicle is in a driving state or a stationary state, and the driving state is divided into a high-speed driving state and a low-speed driving state. Be done. It should be understood that depending on the current usage of the smart rearview mirror, the corresponding smart rearview mirror and the interaction method suitable for the user also differ.

In S202 described above, after the current usage state of the smart rearview mirror is determined, the smart rearview mirror can be controlled to interact with the user based on the current usage state of the smart rearview mirror. It should be understood that in the present embodiment, the interaction method between the smart rearview mirror and the user includes, but is not limited to, voice interaction and motion interaction. Correspondingly, controlling the smart rear-view mirror and the user to perform voice interaction means collecting the user's voice and performing an appropriate operation based on the user's voice. Controlling the smart rear-view mirror and the user to perform motion interaction, that is, controlling the camera of the smart rear-view mirror to collect the user's motion and performing an appropriate operation based on the user's motion. You can do it.

The interaction method between the corresponding smart rear-view mirror and the user, for example, voice interaction or motion interaction, can be preset for different usage states of the smart rear-view mirror. Further, in the present embodiment, after determining the current usage state of the smart rear-view mirror, it is determined that the smart rear-view mirror and the user perform voice interaction or motion interaction based on a preset correspondence relationship. Can be done.

Illustratively, when the current usage state of the smart rear-view mirror is the state of the environment with the smart rear-view mirror, the correspondence between the different states of the environment with the smart rear-view mirror and the interaction method can be preset. For example, in order to ensure the driving safety of the user, when the vehicle with the smart rear-view mirror is in a high-speed driving state, the interaction method between the smart rear-view mirror and the user is set to be the motion interaction method, and the vehicle is slow. When in the driving state, the interaction method between the smart rear-view mirror and the user can be set to be the voice interaction method.

In the smart rear-view mirror interaction method according to this embodiment, the front camera installed in the smart rear-view mirror can realize motion interaction with the user, and on the premise that the interaction method with the user is increased, the conventional voice is used. Interaction can also solve the problem of not being able to recognize the user's voice in a specific scene, improving the user experience. Moreover, in this embodiment, the smart rearview mirror can also be controlled to interact with the user based on the current usage state of the smart rearview mirror, making it more portable based on the current usage state of the smart rearview mirror. It interacts with the user using an interaction method to increase the interaction success rate and further improve the user experience.

Based on the above embodiment, in this embodiment, the current usage state of the smart rearview mirror is the state of the environment in which the smart rearview mirror is located, the type of interaction application currently provided to the user by the smart rearview mirror, or the smart. The rearview mirror may be the interaction content that currently performs voice interaction with the user. Below, the current usage of the Smart Rearview Mirror is the state of the environment in which the Smart Rearview Mirror is located, the type of interaction application currently provided to the user by the Smart Rearview Mirror, or the interaction in which the Smart Rearview Mirror currently engages in voice interaction with the user. The interaction method of the smart rear-view mirror according to this embodiment will be described from the angles of the three scenes of the content.

The state of the environment with the smart rearview mirror may be noisy or non-noisy, and the type of interaction application currently provided to the user by the smart rearview mirror is the display service type, audio or video service type or memory. The service type may be used, and the interaction content in which the smart rear-view mirror currently engages in voice interaction with the user may be voice interaction content, display interaction content, or the like. In this embodiment, in order to acquire the state of the environment in which the smart rearview mirror is present, it can be acquired by a method of interacting with a sensor in the vehicle. The state of the environment in which the smart rearview mirror is located can be obtained by interacting with a decibel meter or other sensor that detects environmental noise in the vehicle. In this embodiment, in order to obtain the type of interaction application currently provided to the user by the smart rear-view mirror and the interaction content in which the smart rear-view mirror currently performs voice interaction with the user, the content in which the smart rear-view mirror is currently activated is used. It can be obtained by detecting. For example, it can detect the application currently displayed in the smart rearview mirror and set the type of application to the type of interaction application currently provided to the user by the smart rearview mirror, or it is currently displayed in the smart rearview mirror. It is possible to detect what is present or what is being played by the smart rear-view mirror, and use that as the interaction content in which the smart rear-view mirror currently engages in voice interaction with the user.

In the first type of scene, the current usage state of the smart rear-view mirror is the state of the environment in which the smart rear-view mirror is located.

The smart rear-view mirror obtains the decibel value of the noise in the current environment, and based on the decibel value of the noise, the smart rear-view mirror can determine whether or not the environment is in a noisy state. Currently, when the environment is in a noisy state, the smart rear-view mirror determines that the user's voice cannot be recognized, determines that it is better to use motion interaction, and controls the smart rear-view mirror and the user to perform motion interaction. The smart rear-view mirror is determined to be able to accurately recognize the user's voice when the current environment is in a non-noisy state, and the smart rear-view mirror and the user should perform voice interaction to ensure the user's driving safety. Can be controlled.

Selectably, the vehicle in this embodiment is equipped with a decibel meter or other device for detecting decibels, and interacts with the decibel meter in the vehicle or another device for detecting decibels to be smart. The rearview mirror can get the decibel value of the noise in the current environment. Selectably, the functions of the decibel meter or other device that detects decibels may be integrated on the smart rear-view mirror, in order for the smart rear-view mirror to obtain the decibel value of noise in the current environment. used.

In this embodiment, the preset decibel threshold value can be set in advance, the preset decibel threshold value may be a decibel value that cannot recognize the user's voice, and the preset decibel threshold value can be obtained by a simulation experiment. Alternatively, the preset decibel threshold can be customized by the user. When the smart rearview mirror has a noise decibel value in the current environment greater than or equal to the preset decibel threshold, the smart rearview mirror can determine that the environment is in a noisy state, and the smart rearview mirror has a noise decibel in the current environment. When the value is less than the preset decibel threshold, the smart rearview mirror can determine that the environment is in a non-noisy state. Further, in this embodiment, the smart rearview mirror is determined to be in a noisy environment at present based on the decibel value of noise in the current environment, and further, the accuracy between the smart rearview mirror and the user is determined. Lay the foundation for establishing the interaction method.

In such a scene, ensure that the user can interact with the user using an accurate interaction method, and in a noisy environment, the user's voice cannot be accurately recognized, and the appropriate operation should be performed accurately. It can increase the success rate of interaction and improve the user experience, avoiding the problem of not being able to do it.

In the second type of scene, the current usage of the smart rearview mirror is the type of interaction application currently provided to the user by the smart rearview mirror.

The types of interaction applications currently provided to the user by the smart rearview mirror can include display service types, audio and video service types, storage service types, and the like. Display service type interaction applications can include weather query applications, shopping applications, social applications, navigation applications, vehicle parameter control applications, etc., and audio and video service type interaction applications are music applications, video applications, broadcast applications, etc. Storage service type interaction applications may include driving recording applications, image recording applications, and the like. It should be understood that vehicle parameter control applications can be used for controlling vehicle temperature, purifying vehicle interior air, and the like.

In this embodiment, the smart rearview mirror and the user can be controlled to perform voice interaction or motion interaction based on the type of interaction application currently provided to the user by the smart rearview mirror. Since different types of interaction applications have their own attributes, in this embodiment it is possible to determine a more suitable interaction method for the current interaction application and increase the interaction success rate based on the type of interaction application. it can.

It should be understood that audio and video service type interaction applications constantly play audio during operation, and the played audio may not be able to accurately recognize the user's voice.

In one possible embodiment, in this embodiment, the audio or video service type can be set as the first type, i.e., the first type of interaction application is an interaction application that provides an audio service to the user. is there.

In this embodiment, when the smart rear-view mirror detects that the type of interaction application currently provided to the user is the first type, the smart rear-view mirror and the user are controlled to perform motion interaction. If it is determined that the type of interaction application currently provided to the user by the smart rearview mirror is not the first type, the smart rearview mirror and the user are controlled to perform voice interaction. Audio service-type interaction applications should be motion-aware because the audio is constantly played while the audio service-type interaction application is running, and the played audio may not be able to accurately recognize the user's voice. In use, speech recognition can be used for interaction applications that are not of the audio service type, and the interaction success rate can be increased.

In order to ensure various user choices for the interaction method, in this embodiment, the smart back mirror and the user are controlled to perform motion interaction or voice interaction according to the user's interaction habits. By confirming, the user experience can be improved by interacting with the user using an interaction method that is more suitable for the user's habits.

In one possible embodiment, in the present embodiment, the user's preferred interaction with various types of interaction applications on the smart rearview mirror is acquired, and the interaction application currently provided to the user by the smart rearview mirror. After determining the type, the smart rearview mirror and the user can be controlled to perform the desired interaction of the type of interaction application. It should be understood that the preferred interaction is an interaction method used according to the user's habits or preferences, and the preferred interaction is a voice interaction or a motion interaction.

In this embodiment, the user's preferred interaction with the various types of interaction applications can be determined based on the historical interaction data between the user and the various types of interaction applications on the smart rearview mirror. Optionally, the historical interaction data may be interaction data within a preset period prior to the wakeup of the current smart rearview mirror. It should be understood that instead of using the historical interaction data long ago, it is the user's preference to use the interaction data within the preset period before the smart rearview mirror was woken up as the historical interaction data. This is to more accurately determine the recent favorite interaction, improve the accuracy of the confirmed user's favorite interaction, and further improve the accuracy of controlling the interaction method between the smart rearview mirror and the user.

In this embodiment, the method of determining the user's favorite interaction with various types of interaction applications based on the historical interaction data is based on the historical interaction data, the ratio of the motion interaction between the user and the various types of interaction applications, and Determine the percentage of voice interaction. It should be understood that, in this embodiment, the movement of the favorite interaction between the user and the interaction application, the number of motion interactions and the number of voice interactions are statistic, and the ratio of the number of motion interactions to the total number of interactions is the ratio of the motion interaction. Then, the ratio of the number of voice interactions to the total number of interactions can be used as the ratio of voice interactions. Each movement in the interaction process between the user and the interaction application can be regarded as one movement interaction, and each talk in the interaction process between the user and the interaction application can be regarded as one voice interaction.

When it is determined that the proportion of motion interaction of the second type of interaction application is greater than or equal to the proportion of voice interaction, it is determined that the user's preferred interaction with the second type of interaction application is motion interaction. If it is determined that the proportion of motion interaction of the second type of interaction application is less than the proportion of voice interaction, then it is determined that the user's preferred interaction with the second type of interaction application is voice interaction. The second type is any type of interaction application type in the smart rearview mirror, for example, the second type may be an audio or video service type, a display service type or a storage service type. ..

Illustratively, based on the historical interaction data of an audio or video service type interaction application, the percentage of motion interaction between the user and that type of interaction application is 80% and the percentage of voice interaction between the user and that type of interaction application. If is determined to be 20%, then the user's preferred interaction with the audio or video service type interaction application is determined to be the motion interaction. Based on the historical interaction data of the display service type interaction application, it is confirmed that the ratio of motion interaction between the user and the type of interaction application is 30%, and the ratio of voice interaction between the user and the type of interaction application is 70%. If so, it is determined that the user's preferred interaction with the display service type interaction application is the voice interaction.

In addition, after determining the user's preferred interactions for various types of interaction applications on the smart rearview mirror, the smart rearview mirror currently provides the user with the type of interaction application currently provided to the user by the smart rearview mirror. It is possible to determine the user's preferred interaction with the provided interaction application and further control the smart rearview mirror and the user to perform the preferred interaction. In this embodiment, based on the historical interaction data of various types of interaction applications on the smart rear-view mirror, the user's favorite interaction with various types of interaction applications is determined, thereby determining the user's favorite interaction. Establish the foundation to establish and further determine the exact interaction method.

In such scenes, the smart back mirror is more accurate and user-friendly, based on the type of interaction application currently provided to the user by the smart back mirror, or the user's preferred interaction with various types of interaction applications. It is possible to establish interaction with the user using an interaction method that is more suitable for the habit and improve the user experience.

In the third type of scene, the current usage state of the smart rear-view mirror is the interaction content in which the smart rear-view mirror currently engages in voice interaction with the user.

In this embodiment, it is possible to control the smart rear-view mirror and the user to perform voice interaction or motion interaction based on the content of the interaction in which the smart rear-view mirror currently performs voice interaction with the user. The interaction content in which the smart rear-view mirror currently engages in voice interaction with the user may be the content of the audio currently being played in the smart rear-view mirror. It should be understood that the audio currently being played in the smart rearview mirror may be the audio played in response to the user's voice or movement, such as "Are you sure you want to play this song?" Good. Alternatively, the audio currently being played in the smart rearview mirror is also the audio that is played by itself after turning on the smart rearview mirror, for example, "Are you planning a route to your home now?" There may be. It should be understood that the self-played audio may be predefined or user-configured. In the design, the method in which the user responds may differ depending on the interaction content, and in this embodiment, an interaction method more suitable for the user based on the interaction content can be used, and the user experience can be improved.

In one possible embodiment, in the present embodiment, the smart rear-view mirror controls the user to perform voice interaction or motion interaction based on the meaning of the interaction content in which the smart rear-view mirror currently interacts with the user. It can be confirmed. Specifically, the smart rearview mirror and the user perform voice interaction or motion interaction by determining whether or not the user is instructed to perform a confirmation, selection, or page turning operation based on the interaction content. Can be determined to control. When the interaction content instructs the user to perform a confirmation, selection, or page-turning operation, the smart rear-view mirror and the user are controlled to move and interact, otherwise the smart rear-view mirror and the user are voiced. Control to interact.

Illustratively, the content of the interaction that the smart rearview mirror currently has voice interaction with the user is "Are you sure you want to plan a route to your house from now on?" When instructed to, the smart rearview mirror and the user can be controlled to perform motion interaction. The user may perform the confirmation operation by moving the head, and specifically, the user can nod or shake the head.

Illustratively, the content of the interaction in which the smart rearview mirror currently engages in voice interaction with the user is "I found two songs. The first is song A and the second is song B." When instructed to perform a selection operation, the smart rearview mirror and the user can be controlled to perform motion interaction. The selection operation performed by the user may be a hand movement, and specifically represents a song selected by the number of extended fingers. For example, when one finger is extended, it indicates that song A is selected. , When two fingers are extended, it means that song B is selected.

Illustratively, the interaction content that the smart rearview mirror currently has a voice interaction with the user is "Enter your phone number on the next page" so that the user can turn the page according to the interaction content. When instructed to, the smart rearview mirror and the user can be controlled to perform motion interaction. The user operation of turning the page may be a hand movement, and specifically, it can be expressed by sliding the palm from right to left to turn the page.

In one possible embodiment, the interaction content in which the smart rear-view mirror in the present embodiment currently performs voice interaction with the user is marked with a label, and the label indicates the user's interaction method based on the interaction content. .. Illustratively, the interaction content that the smart back mirror currently interacts with the user is "Enter your phone number on the next page" and the interaction content label is "Voice interaction, motion interaction". In some cases, the label indicates that the user's interaction scheme based on the interaction content includes a voice interaction and a motion interaction, that is, the label indicates that the interaction scheme in which the user responds to the interaction content is a voice interaction. Instruct to include motion interaction.

In this embodiment, it is possible to control the smart rearview mirror and the user to perform voice interaction or motion interaction based on the interaction content and the label of the interaction content. The label of the interaction content can be controlled so that the smart rearview mirror and the user perform the motion interaction as long as there is a motion interaction. In other words, if the interaction content label contains motion interaction, the smart rearview mirror and the user are controlled to perform motion interaction, and if the interaction content label does not include motion interaction, the smart rearview mirror is used. Controls voice interaction with the user.

In such scenes, the smart rearview mirror may interact with the user using an interaction method that is more accurate and more appropriate to the user's habits, depending on the meaning or label of the interaction content in which the smart rearview mirror currently interacts with the user. It can be confirmed and the user experience can be improved.

Based on the above embodiment, the smart rear-view mirror interaction method according to the present application will be described in more detail below with reference to FIG. FIG. 3 is a flowchart 2 of an embodiment of the smart rear-view mirror interaction method according to the present application. As shown in FIG. 3, the smart rear-view mirror interaction method according to the present embodiment can include the following steps.

After wake-up of S301 and the smart rear-view mirror, the current usage state of the smart rear-view mirror is determined.

S302, Based on the current usage state of the smart rear-view mirror, when it is determined that the smart rear-view mirror and the user are controlled to perform motion interaction, the camera of the smart rear-view mirror is controlled to be opened, and the camera is used by the user. Control to collect movement.

S303, an appropriate operation is executed based on the movement of the user.

S304, based on the current usage state of the smart rear-view mirror, when it is determined that the smart rear-view mirror and the user are controlled to perform voice interaction, the user's voice is collected and the meaning of the voice is acquired.

S305, based on the meaning of the voice, perform the appropriate operation.

Regarding the embodiment in S301, the related description of Example S201 can be referred to, and without repeating the description here, in S302, the smart rearview mirror and the user are controlled to perform motion interaction. The related description of the above-described embodiment S202 can be referred to with respect to the method of determining the method and the method of determining that the smart rear-view mirror and the user are controlled to perform voice interaction in S304. I will not explain. As one to be understood, one of the above S302-S303 and S304-S305 is selected.

If it is determined in S302 above that the smart rear-view mirror and the user are controlled to perform motion interaction based on the current usage state of the smart rear-view mirror, the smart rear-view mirror and the user are to perform motion interaction. It can be controlled and specifically controlled to open the camera in the smart rearview mirror, and the camera can be controlled to collect user movements. It should be understood that in this embodiment, when the smart rearview mirror is activated, the camera of the smart rearview mirror can be controlled to open. Among them, if the smart rearview mirror and the user perform voice interaction within the preset period after the camera is opened, the camera can be controlled to enter a dormant state. In this embodiment, when it is determined that the smart rearview mirror and the user perform motion interaction, the camera is controlled to be opened, whereby the camera is controlled to collect the user's motion, and the power consumption of the camera is controlled. Can be reduced. It should be understood that when the camera controls to collect user movements, the user movements that need to be collected can be transmitted to the camera. User movements can include head movements, gesture movements, body movements, and the like. Head movements can include movements such as nodding, shaking the head, blinking eyes, and making a funny face. Gesture movements can include movements such as extension of several fingers, slide of the palm, OK by hand, and love by hand. The movement of the body may be a crossing of both hands, a fight pose, or the like.

In order to ensure the driving safety of the user, it can be determined that the movement of the user is set based on the running state of the vehicle. Selectably, when the vehicle is in a running state, the movement of the user is a head movement to ensure that both hands of the user are kept on the steering wheel. When the vehicle is in a non-running state, the user's movements include head movements and / or hand movements.

In S303 described above, the camera can be controlled to collect the user's movements, and the user's movement marks that need to be collected, that is, the preset movement marks can be transmitted to the camera. The preset movement mark is for expressing the preset movement. The camera can store the correspondence between the preset movement mark and the preset movement mark. The camera can determine the user's movements that need to be collected based on the preset movement marks, and after collecting the user's movements, the collected movements and the movements that need to be collected (ie, preset movements). ), And it can be determined whether or not it is the preset movement.

After confirming that the preset movements have been collected, the successful collection information and the user's movement mark (that is, the preset movement mark) can be transmitted to the smart rearview mirror interaction device, which is the smart rearview mirror interaction device. , The user's movement is determined to be a preset movement, and an appropriate operation is executed based on the user's movement mark. It should be understood that the smart rearview mirror interaction device can store the correspondence between preset motion marks and commands, and when receiving a successful collection response sent by the camera, the preset motion. Based on the correspondence between the mark and the command, the command corresponding to the movement can be acquired, and further, an appropriate operation can be executed based on the command.

Illustratively, Table 1 shows the correspondence between preset motion marks and commands stored in the interaction device of the smart rearview mirror.

If the smart rear-view mirror plays the audio "Are you planning a route to your home now?" And the audio tells the user to perform a confirmation operation, the camera in the smart rear-view mirror is the user's. It can be controlled to collect movements. The preset movement confirmed by the user may be a head movement, specifically, nodding or shaking the head. It is possible to send a mark of the user's movement, which is 1 or 2 that needs to be collected, to the camera. When the camera collects the user's nodding movement, it can be compared with the preset movement 1 or 2, and when the user's movement is determined to be the preset movement 1, the mark 1 of the preset movement and the successful collection information are smart. It can be transmitted to the rear-view mirror interaction device, and the smart rear-view mirror interaction device determines that the user's movement is preset movement 1, and performs an appropriate operation based on the command corresponding to the preset movement 1. Can be executed.

FIG. 4 is a schematic view 1 of the interaction between the smart rearview mirror and the user according to the present application. As shown in a in FIG. 4, as an example, the smart rear-view mirror plays the audio "Are you planning a route to your house now?", And the user performs a confirmation operation by the audio. The smart rear-view mirror camera can be controlled to collect nodding or shaking motions of the user. For example, as shown in a in FIG. 4, when the camera collects the nodding motion of the user, the mark of the nodding motion of the user can be transmitted to the interaction device of the smart rear-view mirror, and the interaction device of the smart rear-view mirror can be used. The nodding movement of the user is determined to be a preset movement, and the instruction corresponding to the nodding movement of the user and agreeing with the proposal is determined based on the correspondence between the mark of the preset movement and the instruction. Therefore, as shown in b in FIG. 4, the planned route can be displayed on the interface of the smart rear-view mirror.

Selectably, when performing the corresponding operation corresponding to the preset movement, the audio can be played, and the audio indicates that the corresponding operation is being performed. In such situations, by replaying the appropriate operation being performed, the user can be alerted to the movement being performed, without the user having to raise his head to look at the smart rearview mirror. Driving safety can be ensured. Illustratively, as shown in b in FIG. 4, for example, "I have planned a route to my house. I will be leaving soon, so please drive to road XX."

FIG. 5 is a schematic view 2 of the interaction between the smart rearview mirror and the user according to the present application. As shown in a in FIG. 5, for example, the smart rearview mirror plays the audio "I found two songs. The first is song A and the second is song B. Which do you want?" The audio can be controlled so that the camera in the smart rearview mirror collects the user's movements when instructing the user to select an operation. Among them, the preset movement selected by the user may be a hand movement, specifically, a movement of extending a finger, and the number of fingers represents the user's selection. As shown in a in FIG. 5, when the camera collects the movement of the user extending one finger, the camera can transmit the mark of the movement of the user extending one finger to the interaction device of the smart rearview mirror, and is smart. The rear-view mirror interaction device allows the user to determine that the movement of extending one finger is a preset movement, and one finger of the user based on the correspondence between the preset movement mark and the command. It can be determined that the user's selection is 1 for the command corresponding to the stretching motion, and as shown in b in FIG. 5, the song A is played by the command in accordance with the audio played by the smart rearview mirror. Since it was confirmed by instructing to, the song A can be played. Optionally, in this embodiment, as shown in b in FIG. 5, for example, "play song A" can be played, and an instruction is instructed to perform an appropriate operation. become.

FIG. 6 is a schematic view 3 of the interaction between the smart rearview mirror and the user according to the present application. As shown in a in FIG. 6, for example, the smart rearview mirror plays the audio "Enter your phone number on the next page", and the audio allows the user to turn the page. When instructed to do so, the camera in the smart rearview mirror can be controlled to collect the user's movements. The preset movement in which the user turns the page may be a hand movement, specifically, a left / right slide of the palm, and sliding the palm from right to left represents turning the page to the right. However, sliding the palm from left to right represents turning the page to the left. As shown in a in FIG. 6, when the camera collects the movement of the user's palm sliding from right to left, the camera transmits a mark of the movement of the palm sliding from right to left to the interaction device of the smart rearview mirror. The smart rear-view mirror interaction device ensures that the movement of the palm sliding from right to left is a preset movement, and the user's palm is right based on the correspondence between the preset movement mark and the command. Since it can be determined that the command corresponding to the movement of sliding from to the left is the command to turn the page to the right, the operation of turning the page to the right can be executed by the smart rearview mirror, which is shown in b in FIG. As described above, the interface of the smart rear-view mirror has changed to an interface for inputting a telephone number by a user. Optionally, in this embodiment, as shown in b in FIG. 6, and for example, "turning the page" or "turning the page" can be played, and the corresponding operation can be performed. Instruct them to be running.

It should be noted that when the smart rearview mirror camera collects the user's movement and compares it with the preset movement, if it is determined that the user's movement is not a preset movement, the smart rearview mirror's interaction device will receive the failed collection information. If it can be transmitted and it can be determined that the user's movement is not a preset movement after receiving the collection failure information, the smart rearview mirror and the user are controlled to perform voice interaction.

FIG. 7 is a schematic view of the interaction between the smart rearview mirror and the user according to the present application. As shown in a in FIG. 7, for example, the smart rearview mirror plays the audio "Enter your phone number on the next page", and the audio allows the user to turn the page. When instructed to do so, the camera in the smart rearview mirror can be controlled to collect the user's movements. If the user did not make a preset movement, or if the movement was not a palm-sliding page-turning operation, the camera could send the failed collection information to the smart rear-view mirror interaction device, at which time. , The smart rearview mirror and the user can be controlled to perform voice interaction. As shown in b in FIG. 7, the user can speak "turn to the next page" and further perform a page turning operation based on the user's voice. The interface of the mirror is an interface for inputting a telephone number by a user.

In S304 described above, when it is determined in the present embodiment that the smart rear-view mirror and the user are controlled to perform voice interaction based on the current usage state of the smart rear-view mirror, the smart rear-view mirror and the user are voiced. It is possible to control the interaction so that the user's voice can be collected and the meaning of the voice can be acquired.

In S305 described above, an appropriate operation is executed based on the meaning of the voice.

In this embodiment, it is possible to specifically refer to the voice interaction method in the conventional technique for the method of acquiring the meaning of the voice of the user and executing the corresponding operation. Illustratively, as shown in FIG. 7, when controlling the smart rearview mirror and the user to perform voice interaction, the user's voice, for example, "turn to next page", can be collected, FIG. As shown in b in the above, when the meaning of the voice is to instruct to turn the page, the corresponding operation of turning the page is executed.

It should be understood that in FIGS. 4-Fig. 7, when the smart rear-view mirror and the user move and interact with each other, the smart rear-view mirror is in an open state due to the white ring. This means that when the smart rear-view mirror and the user perform voice interaction, the smart rear-view mirror is a black ring, which means that the smart rear-view mirror is in a closed state.

In this embodiment, when the smart back mirror and the user are controlled to perform motion interaction, if the collected user's motion is a preset motion, an appropriate operation can be executed, and the collected user's motion can be executed. When the movement is not a preset movement, the smart back mirror and the user can be controlled to perform voice interaction, and in this embodiment, the smart back is based on the current usage state of the smart back mirror and the movement of the user. It can be determined that the mirror will perform the appropriate operation or perform voice interaction with the user, and moreover, the interaction method with the user has been increased by interacting with the user using an interaction method that is easier and more suitable for the user behavior. As a premise, the interaction success rate can also be increased.

FIG. 8 is a schematic structural diagram of the interaction device of the smart rear-view mirror according to the present application. As shown in FIG. 8, the interaction device 800 of the smart rear-view mirror includes a first processing module 801 and a second processing module 802.

The first processing module 801 is used to determine the current usage state of the smart rear-view mirror after wake-up of the smart rear-view mirror.

The second processing module 802 is used to control the smart rearview mirror to interact with the user based on the current usage state of the smart rearview mirror.

In one possible design, the first processing module 801 is specifically used to determine if the smart rearview mirror is currently in a noisy environment.

Correspondingly, the second processing module 802 specifically controls the smart rearview mirror to move and interact with the user when the environment is currently in a noisy state, and the smart rearview mirror. Mirrors are currently used to control smart rearview mirrors and users to engage in voice interaction when the environment is in a non-noisy state.

In one possible design, the first processing module 801 also obtains the noise decibel value of the environment in which the smart rearview mirror is currently located, and if the noise decibel value is greater than the preset decibel threshold, the smart rearview mirror is currently Used to determine that the environment is in a noisy state.

In one possible design, the first processing module 801 is specifically used to determine the type of interaction application currently provided to the user by the smart rearview mirror.

Correspondingly, the second processing module 802 specifically performs voice interaction or motion interaction between the smart rearview mirror and the user based on the type of interaction application currently provided to the user by the smart rearview mirror. Used to control.

In one possible design, the interaction application includes a music application, a navigation application or a vehicle parameter control application.

In one possible design, the second processing module 802 specifically determines that the type of interaction application currently provided to the user by the smart rearview mirror is the first type, the smart rearview mirror and the user. The first type of interaction application is that it is an interaction application that provides an audio service to the user, and that the type of interaction application currently provided to the user by the smart rearview mirror is When it is determined that it is not the first type, it is used to control the smart rearview mirror and the user to perform voice interaction.

In one possible design, the second processing module 802 is specifically based on the historical interaction data between the user and the various types of interaction applications on the smart back mirror, and the user's preference for the various types of interaction applications. The interaction of the user is determined and the favorite interaction is a voice interaction or a motion interaction, the user's favorite interaction with various types of interaction applications, and the interaction application currently provided to the user by the smart back mirror. Based on the type, it is used to determine the user's preferred interaction with the interaction application currently provided to the user by the smart back mirror, and to control the smart back mirror and the user to perform the preferred interaction. ..

In one possible design, the second processing module 802 specifically determines the proportion of motion interaction and the proportion of voice interaction between the user and various types of interaction applications based on historical interaction data. If it is determined that the proportion of motion interaction of the type of interaction application is greater than or equal to the proportion of voice interaction, then the user's preferred interaction with the second type of interaction application is determined to be motion interaction, and the second type is The second type, if it is determined that it is any one of the interaction application types in the smart back mirror, and that the proportion of motion interaction of the second type of interaction application is less than the proportion of voice interaction. Interaction Used to determine that the user's preferred interaction with the application is a voice interaction.

In one possible design, the first processing module 801 is specifically used to determine the content of the interaction in which the smart rearview mirror currently engages in voice interaction with the user.

Correspondingly, the second processing module 802 specifically causes the smart rearview mirror and the user to perform voice interaction or motion interaction based on the content of the interaction in which the smart rearview mirror currently performs voice interaction with the user. Used to control.

In one possible design, the second processing module 802 specifically determines, depending on the interaction content, whether or not the user is instructed to perform a confirmation, selection, or page-turning operation. , Used to control the smart rearview mirror and the user to perform motion interaction, and otherwise control the smart rearview mirror and the user to perform voice interaction.

In one possible design, the second processing module 802 specifically controls the smart rearview mirror and the user to perform voice or motion interaction based on the interaction content and the interaction content label. The label is used to indicate the user's interaction method based on the interaction content.

In one possible design, the second processing module 802 specifically controls the smart rearview mirror and the user to perform the motion interaction when the interaction content label contains the motion interaction. It is used to control the smart rearview mirror and the user to perform voice interaction when the interaction content label does not include motion interaction.

In one possible design, the second processing module 802 specifically controls the smart rearview mirror to open the camera, controls the camera to collect the user's movements, and is based on the user's movements. , Used to perform the appropriate operation.

In one possible design, specifically, when the movement is determined to be a preset movement, the second processing module 802 acquires a command corresponding to the movement and executes an appropriate operation based on the command. Used for

In one possible design, the second processing module 802 is also used to control the smart rearview mirror and the user to perform voice interaction if the movement is determined to be non-preset movement.

In one possible design, the second processing module 802 is specifically used to collect the user's voice, acquire the meaning of the voice, and perform appropriate operations based on the meaning of the voice. Ru.

In one possible design, the second processing module 802 is further used to play the audio when performing the appropriate operation and to instruct the audio to perform the appropriate operation.

In one possible design, when the vehicle is in a running state, the user's movement is a head movement, and when the vehicle is in a non-running state, the user's movement includes a head movement and / or a hand movement.

The smart rear-view mirror interaction device according to the present embodiment will not be repeatedly described here because the principle and technical effect to be realized are similar to the above-mentioned smart rear-view mirror interaction method.

According to the embodiments of the present application, the present application further provides electronic devices and readable storage media. FIG. 9 is a schematic structural diagram of the electronic device according to the present application. The electronic device may be a smart rear-view mirror. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the realization of the present application described and / or desired herein. ..

As shown in FIG. 9, the electronic device includes one or more processors 901, memory 902, and an interface for connecting each component having a high speed interface and a low speed interface. The components are interconnected by different buses and may be installed on a common motherboard or, if desired, in other ways. The processor outputs instructions executed in memory or in electronic devices, including instructions stored in memory, so that GUI graphic information can be displayed on an external input / output device (for example, a display device coupled to an interface). Can be processed. In other embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple electronic devices can be connected, and each device provides some required operating. In FIG. 9, one processor 901 is taken as an example.

The memory 902 is a non-temporary computer-readable storage medium according to the present application. Instructions that can be executed by at least one processor are stored in the memory, and the at least one processor is made to execute the smart rear-view mirror interaction method according to the present application. The non-temporary computer-readable storage medium of the present application stores computer instructions, which are used to cause a computer to execute the smart rear-view mirror interaction method according to the present application.

The memory 902 is a non-temporary computer-readable storage medium that can be executed by a non-temporary software program such as a program instruction / module corresponding to the smart back mirror interaction method in the embodiment of the present application, or a non-temporary computer. It may be used to store programs and modules. Processor 901 executes various functional applications and data processing of the server by executing non-temporary software programs, instructions and modules stored in memory 902, ie, the smartback in the embodiment of the above method. Realize a mirror interaction method.

The memory 902 may include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required for at least one function, and the storage data area is a smart back mirror. It is possible to store data created for the use of electronic devices to execute the interaction method of. The memory 902 can also include fast random access memory and also includes non-temporary memory such as at least one magnetic disk storage device, flash memory device, or other non-temporary solid state storage device. You can also do it. In some embodiments, the memory 902 can selectively include memory that is remotely located relative to the processor 901, which remote memory implements a smart rearview mirror interaction method over the network. Can be connected to electronic devices for Examples of the above networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

Electronic devices for the interaction method of the smart rearview mirror can further include an input device 903, an audio output device 904, a camera 905 and a display device 906. The camera 905 may be a front-view camera in a smart rear-view mirror, and the camera 905 is used to collect user movements under the control of processor 901 and perform camera movements in the above embodiment. The audio output device 904 is used to output audio under the control of the processor 901 and realize interaction with the user, and the display device 906 is used for an interface of an electronic device, for example, the above. It is used to display the interface of the smart rearview mirror shown in FIGS. 4 to 7.

The processor 901, the memory 902, the input device 903, the audio output device 904, the camera 905 and the display device 906 can be connected by a bus or other method, and in FIG. 9, the connection by a bus is taken as an example.

The input device 903 can receive the input numerical or character information and generate user settings of the electronic device for executing the interaction method of the smart back mirror, and input of key signals related to function control. Input devices such as touch screens, keypads, mice, trackboards, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, etc. The display device 906 can include a display device, an auxiliary lighting device (LED or the like), a tactile feedback device (vibration motor or the like), or the like. The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, or a plasma display. In some embodiments, the display device may be a touch screen.

The systems and various technical embodiments described herein are in digital electronic circuit systems, integrated circuit systems, application specific ASICs (ASICs), computer hardware, firmware, software, and / or combinations thereof. realizable. These various embodiments may include: running on one or more computer programs, the one or more computer programs running on and / or on a programmable system including at least one programmable processor. Alternatively, the programmable processor may receive data and instructions from the storage system, at least one input device, and at least one output device, and transfer the data and instructions to the storage system, at least one input device, and. It may be a dedicated or general purpose programmable processor capable of transmitting to at least one output device.

These computer programs (also called programs, software, software applications, or code) include programmable processor machine instructions and utilize high-level process and / or object-oriented programming languages and / or assembly / machine languages. These computer programs can be implemented. As used in the text, the terms "machine readable medium" and "computer readable medium" are any computer program products, devices used to provide machine instructions and / or data to programmable processors. , And / or devices (eg, magnetic disks, optical disks, memories, programmable logic devices (PLDs), including machine-readable media that receive machine instructions that are machine-readable signals. "Machine-readable signals." The term "" refers to any signal used to provide machine instructions and / or data to a programmable processor.

In order to provide interaction with the user, the computer may implement the systems and techniques described herein, which computer may be a display device (eg, a CRT) or a display device for displaying information to the user. It is equipped with an LCD (liquid crystal display) monitor) and a keyboard and a pointing device (mouse, trackball, etc.), and a user can provide input to a computer through the keyboard and the pointing device. Other types of devices are also used to provide interaction with the user, eg, the feedback provided to the user is any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback). It is also possible to receive input from the user in any format (including sound input, voice input, or tactile input).

The systems and technologies described herein are computer systems that include back-end components (eg, data servers), computer systems that include middleware components (eg, application servers), or computer systems that include front-end components (eg, graphical user interfaces). Or any user computer that is equipped with a web browser and allows the user to interact with system and technology examples in the rough user interface or web browser), or any such back-end component, middleware component, or front-end component. It may be implemented in combination and the components of the system may be interconnected via digital data communication of any form or medium (such as a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

The computer system may include a client and a server. Clients and servers are generally separated from each other and usually interact over a communication network. Create a client-server relationship by running computer programs that have a client / server relationship with each other on the corresponding computer.

According to one embodiment of the present application, the present application further provides a computer program stored in a computer-readable storage medium, and when an instruction in the computer program is executed by a processor, the smartback of the embodiment of the present application is performed on the computer. Execute the mirror interaction method.

As you should understand, you can sort, add and remove steps using the various forms of processes shown above. For example, each step described in the present application may be performed in parallel, sequential, or different order as long as the desired result of the technical solution disclosed in the present application can be achieved, and is not limited herein.

The specific embodiments described above do not limit the scope of the patents of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and replacements can be made, depending on design requirements and other factors. All amendments, equivalent substitutions, improvements, etc. made without departing from the spirit and principles of the present application should be included in the claims of the present application.

Claims (22)

It ’s a smart rearview mirror interaction method.
After waking up the smart rearview mirror, the step of determining the current usage state of the smart rearview mirror and
A method of interacting with a smart rearview mirror, comprising: controlling the smart rearview mirror to interact with a user based on the current state of use of the smart rearview mirror. The step of determining the current usage status of the smart rearview mirror is
The smart rearview mirror includes a step to determine if the environment is currently in a noisy state.
Correspondingly, the step of controlling the smart rearview mirror to interact with the user based on the current usage state of the smart rearview mirror is
The smart rear-view mirror currently has a step of controlling the smart rear-view mirror and the user to move and interact with each other when the environment is in a noisy state.
The method according to claim 1, wherein the smart rear-view mirror includes a step of controlling the smart rear-view mirror and a user to perform voice interaction when the environment is not currently in a noisy state. The step to determine if the smart rearview mirror is currently in a noisy environment is
The step of acquiring the decibel value of noise in the environment where the smart rearview mirror is currently present,
The method of claim 2, wherein the smart rearview mirror comprises a step of determining that the environment is currently in a noisy state when the decibel value of the noise is greater than the preset decibel threshold. The step of determining the current usage status of the smart rearview mirror is
Including the step of determining the type of interaction application currently provided to the user by the smart rearview mirror.
Correspondingly, the step of controlling the smart rearview mirror to interact with the user based on the current usage state of the smart rearview mirror is
A claim comprising a step of controlling the smart rearview mirror and the user to perform a voice interaction or a motion interaction based on the type of interaction application currently provided to the user by the smart rearview mirror. The method according to 1. The interaction application
The method according to claim 4, wherein the method includes a music application, a navigation application, or a vehicle parameter control application. Based on the type of interaction application currently provided to the user by the smart rearview mirror, the step of controlling the smart rearview mirror and the user to perform voice interaction or motion interaction is a step.
When it is determined that the type of interaction application currently provided to the user by the smart rear-view mirror is the first type, it is a step of controlling the smart rear-view mirror and the user to perform a motion interaction. A step in which the first type of interaction application is an interaction application that provides an audio service to the user.
If it is determined that the type of interaction application currently provided to the user by the smart rearview mirror is not the first type, the smart rearview mirror and the user are controlled to perform voice interaction. The method according to claim 4 or 5, wherein the method is characterized by the above. Based on the type of interaction application currently provided to the user by the smart rearview mirror, the step of controlling the smart rearview mirror and the user to perform voice interaction or motion interaction is a step.
A step of determining a user's preferred interaction with various types of interaction applications based on historical interaction data between the user and various types of interaction applications on the smart rearview mirror, wherein the preferred interaction is , Voice interaction, or motion interaction, and
The user's preferred interaction with various types of interaction applications, and said with respect to the interaction application currently provided to the user by the smart rearview mirror, based on the type of interaction application currently provided to the user by the smart rearview mirror. Steps to determine the user's preferred interaction,
4. The fourth or fifth aspect comprises the step of controlling the smart rear-view mirror and the user to perform the desired interaction of the interaction application currently provided to the user by the smart rear-view mirror. The method described. The step of determining the user's preferred interaction with the various types of interaction applications based on the historical interaction data between the user and the various types of interaction applications on the smart rearview mirror.
A step of determining the proportion of motion interaction and the proportion of voice interaction between the user and various types of interaction applications based on the historical interaction data.
When it is determined that the proportion of motion interaction of the second type of interaction application is greater than or equal to the proportion of voice interaction, it is determined that the user's preferred interaction with the second type of interaction application is motion interaction, and the first A step in which type 2 is any one of the interaction application types in the smart rearview mirror.
If it is determined that the proportion of motion interaction of the second type of interaction application is less than the proportion of voice interaction, then the step of determining that the user's preferred interaction with the second type of interaction application is voice interaction. The method according to claim 7, wherein the method comprises. The step of determining the current usage status of the smart rearview mirror is
The smart rearview mirror includes a step of determining the content of the interaction that currently performs voice interaction with the user.
Correspondingly, the step of controlling the smart rearview mirror to interact with the user based on the current usage state of the smart rearview mirror is
The first aspect of the present invention includes a step of controlling the smart rear-view mirror and the user to perform voice interaction or motion interaction based on the content of the interaction in which the smart rear-view mirror currently performs voice interaction with the user. The method described. Based on the content of the interaction in which the smart rear-view mirror currently engages in voice interaction with the user, the step of controlling the smart rear-view mirror and the user to perform voice interaction or motion interaction is
A step of determining whether or not the user is instructed to perform a confirmation, selection, or page-turning operation based on the interaction content.
If so, the step of controlling the smart rearview mirror and the user to perform motion interaction,
The method of claim 9, wherein if not, the smart rearview mirror comprises a step of controlling the user to perform voice interaction. Based on the content of the interaction in which the smart rear-view mirror currently engages in voice interaction with the user, the step of controlling the smart rear-view mirror and the user to perform voice interaction or motion interaction is
Based on the interaction content and the label of the interaction content, the smart rearview mirror and the user are controlled to perform voice interaction or motion interaction, and the label allows the user to perform an interaction method based on the interaction content. The method of claim 9, wherein the method comprises instructing steps. The step of controlling the smart rearview mirror and the user to perform voice interaction or motion interaction based on the interaction content and the label of the interaction content is
When the label of the interaction content contains a motion interaction, a step of controlling the smart rearview mirror and the user to perform the motion interaction, and
11. The method of claim 11, wherein the label of the interaction content includes a step of controlling the smart rearview mirror and the user to perform voice interaction when the label does not include motion interaction. The step of controlling the smart rearview mirror and the user to perform motion interaction is
A step of controlling the camera of the smart rear-view mirror to open and controlling the camera to collect the movement of the user.
The method according to any one of claims 2 to 12, wherein a step of executing an appropriate operation based on the movement of the user is included. The step of performing an appropriate operation based on the user's movement is
When it is determined that the movement is a preset movement, the step of acquiring the command corresponding to the movement and the step
13. The method of claim 13, wherein the method comprises performing the appropriate operation based on the instruction. The method also
The method according to claim 13 or 14, wherein when it is determined that the movement is not a preset movement, a step of controlling the smart rearview mirror and the user to perform voice interaction is included. The step of controlling the smart rearview mirror and the user to perform voice interaction is
The step of collecting the voice of the user and acquiring the meaning of the voice,
The method according to any one of claims 2 to 15, wherein a step of performing an appropriate operation based on the meaning of the voice is included. The method further
Any one of claims 13 to 16, wherein when the corresponding operation is performed, the audio is played back and the audio includes a step indicating that the corresponding operation is being performed. The method described in. When the vehicle is in a running state, the movement of the user is a head movement.
The method according to any one of claims 13 to 16, wherein when the vehicle is in a non-traveling state, the movement of the user includes a head movement and / or a hand movement. It is an interaction device for smart rearview mirrors.
After wake-up of the smart rear-view mirror, a first processing module used to determine the current usage state of the smart rear-view mirror, and
An interaction device for a smart rearview mirror, comprising: a second processing module used to control the smart rearview mirror to interact with a user based on the current usage state of the smart rearview mirror. .. It ’s an electronic device,
With at least one processor
A memory communicatively connected to the at least one processor,
Including a camera and an audio output device,
An instruction that can be executed by the at least one processor is stored in the memory, and when the instruction is executed by the at least one processor, the at least one processor has claims 1 to 18. The method described in any one of the above items can be executed, and the camera collects the user's movement under the control of the processor and realizes the movement interaction between the electronic device and the user. An electronic device, wherein the audio output device outputs audio under the control of the processor and realizes a voice interaction between the electronic device and a user. A non-temporary computer-readable storage medium in which computer instructions are stored, wherein the computer instructions are for causing a computer to perform the method according to any one of claims 1 to 18. A non-temporary computer-readable storage medium characterized by. A computer program, which, when executed by a processor, realizes the method according to any one of claims 1 to 18.

Images