WO2023016126A1

WO2023016126A1 - Terminal device, server, and multi-device collaboration login method

Info

Publication number: WO2023016126A1
Application number: PCT/CN2022/102579
Authority: WO
Inventors: 薛健; 李金昆
Original assignee: 聚好看科技股份有限公司
Priority date: 2021-08-11
Filing date: 2022-06-30
Publication date: 2023-02-16
Also published as: CN117652134A

Abstract

The present application discloses a terminal device, a server, and a multi-device collaboration login method. The terminal device comprises a controller, configured to: receive user login information input by a user, so that the terminal device is used as a master device, the user login information corresponding to a specified account; send an account login request to a server on the basis of the user login information to request master token information used for login, wherein the master token information is generated by the server on the basis of success in verification of the user login information, and the master token information comprises an account ID used for identifying the specified account; receive the master token information from the server, and log in to the corresponding specified account on the basis of the account ID in the master token information; and send the master token information to a slave device, so that the slave device sends an automatic login request to the server on the basis of the master token information to request slave token information used for login and logs in to the corresponding specified account on the basis of the account ID in the slave token information, the slave device being in a pairing relationship with the master device.

Description

Terminal device, server and multi-device cooperative login method

This application claims priority to Chinese patent applications filed on August 11, 2021 with application numbers 202110917713.4 and 202111093415.4 filed on September 17, 2021, the entire contents of which are incorporated by reference into this disclosure.

technical field

The present application relates to the technical field of device cooperative login, and in particular to a terminal device, a server and a multi-device cooperative login method.

Background technique

With the popularization of terminal devices, there may be multiple terminal devices in a user's home, such as remote controllers, televisions, refrigerators, and air conditioners. In the home, various terminal devices in the home, such as lighting equipment, home appliances, security equipment, etc., can be connected to the server through the IoT module, and users can realize home appliance control, lighting control, anti-theft alarm, etc. through the mobile terminal Function.

In order to obtain a rich functional experience, each device needs to log in to the user account to achieve the best use requirements.

At present, in the home, when the user wants to control a terminal device, he can send a voice command to the smart speaker, and the smart speaker sends the voice command to the server, and the server controls the corresponding terminal device according to the voice command.

Contents of the invention

According to some embodiments of the present application, there is provided a terminal device, including a controller configured to: receive user login information input by a user to make the terminal device act as a master device, the user login information corresponds to a specified account; based on User login information and send an account login request to the server to request the primary token information for login, the primary token information is generated by the server based on the verification of the user login information, the primary token The card information includes an account ID for identifying the specified account; receiving the primary token information from the server, and logging into the corresponding specified account based on the account ID in the primary token information; The card information is sent to the slave device, so that the slave device sends an automatic login request to the server based on the master token information to request the slave token information for login and the corresponding specified account based on the account ID in the slave token information, so The slave device is in a pairing relationship with the master device.

According to some embodiments of the present application, there is provided a terminal device, including a controller configured to: receive the main token information sent by the main device after logging in to a specified account based on the user login information input by the user, so that the terminal device can be used as From the device, the master token information is generated by the server based on the verification of the user login information sent by the master device and sent to the master device, and the master token information includes information for Identifying the account ID of the specified account; sending an automatic login request to the server based on the primary token information to request secondary token information for logging in, the secondary token information being the information obtained by the server based on the primary token information The verification of the card information is generated by passing the verification, and the slave token information includes the account ID; receiving the slave token information from the server, based on the account ID in the slave token information, logging in to the corresponding specified account.

According to some embodiments of the present application, a multi-device cooperative login method is provided, which is applied to a master device, wherein the method includes: receiving user login information input by a user, the user login information corresponding to a specified account; An account login request is sent to the server to request primary token information for login, the primary token information is generated by the server based on the verification of the user login information, and the primary token information includes An account ID for identifying the specified account; receiving the primary token information from the server, and logging into a corresponding specified account based on the account ID in the primary token information; sending the primary token information To the slave device, so that the slave device sends an automatic login request to the server based on the master token information to request the slave token information for login and log in the corresponding specified account based on the account ID in the slave token information, the slave device is in a pairing relationship with the master device.

According to some embodiments of the present application, a multi-device cooperative login method is provided, which is applied to a slave device, wherein the method includes: receiving the master token information sent by the master device after logging in to a specified account based on the user login information input by the user; The terminal device is used as a slave device, the master token information is generated by the server based on the verification of the user login information sent by the master device and sent to the master device, the The primary token information includes an account ID for identifying the designated account; based on the primary token information, an automatic login request is sent to the server to request secondary token information for logging in, and the secondary token information is Generated by the server based on passing the verification of the primary token information, the secondary token information includes the account ID; receiving the secondary token information from the server, based on the secondary token information in the secondary token information Account ID, log in to the corresponding specified account.

According to some embodiments of the present application, a multi-device cooperative login method is provided, including: a first terminal device receives user login information input by a user to make the first terminal device act as a master device, and the user login information corresponds to a specified account; the main device sends an account login request to the server based on the user login information to request primary token information for login; the server responds to the account login request carrying the user login information sent by the main device, Verify the information, and generate primary token information after the verification is passed, the primary token information includes an account ID for identifying the specified account; the server sends the primary token information to the primary device; The master device receives the master token information from the server, and logs in the corresponding designated account based on the account ID in the master token information; the master device sends the master token information; the second terminal device acts as a slave device And receive the master token letter from the master device; the slave device sends an automatic login request to the server based on the master token information to request the slave token information for logging in; the server responds to the slave token information sent from the device An automatic login request, verifying the primary token information carried in the automatic login request, generating secondary token information based on the primary token information after the verification is passed, and sending the secondary token information to the secondary device; The device receives the slave token information from the server, and logs in a corresponding designated account based on the account ID in the slave token information.

Description of drawings

Fig. 1 is a schematic diagram of an operation scene between an intelligent control device and a control device according to some embodiments;

FIG. 2 is a block diagram of a hardware configuration of an intelligent control device 200 according to some embodiments;

FIG. 3 is a block diagram of a hardware configuration of a control device 100 according to some embodiments;

Fig. 4 is a software configuration diagram in the intelligent control device 200 according to some embodiments;

Fig. 5 is an interactive diagram of establishing a pairing relationship between multiple devices according to some embodiments;

Fig. 6 is a flowchart of a multi-device coordinated login method performed by a master device according to some embodiments;

Fig. 7 is an interaction diagram of a multi-device cooperative login method according to some embodiments;

Fig. 8 is a flowchart of a multi-device cooperative login method performed by a slave device according to some embodiments;

Fig. 9 is a flowchart of a multi-device cooperative login method performed by a server according to some embodiments;

Fig. 10 is an application scenario diagram of a terminal device according to some embodiments;

Fig. 11 is another application scenario diagram of a terminal device according to some embodiments;

Fig. 12 is a schematic flowchart of a voice control method according to some embodiments;

Fig. 13 is a schematic flow diagram of screening terminal devices according to some embodiments.

Detailed ways

Some embodiments of the present application will be described below with reference to the drawings in some embodiments of the present application. Apparently, some of the described embodiments are only some of the embodiments of the present application, not all of them.

Fig. 1 is a usage scenario of an intelligent control device according to some embodiments. As shown in FIG. 1 , the intelligent control device 200 may be a display device, and also communicates with the server 400 , and the user may operate the intelligent control device 200 through the smart device 300 or the control device 100 .

In some embodiments, the control device 100 may be a remote control, and the communication between the remote control and the intelligent control device includes at least one of infrared protocol communication, Bluetooth protocol communication, and other short-distance communication methods, and is controlled by wireless or wired Intelligent control device 200. The user can control the intelligent control device 200 by inputting at least one user instruction through buttons on the remote control, voice input, and control panel input.

In some embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a notebook computer, an AR/VR device, and the like.

In some embodiments, the smart device 300 can also be used to control the smart control device 200 . For example, the smart control device 200 is controlled using an application program running on the smart device. In some embodiments, the smart device 300 and the smart control device may also be used for data communication.

In some embodiments, the intelligent control device 200 can also be controlled in a manner other than the control device 100 and the intelligent device 300, for example, the voice instruction module configured inside the intelligent control device 200 can directly receive the user's voice The command control can also be controlled by receiving the user's voice command through the voice control device installed outside the intelligent control device 200 .

In some embodiments, the intelligent control device 200 also performs data communication with the server 400 . The intelligent control device 200 can be allowed to communicate via local area network (LAN), wireless local area network (WLAN) and other networks. The server 400 may provide various contents and interactions to the smart control device 200 .

Fig. 2 is a block diagram of hardware configuration of the control device 100 according to some embodiments. As shown in FIG. 2 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply. The control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the intelligent control device 200 can recognize and respond to, and acts as an intermediary interaction between the user and the intelligent control device 200 .

In some embodiments, the communication interface 130 is used for communicating with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module. In some embodiments, the user input/output interface 140 includes at least one of a microphone, a touch pad, a sensor, a button, or an alternative module.

Fig. 3 is a block diagram of a hardware configuration of an intelligent control device 200 according to some embodiments. 3, in some embodiments, the intelligent control device 200 includes a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 285, a memory, and a power supply , at least one of the user input interface 280 . In some embodiments, the controller includes a CPU, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for presenting images, and a drive component for driving image display, for receiving image signals output from the controller, and displaying video content, image content, and menu manipulation interface. Components and user manipulation of the UI interface, etc. In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box wait. In some embodiments, the controller 250 controls the work of the intelligent control device and responds to user operations by storing various software control programs in the memory. The controller 250 controls the overall operation of the smart control device 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the user input interface 280 is an interface that can be used to receive control input (such as: physical buttons on the body of the smart control device, or others).

Fig. 4 is a software configuration diagram in the intelligent control device 200 according to some embodiments. Referring to Fig. 4, in some embodiments, the system is divided into four layers, from top to bottom are respectively the application (Applications) layer (abbreviated as "application layer"), application framework (Application Framework) layer (abbreviated as "framework layer") "), Android runtime (Android runtime) and system library layer (referred to as "system runtime layer"), and the kernel layer.

In some embodiments, an operating system is configured in the display device. In order to obtain a richer functional experience from the operating system, it is usually necessary to log in with a system account first. There may be multiple terminal devices in the user's home, such as smart remote controllers, TVs, refrigerators, and air conditioners.

In order to obtain a rich functional experience, each device needs to log in with a user account. At present, the commonly used login method is that each device needs to log in with an account separately, for example, the user actively scans the QR code on each device to log in. This SSO (single sign-on) method needs to be based on the user's initiative to trigger different devices to log in to the account. This login method is not convenient enough, resulting in low login efficiency and poor user experience.

In order to improve user experience, different terminal devices in the home environment can share the same account. After the user uses one of the devices to log in to the account, the other devices can automatically log in without human intervention. Therefore, some embodiments of the present application provide a terminal device and a server, in which multiple devices can log in to an account cooperatively, that is, after a terminal device (such as a display device, or a non-display device) logs in to an account, other devices also automatically log in. The same account can not only provide convenience to users, but also improve user experience. In some embodiments, other devices may also be display devices, such as various devices with display screens, such as televisions, mobile phones, refrigerators, etc., or may also be non-display devices. Users can only log in to an account on one device, and according to the habits defined by the user, they can automatically log in to multiple devices on the device they want to log in to, achieving smart login and improving user interaction experience.

The embodiment of the present application provides a display device, such as the display device with the configurations shown in FIG. 1 to FIG. 4 , where the display device is a master device, that is, a device that performs account login first. In some other implementation manners, the display device may serve as a slave device. The display device includes: a display configured to present a user interface; and a controller connected to the display. In order to realize the collaborative account login between multiple devices, a pairing relationship needs to be established between each display device in the home environment. For this purpose, the controller is further configured to perform the following steps:

Step 1. Start the Bluetooth module configured on the local end.

In some embodiments, during the system startup process of the display device, the bluetooth module is started along with the startup process, and the user login is performed after the system startup is completed.

In some embodiments, the bluetooth module configured at the local end may also be activated after the user logs in. In some embodiments, some terminal display devices will disable the Bluetooth function by default, so the terminal display device can choose to enable the Bluetooth function through the setting menu at any time before or after the user logs in, and then activate the Bluetooth module configured locally.

Step 2. Establish a pairing relationship with the slave device based on the Bluetooth module to transmit data with the slave device through the Bluetooth module. Wherein, the Bluetooth module configured from the device is in the startup state, and the slave device is registered with a monitoring token (token) event, and monitors the token The event is used to monitor the token event sent by the slave device to the master device.

Each display device for collaborative login in the home environment is equipped with a Bluetooth module. Generally, in the state of the Internet of Things network, the Bluetooth module is configured to be turned on by default or must be turned on forcibly. Therefore, when each display device is initialized, start Each display device has its own bluetooth module, and each display device realizes the pairing relation connection between itself and other devices through the bluetooth module. After the pairing connection is made, data transmission can be performed between each device. Data transmission includes communication data transmission and token data transmission. Communication data transmission is used to ensure communication between devices. Token data transmission can receive master token information sent by the master device from the device.

In some embodiments, the display device can also manually turn on or turn off the Bluetooth function according to requirements. After the Bluetooth function is turned off, it cannot be paired with other devices through the Bluetooth module. After the Bluetooth function is turned on, it can be paired with other devices through the Bluetooth module. .

Multi-device collaborative login is applicable not only to various display devices in the home environment, but also to various display devices in classrooms, shopping malls and other places. Each display device meets the distance connection requirements, or is located in the same local area network.

Fig. 5 is an interaction diagram for establishing a pairing relationship between multiple devices according to some embodiments. See the example shown in part (a) of Figure 5.

S101. The main device is a TV, and the slave devices are a Rubik's Cube screen and a refrigerator screen, and the Bluetooth modules of the TV, the Rubik's Cube screen, and the refrigerator screen are activated in sequence.

In the above examples, the Rubik's Cube screen is a portable terminal device with a display screen and a Bluetooth module, and the refrigerator screen refers to a refrigerator equipped with a display screen and a Bluetooth module.

S102. Through the respective Bluetooth modules, the Bluetooth pairing connection between the TV and the Rubik's Cube screen, the Bluetooth pairing connection between the TV and the refrigerator screen, and the Bluetooth pairing connection between the Rubik's Cube screen and the refrigerator screen are sequentially realized.

S103. After completing the pairing relationship connection among the multiple devices, each device registers the monitoring token event for each connected display device in response to the establishment of the pairing connection.

Monitoring the token event is used to monitor the token event sent by the master device, and the token event can be the master token information sent by the master device.

In some embodiments, in a home environment, the display device that performs account login based on user input is the master device, and the remaining display devices that implement the automatic login function through the login of the master device are slave devices.

In some embodiments, in order to improve system performance, only the monitoring of the token event of the master device by the slave device is reserved. For example, if the user selects the TV as the master device to log in the first account, the monitor token event of the TV is canceled, and the Rubik’s Cube screen The monitoring token event of the refrigerator screen is reserved, that is, the master device is not in the monitoring state, and other slave devices are in the monitoring state. The master device that is not in the listening state has the function of receiving token data and sending token data, and the slave device in the listening state only has the function of receiving token data, but not the function of sending token data, and the communication data can be normal transmission.

After receiving the master token information sent by the master device to log in to the corresponding account, the slave device no longer sends token information to other connected devices, but can send related data for Bluetooth connection between devices. The Bluetooth connection between the slave device in the listening state and other devices will not be affected. For example, if a new display device is started from the home environment, and the Bluetooth module of the device is activated, the slave device in the listening state can still communicate with the display device. The new device makes a Bluetooth connection.

Since one of the devices will send master token information to other paired devices after logging in to the account, the slave device in the listening state is only configured to receive token data and send/receive communication data functions, but not to send tokens The data function can prevent one of the slave devices from sending the master token information to other slave devices after receiving the master token information sent by the master device to achieve account login, thereby avoiding data transmission confusion and affecting other slave devices. Account login efficiency. That is to say, each slave device can respond to the master token information and communication data sent by the master device that logged in the account first, and respond to the communication data sent by other slave devices, but cannot send token information to other devices.

In some embodiments, if the above-mentioned three devices (TV, Rubik's cube screen and refrigerator screen) are not logged into a certain account, after the three devices start the Bluetooth module and perform Bluetooth pairing and connection with each other, all Register and monitor token events for the three devices respectively, and the three devices monitor each other. At this point, the data transmission between the three devices is only normal communication data. However, if a certain device is prioritized for account login and is determined to be the master device, the monitoring token event of the master device will be canceled and switched to the non-monitoring state. In this state, the master device has the function of transmitting communication data and token data at the same time. The monitoring token events of the other two devices are reserved, and they are determined to be slave devices, and both continue to be in the monitoring state. The two slave devices only have the function of communication data transmission. Therefore, the server can determine which device is the master device of the priority login account based on monitoring the state change of the token event, and inform other devices that there is already a master device of the login account.

In some embodiments, the listening status of each display device can be switched in real time according to the identity of the display device itself. For example, when the display device is the master device, the display device is not in the listening state; when the display device is the slave device, the display device is in the listening state.

In some embodiments, after each display device is paired and connected through its respective Bluetooth module, the user can preferentially log in the account on one of the display devices, while the other display devices can log in automatically, realizing the cooperative login account of multiple devices Purpose.

In some embodiments, after each display device is paired and connected through its own Bluetooth module, it can automatically log in to other display devices according to the account login information on one display device, so as to realize the purpose of multi-device cooperative login account. For example, when each display device does not turn on the Bluetooth module, that is, when no pairing connection is established, if a certain display device has a priority login account, and then turns on the Bluetooth module to establish a pairing connection with other devices, the main device of the priority login account can still be based on The account login information sends token information to other slave devices, so that each slave device can automatically log in. That is to say, in the scenario where a certain display device logs in the account first and then turns on Bluetooth pairing, other slave devices that have not logged in the account can still achieve automatic login.

Fig. 6 is a flowchart of a multi-device coordinated login method performed by a master device according to some embodiments; Fig. 7 is an interaction diagram of a multi-device coordinated login method according to some embodiments. Specifically, after the pairing connection is completed, each display device realizes the multi-device cooperative login account, that is, when performing the multi-device cooperative login method shown in FIG. 6 and FIG. 7, the controller in the main device is configured to perform the following steps: :

S11. Receive user login information input by the user, the user login information corresponds to a specified account, and send an account login request to the server based on the user login information to request primary token information for login, and the primary token information is the specified account. generated by the server based on passing the verification of the user login information, and the primary token information includes an account ID used to identify the specified account. When the server determines that the username and password in the login information are consistent with the stored registered username and password corresponding to the main device ID, the verification of the user login information passes.

In a home environment, there are usually multiple display devices, and the user can select one of the display devices as the primary device for first logging in to the account.

S201. A feasible way for the main device to implement account login is to display a login QR code on the display of the main device, and after the user scans the login QR code on the display through the smart terminal, a login interface is displayed on the smart terminal.

The login interface presents a user name input box and a password input box. The user enters a user name in the user name input box and a password in the password input box to obtain user login information for account login of the master device. Wherein, the smart terminal may be a device with a scanning function such as a mobile phone and a Pad. In some embodiments, after successful login, a successful login interface may pop up on the display, prompting, for example, that account A has successfully logged in. Another feasible way for the main device to implement account login is to directly present the login interface on the display of the main device. After the user enters the user name and password, the user login information is obtained to log in the account of the main device. The two ways for the user to input the user login information may be determined according to actual conditions, and other input ways may also be used, which are not limited here.

Referring to the content shown in part (a) of FIG. 7, the user login information includes a user name and password, etc., and the user login information corresponds to a designated account. After the user enters the user login information, the main device requests the server to verify the user login information, and the account login can only be realized after the verification is passed.

At this time, the main device generates an account login request based on the user login information input by the user, and sends it to the server to access a login (login) interface of the server.

S202. After receiving the account login request sent by the master device, the server can verify the user login information carried in the account login request.

The account login request carries user login information, a user ID and a device ID (that is, the main device ID), the user ID is used to identify the user, and the device ID is used to identify the device that implements account login.

The database in the server stores account information generated when the user registers an account, and the account information includes a registered user name and a registered password. Therefore, when verifying, the server compares the user name and password in the user login information with the registered user name and password stored in the database configured by the server. If they are consistent, it means that the user name and password entered by the user are correct and the account can be logged in. .

After the user login information verification is passed, in order to facilitate account login on the main device side, the server will generate corresponding main token information (token information). Token is a string of numbers, including user ID, device ID, account ID, and token creation. time and duration. Using the master token information, the master device and the slave device can realize the specified account corresponding to the login user login information, where the user ID is used to identify the user, the device ID is used to identify the device that implements the account login, and the account ID is used to identify the account (user name and password), the token creation time is the starting point when the token becomes effective, and the duration is the effective duration of the token. For example, if the token was created on September 1, 2021, and the corresponding token duration is 3 days, then the validity period is from September 1, 2021 to September 3, 2021. Among them, the token creation time is calculated from the time when the server verifies the user login information.

S203. After verifying and passing the user login information based on the account login request sent by the master device, the server obtains the user ID and device ID carried in the account login request, obtains the account ID corresponding to the account information from the server database, and Generate the token creation time and duration, based on the user ID, device ID, account ID, token creation time and duration, then generate the main token information, and return it to the main device to realize the account login of the main device.

In some embodiments, if there is a new device with a newly turned on Bluetooth module for pairing connection after the main device logs in with the account, then after pairing and connection with the new device, in response to the Bluetooth connection of the new device, the main device according to the established Login account information, and request the server to return the primary token information. At this time, the primary token information only updates the token creation time and duration, and the rest remains unchanged.

S12. Receive primary token information from the server, and log in to a corresponding specified account based on the account ID in the primary token information.

After the server receives the request from the master device, it can request to return the master token information to the master device through the HTTP interface. Due to the uniqueness of the HTTP request, only the requester can receive the returned information. The master device receives the master token information, and when it is determined that the device ID included in the master token information is consistent with the device ID of the master device, the master device can perform account login based on the account ID in the master token information.

S13. Send the master token information to the slave device, so that the slave device sends an automatic login request to the server based on the master token information to request the slave token information for login and the account ID based on the slave token information Log in the corresponding designated account, and the slave device is in a pairing relationship with the master device.

In the home environment, after one of the display devices first logs in with an account, in order to automatically log in the accounts of other display devices in the home environment, the master device can log in with the specified account or establish a new Bluetooth connection with other slave devices. The main token information used to realize the login of the specified account is automatically sent to each slave device through the Bluetooth module. The slave device refers to other display devices in the home environment except for the first login to the specified account.

If there are five display devices in the home environment, and the user selects display device A as the main device to log in with an account, then after display device A completes the account login, display device A sends display devices to display device B, display device C, and display device respectively through the Bluetooth module. Device D and display device E send master token information, wherein display device B, display device C, display device D, and display device E are all slave devices.

After each slave device receives the master token information sent by the master device through the bluetooth module, it can request the slave token information from the server based on the master token information. Specifically, the slave device generates an automatic login request based on the master token information and the slave device ID corresponding to the slave device and sends it to the server to request the slave token information; after the server receives the master token information sent by the slave device Verify the primary token information, and generate secondary token information according to the primary token information and the secondary device ID and send it to the secondary device after the verification is passed. The slave device logs in the specified account corresponding to the user login information based on the account ID in the slave token information. At this time, the master device and the slave device log in with the same designated account, so that after the user logs in on one device, he can automatically log in on other devices in the family, which is not only convenient for the user to log in, but also improves the user experience.

In some embodiments of the present application, the process for the server to verify the primary token information includes: after the server receives the primary token information sent from the device, decrypts the primary token information according to the salt value; Obtain the main device ID, account ID, token creation time, and valid duration; determine whether the main token information has expired, if it is determined that the main token information has expired, then determine that the verification of the main token information has failed; if it is determined that the main token information has not expired, Then it is determined that the verification of the primary token information is passed.

The contents of the master token information and the slave token information are different except that the user ID and account ID are the same, that is, the device ID, token creation time and duration will be updated. For a specific implementation process of the slave device automatically logging in to the same designated account based on the master token information sent by the master device, reference may be made to the contents of steps S21 to S23 provided in subsequent embodiments.

In some embodiments, after each display device completes an account login, the account will usually log out automatically after a period of time. For example, if the validity period of the token expires, then the user needs to log in to the account again on the corresponding display device during subsequent use.

The account for re-login of the display device may be the specified account for the previous login, or the second account that needs to be logged in currently. For example, if a slave device has logged in a specified account, and in the subsequent login, the slave device switches to log in to the second account, it is necessary to notify the master device that logs in to the specified account, and inform the master device that it does not need to send the specified account to the slave device. Corresponding primary token information.

To this end, when the login account is switched from the device, the controller is further configured to perform the following steps:

When the account logged in from the device is the second account, receiving a monitoring cancel instruction generated by the device;

In response to the listen cancel command, stop sending master token information to the slave device.

When the login account of the slave device is switched from the specified account to the second account, the slave device will cancel the token event monitoring of the master device that logs in to the specified account, that is, the slave device generates a monitoring cancel command and sends it to the master device that logs in to the specified account .

After the master device receives the monitoring cancellation instruction sent by the slave device, it will no longer send the master token information for logging in to the specified account to the slave device, so that the slave device can receive the master token information sent by the new master device logging in to the second account .

In some embodiments, when both the master device and the slave device that have logged in the specified account in the first round need to log in again, each device is in a monitoring state. If the user chooses another device to log in to the second account, the master device that originally logged into the specified account does not have priority to log in to the account in this round, so it no longer serves as the master device to send information, but switches to be a slave device. In this scenario, the newly switched slave device, together with other slave devices, responds to the token information sent by the device that preferentially logs in to the second account in this round, and realizes the login of the second account.

In some embodiments, when both the master device and the slave device have logged into the specified account in the first round, if the user operates any slave device to input user login information to replace the login account with the second account, at this time, the login The slave device of the second account is used as the master device for the second round of account login, and its monitoring state for the first round of the master device is canceled and switched to the non-monitoring state. In some embodiments, the new master device (the original first-round slave device) logged in with the switching account may generate a monitoring cancellation instruction. Since there will be two master devices logged in with different accounts in this scenario, the second round of new master devices (the original first round of slave devices) that switch accounts to log in can send a monitoring cancel command to the first round of master devices, so that Inform the master device of the first round that there is no need to send the master token information of the first round to the new master device. In some embodiments, the new master device may refuse to receive the master token information sent by the master device in the first round.

In the scenario of logging in to the second account, other slave devices (other slave devices in the first round) will still receive the token information sent by the new master device. At this time, a prompt interface can be displayed on the display screen of the slave device for The user chooses to continue to log in to the original designated account or switch to log in to the second account. If a slave device generates an instruction to log in to the second account, the slave device generates a monitoring cancellation command and sends it to the first round of the master device to cancel the monitoring of the token information of the first round of the master device. At this time, the The slave device will monitor the token information of the new master device in the second round to realize the automatic login of the second account. If a slave device generates an instruction to continue to log in to the original designated account status, the slave device generates a monitoring cancellation command and sends it to the new master device in the second round, canceling the monitoring of the token information of the new master device in the second round , at this time, the slave device will continue to monitor the token information of the master device in the first round, and continue to maintain the login status of the specified account.

In some embodiments, the display device logged into the second account may be other display devices in the home environment. For example, in the first round of account login, display device A is the master device, display device B and display device C are slave devices, and the three display devices are all registered with the specified account.

Since each device may have to re-login to the account, in the second round of account login, the user selects display device B to log in to the second account, and the identity of display device B changes from the original slave device to the master device, while the display device A and display device C are slave devices. At this time, display device B belonging to the identity of the master device sends token information for logging in the second account to the slave devices (display device A and display device C), so that display device A and display device C log in to the second account.

After the display device B logs in to the second account, its identity changes from the original slave device to the master device. Therefore, in order to facilitate the display device B to send the master token information to the remaining slave devices, the monitoring token event of the display device B is cancelled. , in the unlistened state, so that the display device B has the function of receiving/sending token data. Then restore the monitoring token event of the display device A, and it is in the monitoring state, so that the display device A only has the function of receiving token data and receiving/sending communication data, but not the function of sending token data. At this time, the display device B is not in the listening state, and both the display device A and the display device C are in the listening state.

After the display device C as the slave device receives the master token information sent by the display device B as the master device in the second round, if the account logged in by operating the display device C is also switched from the original designated account to the second account, the login After the account is switched, the display device C correspondingly generates a monitoring cancel command, and sends it to the display device A that originally logged in with the specified account, and informs the display device A that was originally the master device of the first round, to stop sending the specified account to the original slave device (display device C) Corresponding primary token information.

In some embodiments, the display device logged in with the second account may also be the original main device, that is, the main device originally logged in with the specified account. At this time, the identities of the master device and the slave device are the original configuration. Then, when switching to log in to the second account, you can directly refer to the login process of the specified account. Each slave device does not need to cancel the monitoring token event, that is, there is no need to generate a monitoring cancel command. . During this process, the master device is always in a non-monitoring state, and each slave device is always in a listening state.

A display device provided in an embodiment of the present application generates an account login request based on the user login information input by the user when the master device that first logs in to the account performs multi-device collaborative login, sends it to the server, and receives the server's request for the user to log in to the account. The master token information generated after the login information verification is passed, logs in the specified account corresponding to the account ID in the master token information, and at the same time, automatically sends the master token information to the slave device with a pairing relationship, so that the slave device is based on the master token After the information requests the slave device information from the server, log in the same specified account based on the account ID in the slave device information. After the user first logs in the account on one of the display devices, the other display devices can log in automatically, without the need for the user to log in manually, achieving the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.

According to some embodiments, a multi-device cooperative login method performed by a slave device is provided. A display device provided in an embodiment of the present application is a slave device, that is, a device that performs automatic account login based on the master token information sent by the master device. The display device includes: a display configured to present a user interface; a controller connected to the display, and when the slave device executes a multi-device cooperative login method, the controller is configured to perform the following steps:

S21. Receive the main token information sent by the main device after logging in to the specified account based on the user login information input by the user. The main token information refers to the information generated by the server after the main device requests the server to pass the verification based on the user login information. The primary token information includes user ID, primary device ID, account ID, token creation time and duration.

In a home environment, each display device is paired and connected through its own bluetooth module. During account login, if the user selects one of the display devices as the first device to log in the account, then the device is the master device. When the main device logs in to the specified account (such as account A) based on the user login information entered by the user, it will request verification from the server and receive the primary token information generated by the server after the verification is passed, and then the primary device logs in to the specified account based on the primary token information , while automatically sending the master token information to each slave device.

After each slave device receives the master token information sent by the master device, it can automatically log in the corresponding designated account based on the master token information, so that the slave device and the master device log in to the same account.

S22. Based on the user ID, master device ID, account ID, token creation time and duration in the slave device ID and master token information, an automatic login request is generated and sent to the server. The automatic login request is used to instruct the server to verify the slave device ID Check with the primary token information, and generate secondary token information based on the primary token information after the verification is passed. The secondary token information includes user ID, account ID, secondary device ID, updated token creation time and duration.

Since the various display devices in the home environment are paired and connected to each other through the Bluetooth module, and the initial Bluetooth password of many Bluetooth devices is generally 0000 or 1234, and even some Bluetooth pairing does not require a password, just long press it, this will appear The display devices of other users are also connected to the Internet of Things of the local user's home environment through their own Bluetooth modules, causing the display devices of other users to automatically log in to the account used by the local user, resulting in information leakage.

Therefore, when the slave device receives the master token information sent by the master device and performs account login, it needs to verify the information with the server, and the specified account can be logged in only after the verification is passed. At this time, the slave device will generate an automatic login request with its own slave device ID and the user ID, master device ID, account ID, token creation time and duration in the master token information, and send it to the server.

The server responds to the automatic login request sent by the slave device, and verifies the slave device ID and master token information carried in the automatic login request. The verification content includes checking whether the slave device allows automatic login, whether the slave device to perform automatic login is a preset device, and whether the master token information is valid. Preset devices refer to display devices in the same home environment, such as display devices in the same Internet of Things. This is because TV devices in the same Internet of Things generally belong to the same user.

In some embodiments, the preset device can also be a device bound to an account, for example, according to the account information in the slave device ID and the master token information, query the slave device ID in the pre-stored mapping relationship between the account and the device Whether there is a binding relationship with the account information, if it exists, the validity of the primary token is judged, and if it does not exist, the automatic login of the device is rejected. In this way, the user device can bind the account and the device ID through the initial login or other terminal settings, and then in the local area communication network connected by Bluetooth, when there is a non-user device, it can also prevent the non-user device from automatically logging in to the user. account.

The main token information includes user ID, main device ID, account ID, token creation time and duration, and the main device ID is used to identify the source of the main token information. Therefore, it is possible to check whether the slave device allows automatic login based on the user ID and account ID, judge whether the slave device currently requiring automatic account login is a preset device based on the slave device ID, and judge the master token based on the token creation time and duration information is valid. The above three judging processes need to pass the verification of the previous judging process before the next judging process can be executed, and the server will generate slave token information only after the verification of the three judging processes is passed.

In some embodiments, when the server checks whether the slave device currently requesting automatic account login is a preset device, another method can also be used: when the slave device uploads the master token information, the slave device ID is also required, and the server After verifying that the slave device can realize the automatic account login function, the server parses out the account information based on the account ID from the master token information, and then obtains a list of user devices bound to the account information or logged in. Judging whether the slave device ID is in the device list, if it is, then the verification is passed, and then the verification of the validity of the master token information is performed; if not, the slave device is rejected to perform the automatic login process. Based on the comparison between the slave device ID and the device list that has logged in to the account, if the slave device is another user's device, since the other user's device cannot log in to the account, there will be no other user's device identification in the device list , Therefore, if the comparison fails, it can avoid the automatic login account caused by the bluetooth connection of the outsider's device. In addition, in order to make the identification of each display device in the home Internet of Things exist in the device list, in some embodiments, the user can actively log in to the account once in the new device, so that the device ID of the new device can be added to the device corresponding to the account in the server. list, or the user can select the device ID of a new device to add to the user device list by operating the main device.

After the server checks that the slave device can perform an automatic login process based on the master token information, that the slave device is a preset device, and that the master token information is valid, it can be determined that the verification of the master token information passes. Wherein, for the specific implementation process of the server performing the primary token information verification, reference may be made to steps S31 to S32 and related contents provided in subsequent embodiments.

After verifying that the primary token information carried in the automatic login request sent by the slave device is passed, the server will generate secondary token information for realizing the login of the specified account by the slave device. Since the slave token information is updated based on the master token information sent by the master device, the role of the slave token information is for the slave device to log in with an account, and the slave device needs to log in with the same specified account as the master device. Therefore, the user ID and account ID in the master token information sent by the master device do not need to be updated, but the device ID used to identify the slave device and the token creation time and duration for the slave device to log in within the time limit need to be updated.

The update content includes replacing the device ID of the original master device in the master token information with the device ID of the slave device, and updating the token creation time and duration, so that the updated slave token information includes user ID, slave device ID, Account ID, new token creation time and new duration. The starting point of the new token creation time is calculated from the time when the server verifies the primary token information.

After the server verifies that the master token information carried in the automatic login request sent by the slave device has passed, it will return the synchronously generated slave token information to the slave device, so that the slave device can log in to the specified account based on the slave token information.

S23. Receive the slave token information from the server, and log in the corresponding designated account based on the account ID in the slave token information.

Referring to the content shown in parts (b) and (c) of Figure 7, when the master device is a TV, and the slave devices are Rubik's cube screen and refrigerator screen.

S204. The TV sends the main token information required for logging in to the specified account (such as account A) to the Rubik's Cube screen and the refrigerator screen respectively through the Bluetooth module.

S205. The Rubik's Cube screen (or refrigerator screen) requests the server through the primary token information and the secondary device ID in exchange for the secondary token information required by its own device to log in to the account.

S206. After receiving the slave token information returned by the server, the Rubik's Cube screen (or refrigerator screen) can log in the specified account.

At this time, the TV, Rubik’s Cube and Refrigerator are all logged into the same account A, the difference is that the TV is manually logged in by the user, while the Rubik’s Cube and Refrigerator are automatically logged in without the user’s manual login.

In the display device provided by the embodiment of the present application, when the slave device as an automatic login account performs multi-device cooperative login account, the slave device receives the master token information sent by the master device, generates an automatic login request and sends it to the server, and the server receives Check the master token information and generate slave token information based on the slave device information after passing the verification. The slave device logs in the same designated account as the master device based on the slave token information. After the user first logs in the account on one of the display devices, the other display devices can log in automatically, without the need for the user to log in manually, realizing the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.

In some embodiments, after completing an account login, both the master device and the slave device will cache the corresponding token information. That is, after the master device implements the specified account login based on the user login information entered by the user, it will store the corresponding master token information in the local cache of the master device; the slave device will automatically log in the designated account based on the master token information sent by the master device After that, the corresponding slave token information will be stored in the local cache of the slave device.

The cached token information can facilitate the login of the corresponding account by directly using the token information in the cache in the subsequent login process. Therefore, after completing an account login, the controller is further configured to: cache the slave token information returned by the server for implementing the login of the specified account after logging in with the master device with the same specified account.

After the slave device logs in the same specified account corresponding to the master token information sent by the master device, the slave token information returned by the server for realizing the login of the designated account is stored in the local cache of the slave device.

Referring again to the content shown in part (b) of Figure 5, when the master device (TV) logs into account A, the TV sends the master token information corresponding to account A to the slave devices (the Rubik's Cube screen and the refrigerator screen). After receiving the primary token information corresponding to account A, the Rubik’s cube screen and refrigerator screen exchange their own secondary token information from the server, and then realize automatic login of account A based on the secondary token information.

S104 , after the Rubik's Cube screen and the refrigerator screen log in to the account A, store the slave token information for account A login in their respective local caches.

In some embodiments, after each display device completes an account login, usually the account will automatically log out after a period of time. Then, when the user uses it subsequently, the corresponding display device needs to log in to the account again.

The account for re-login of the display device may be the specified account for the previous login, or a second account that needs to be logged in currently, and the second account is different from the specified account.

In order to judge whether the master token information received from the master device when the slave device automatically logs in to the account is the master token information corresponding to the original login to the specified account, the slave device needs to receive the new master token information when it receives the new master token information. Compare with the slave token information used to log in the specified account in the local cache, and determine whether the current account that needs to be logged in is the specified account for the original login through the comparison result, and determine the target account that needs to be logged in currently.

In some embodiments, when re-logging into the account subsequently, the controller is further configured to perform the following steps:

When receiving the second master token information sent by the master device, it is judged whether there is slave token information required for logging in the specified account in the local cache.

If the slave token information does not exist in the local cache, log in the corresponding second account based on the second master token information.

If there is slave token information in the local cache, check whether the account ID in the slave token information is consistent with the account ID in the second master token information.

If the account IDs are consistent, log in to the specified account corresponding to the account ID based on the slave token information.

If the account IDs are inconsistent, the corresponding target account is logged in based on the user's selection operation, and the target account includes one of the specified account or the second account.

When the slave device receives the second master token information sent by the master device, it determines whether the slave token information corresponding to the previous login account is stored in the local cache. In the later account login process, the second primary token information refers to the Nth primary token information sent by the master device, N≥2, which can be the same as the primary token information sent by the aforementioned master device, and is used for login correspondence The specified account; it can also be different, it is used to log in to other accounts.

In some embodiments, if the slave token information is not obtained in the local cache of the slave device, it means that the current account login process of the slave device is the first account login process, then the received second master token information is The master token information sent by the master device described earlier.

Therefore, the process of judging whether there is slave token information in the local cache can be applied in the process of step S21, that is, when the slave device receives the master token information sent by the master device, it judges whether there is a specified account for logging in in the local cache. required slave token information. When it is judged that there is no secondary token information in the local cache, the process of steps S22 to S23 is continued to complete the first account login process. That is to say, the process of logging in the second account by the slave device based on the second primary token information can refer to the implementation content of steps S21 to S23.

In some embodiments, if the slave token information used to log in to the specified account can be obtained in the local cache of the slave device, it means that the current account login process of the slave device is the Nth login process, that is, the slave device has performed N -1 account login process, which belongs to the post-login account process. Then the received second primary token information is the Nth primary token information sent by the master device.

When there is slave token information in the local cache of the slave device, it is necessary to determine whether the slave device needs to perform account switch login, that is, to judge whether the second account corresponding to the second master token information and the specified account corresponding to the slave token information are the same account. At this time, the comparison of judging whether the accounts are the same can be realized through the account ID.

If the account ID in the slave token information is consistent with the account ID in the second master token information, it means that the account to be logged in corresponding to the second master token information currently received from the slave device is the same as the account that has been logged in before. the same one.

At this time, if it is confirmed according to the token creation time and duration in the slave token information in the local cache that the slave token information is still within the creation time, the slave device does not need to exchange for a new token from the server based on the second master token information again. From the token information, the designated account can be directly logged in based on the slave token information in the local cache, thereby improving account login efficiency. If it is confirmed that the slave token information is invalid according to the token creation time and duration, the slave device still needs to request the server to update the slave token information based on the slave token information before logging in with the specified account.

If the account ID in the secondary token information is inconsistent with the account ID in the second primary token information, it means that the account that needs to be logged in has not been logged in before, and account switching and login are required. At this time, since the second account that the slave device currently needs to log in is different from the account in the local cache, the corresponding target account can be logged in based on the user's selection operation, and the target account includes one of the specified account or the second account.

In some embodiments, the controller is further configured to:

When the account IDs are inconsistent, a prompt interface is generated and displayed in the user interface. The prompt interface includes the specified account option corresponding to the secondary token information and the second account option corresponding to the second primary token information.

In response to the user triggering the login instruction of the specified account option, the corresponding specified account is logged in based on the secondary token information.

In response to the user triggering the login instruction of the second account option, the corresponding second account is logged in based on the second primary token information.

When the slave device needs to perform account switching and login, the user can perform a selection operation.

S105. When the account ID corresponding to the second primary token information is inconsistent with the account ID stored in the local cache of the slave device, present a prompt interface on the display of the slave device for user operation.

In some embodiments, if the user does not select an account to be logged in within a predetermined period of time, the slave device can automatically log in to the corresponding second account based on the second primary token information, or it can also be set so that the slave device can automatically log in to the corresponding second account based on the slave token information. The designated account corresponding to the card information login.

In some embodiments, when the account IDs are inconsistent, the specified account option and the second account option are displayed on the prompt interface. The specified account option is used to implement the login of the specified account corresponding to the secondary token information, and the second account option is used to implement the login of the second account (such as account B) corresponding to the second primary token information.

If the user still wants to log in from the device with the specified account that has been logged in before, the specified account option is triggered, and a corresponding login instruction is generated at the same time. Based on the login instruction, the slave device logs in the corresponding specified account according to the slave token information in the local cache. Since the slave token information in the local cache of the slave device is the slave token information required by the slave device to request the server for its own login account based on the master token information, if the token creation time and duration in the slave token information After confirming that the slave token information is still within the creation time, the slave token information can directly realize the login of the specified account. Therefore, the slave device can directly log in to the specified account based on the slave token information in the local cache without requesting to the server. information from the token.

However, if it is confirmed that the slave token information is invalid according to the token creation time and duration in the slave token information, the slave device still needs to exchange for a new token from the server based on the user ID, account ID and slave device ID in the slave token information. After logging in with the specified account after the secondary token information, the updated content of the new secondary token information is the token creation time and duration.

S106. If the user wants to log in a new account from the device, trigger the second account option, and at this time, generate a corresponding login instruction.

Based on the login instruction, the slave device exchanges brand new slave token information from the server according to the second master token information, and logs in the corresponding second account (such as account B) with the second master token information. For the implementation process of the slave device logging into the corresponding second account based on the second master token information, reference may be made to the content of the foregoing steps S22 to S23.

In some embodiments, if the slave device changes its login account during the subsequent account login process, that is, it switches from the original designated account to the second account, it needs to notify the master device that logged in the designated account to inform the master The device no longer needs to send the master token information corresponding to the specified account to the slave device.

Therefore, when the account is switched from the device to log in, the controller is further configured to perform the following steps:

When the logged-in account is switched from the specified account to the second account, a monitoring cancel instruction is generated, and the monitoring cancel instruction is used to instruct the slave device to stop receiving the master token information sent by the master device.

S107. Send a monitoring cancel instruction to the master device, and stop receiving the master token information sent by the master device.

In some embodiments, in the scenario where the slave token information used to log in to the specified account can be obtained from the local cache of the slave device, if the received second master token information is consistent with the account ID of the slave token information , it means that the master device that generates the second master token information can be the master device that originally logged in with the specified account.

If the account IDs are inconsistent, there are two situations. One is that the master device that originally logged into the designated account switches to log in to the second account, so that the master device that generates the second master token information is still the master device that originally logged into the designated account. At this point, there is no need to cancel token monitoring. The second is to log in the second account as another master device. At this time, the master device switches, so that the master device that generates the second master token information is switched to another display device, which is different from the master device that originally logged in with the specified account. At this point, the token monitoring needs to be canceled.

In some embodiments, in the aforementioned embodiments, the process of the slave device judging whether there is slave token information in the local cache occurs in the scenario where the slave device receives the second master token information, by judging whether there is slave token information in the local cache card information, and the result of comparing the second master token information with the account ID of the slave token information to determine the target token information required for account login, and to determine the target account to be logged in from the slave device, and then realize the slave device The account of the device is automatically logged in.

In other embodiments, the process of the slave device judging whether there is slave token information in the local cache can also occur in any scenario where the slave device is in the monitoring state, such as the re-login process of subsequent account login automatically. At this time, there is no need for the slave device to trigger the judgment process after receiving the second master token information sent by the master device. As long as the slave device needs to log in to the account again, it can read the slave token information in the local cache in real time to realize the account login. For example, after all devices log in to the specified account in the first round, if the specified account of the master device is still logged in, but the account of a slave device automatically logs out, at this time, the slave device is re-logging in with the specified account, and if no other In the case of the new token information sent by the master device, the slave token information can be obtained directly from the local cache of the slave device to log in to the specified account. If the secondary token information in the local cache is invalid, it is necessary to exchange new secondary token information from the server to realize the login of the specified account.

The specific implementation process may include: during the automatic account login process of the slave device, if the slave device needs to log in to the account again, at this time, the slave device obtains the login information from the local cache. The login information refers to the information required for logging in to the previous account. information from the token.

If there is no login information in the local cache, continue to listen for token arrival events. If a token event from the master device is received, the corresponding account login process is automatically performed based on the master token information sent by the master device. If there is login information in the local cache, the corresponding account can be directly logged in based on the login information to realize automatic account login.

If new token information (such as the second primary token information) is received, when the account ID in the new token information is inconsistent with the account ID of the login information in the cache, you can prompt to switch the login information through the interface.

For a display device provided in an embodiment of the present application, when a slave device for an automatic login account performs multi-device collaborative login, in the later account login process, an implementation scenario can be: if there is no login information in the local cache, monitor the The event of the arrival of the token will automatically log in after receiving it. If there is login information in the cache and it has not expired, it will be automatically logged in directly. If you receive new token information and the account ID is inconsistent with the cached one, you can switch the login information through the interface prompt. Another implementation scenario may be: when the slave device receives the second primary token information sent by the master device, if the login information does not exist in the local cache, log in the corresponding account based on the second primary token information. If there is login information in the local cache, and the account ID in the login information is consistent with the account ID in the second primary token information, then automatically log in based on the login information in the cache. And if the account comparison is inconsistent, you can switch the login information through the interface prompt. In the later account login process, the slave device can automatically log in the account based on the local cache, so as to log in the same account as the master device, without the need for the user to log in manually, and realize the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.

Fig. 9 is a flowchart of a multi-device cooperative login method executed by a server according to some embodiments. A server provided in an embodiment of the present application includes: a controller. When executing the multi-device cooperative login method shown in FIG. 7 and FIG. 9 , the controller is configured to perform the following steps:

S31. In response to the account login request carrying the user login information sent by the master device, verify the user login information, and generate master token information after the verification is passed, and send it to the master device, so that the master device The account ID in the token information is used to log in the corresponding specified account, and the master token information is sent to the slave device, and the account ID is used to identify the corresponding account after the verification of the user login information is passed.

In a home environment, the user wants to log in to multiple display devices with an account. Based on the multi-device collaborative login method provided by this application, when performing account login, the user can only perform manual account login on one display device, while the other display devices All devices can realize automatic account login.

The display device that the user performs manual account login is the master device, and the other display devices that automatically perform account login are slave devices. The master device generates an account login request based on the user login information input by the user, and sends it to the server to access a login (login) interface of the server. After receiving the account login request sent by the master device, the server can verify the user login information carried in the account login request, and the user login information includes a user name and a password. The account login request carries user login information, user ID and device ID, the user ID is used to identify the user, and the device ID is used to identify the device that implements account login.

When verifying, the server compares the user name and password in the user login information with the user name and password stored in the database configured by the server. If they are consistent, it means that the user name and password entered by the user are correct and the account can be logged in.

After the user login information verification is passed, in order to facilitate account login on the main device side, the server will generate corresponding main token information (token information). Token is a string of numbers, including user ID, device ID, account ID, and token creation. time and duration. The master token information is used to realize the specified account corresponding to the user login information of the master device and the slave device, the user ID is used to identify the user, the device ID is used to identify the device that realizes the account login, and the account ID is used to identify the account after verification (username and password), the token creation time is the valid period of the token, and the duration is the valid period of the token.

After verifying and passing the user login information based on the account login request sent by the master device, the server obtains the user ID and device ID carried in the account login request, obtains the account ID corresponding to the account information from the server database, and generates a token Creation time and duration, based on user ID, device ID, account ID, and token creation time and duration, the main token information is generated immediately and returned to the main device to realize the account login of the main device.

After the master device receives the master token information generated by the server after verifying the user login information and passing it, the master device can log in to the corresponding account based on the account ID in the master token information, that is, realize the designated account based on the user name and password entered by the user. login.

At the same time, in order to facilitate the other display devices in the home environment to automatically log in to the account, the master device sends the master token information to the slave device, and the slave device is used to request the slave token information from the server based on the master token information to The account ID in the information logs into the specified account corresponding to the user login information, and the slave device refers to the device that is in a pairing relationship with the master device. The master device and the slave device log in with the same designated account, so that after the user logs in on one device, he can automatically log in on other devices in the family, which is not only convenient for the user to log in, but also improves the user experience.

S32. In response to the automatic login request generated by the slave device based on the slave device ID after receiving the master token information sent by the master device, the slave device ID and the master token information carried in the automatic login request are verified. Then generate slave token information based on the master token information and send it to the slave device. The master token information includes user ID, account ID, token creation time and duration, and the slave token information includes user ID, account ID, slave The token ID, the updated token creation time and duration, so that the slave device can log in to the specified account based on the account ID in the slave token information, and the slave device is a device in a pairing relationship with the master device.

After each slave device receives the master token information sent by the master device, it can request the slave token information from the server based on the master token information, and automatically log in to the specified account corresponding to the specified ID in the slave token information, so that the slave device and the master The devices log in with the same account.

When the slave device automatically logs in to the same designated account based on the master token information sent by the master device, it needs to verify the information authority with the server, and the designated account can only be logged in after the verification is passed. At this point, the slave device generates an automatic login request based on the primary token information, and sends it to the server.

In order to prevent the display devices of other users from connecting to the Internet of Things of the local user's home environment through their own Bluetooth modules, causing the display devices of other users to automatically log in to the account used by the local user, resulting in information leakage, it is necessary to request automatic login. The slave device performs verification. For this reason, the slave device ID must be carried in the automatic login request generated by the slave device to identify the slave device. At this time, the slave device will generate an automatic login request with its own slave device ID and the user ID, account ID, token creation time and duration in the master token information, and send it to the server.

The server responds to the automatic login request sent by the slave device, and verifies the slave device ID and master token information carried in the automatic login request. The verification content includes checking whether the slave device allows automatic login, whether the slave device to perform automatic login is a preset device, and whether the master token information is valid. Preset devices refer to display devices in the same home environment, such as display devices in the same Internet of Things.

In some embodiments, the controller executes the automatic login request generated based on the slave device ID after receiving the master token information sent by the master device in response to the slave device, and the slave device ID and the master token information carried in the automatic login request For verification, is further configured to perform the following steps:

In response to the automatic login request generated by the slave device based on the slave device ID after receiving the master token information sent by the master device, the account ID, token creation time and duration in the master token information are acquired.

If there is a registered account ID used to identify the specified account of the master device login in the server, the slave device is allowed to perform an automatic login process, and it is checked whether the display device corresponding to the slave device ID is a preset device, and the default device refers to Display devices that are in the same Internet of Things for Bluetooth pairing connection;

If the display device corresponding to the slave device ID is a preset device, check whether the token creation time and duration are valid.

If the token creation time and duration are valid, it is determined that the slave device ID and master token information carried in the automatic login request sent by the slave device pass the verification.

After the server receives the automatic login request sent by any slave device, it obtains the slave device ID, user ID, account ID, token creation time and duration carried in the automatic login request to check whether the slave device allows automatic login Check if the login information is valid.

When checking whether the slave device allows automatic login, the account ID in the primary token information carried in the automatic login request sent by the slave device is compared with the registered account ID stored on the server side, and the registered account ID is used to identify the master device Logged in account. If an account is logged in by the main device, the server will store the corresponding account information, such as the registered account ID. Therefore, if the server stores the registered account ID, the account ID is consistent with the registered account ID, indicating that the specified account corresponding to the account ID is logged in by the master device, and the slave device is allowed to respond to the token information of the master device to realize the corresponding account ID. automatic login. If the account ID is not stored on the server, the account ID is inconsistent with the registered account ID, indicating that the specified account corresponding to the account ID has not been logged in by the master device, and the slave device is not allowed to perform the automatic login account process.

The conditions for the slave device to perform the automatic login process include: responding to the master token information sent by the master device (that is, the master device triggers automatic account login of the slave device), and the master device has logged into the corresponding account.

When the server checks that the server stores a registered account ID that is consistent with the account ID carried in the automatic login request, it means that the master device has logged in the corresponding specified account, and the automatic account login process of the slave device is triggered by the master device , therefore, the server can determine that the slave device is allowed to perform an automatic login procedure.

After the slave device is allowed to perform the automatic login process, the next verification process is performed, that is, to verify whether the display device corresponding to the slave device ID is a preset device. The preset device refers to a display device that is in the same Internet of Things for Bluetooth pairing connection .

In some embodiments, when the server verifies whether the slave device currently sending the automatic login request is a preset device, the slave device ID of the slave device can be connected with the Bluetooth pairing connection with the slave device stored on the server side in the same Internet of Things as the master device If the slave device ID is consistent with one of the display device IDs in the same Internet of Things, it can be determined that the slave device is a preset device, and the automatic account login process can be performed. If the comparisons are inconsistent, it means that the slave device is the display device of another user, and cannot log in with the same account as the display device of the local user to avoid information leakage, that is, the subsequent automatic account login process cannot be performed.

In some embodiments, when the server checks whether the slave device currently requesting automatic account login is a preset device, another method can also be used: when the slave device uploads the master token information, the slave device ID is also required, and the server After verifying that the slave device can realize the automatic account login function, the server parses out the account information based on the account ID from the master token information, and then obtains a list of devices bound to the account information or logged in. Determine whether the slave device ID is in the device list. If it is, the verification is passed, and then the master token verification is performed; if not, the slave device does not perform the automatic login process. Based on the comparison between the slave device ID and the device list that has logged in to the account, if the slave device is another user's device, since the other user's device cannot log in to the account, there will be no other user's device identifier in the device list. Therefore, if the comparison fails, it can prevent the account from being automatically logged in on the outsider's device after the Bluetooth connection of the outsider's device. In this scenario, in order to have the IDs of each display device in the home IoT in the device list, the user needs to actively log in to the account once in the new device, so that the device ID of the new device can be added to the device list corresponding to the account in the server.

After verifying that the display device corresponding to the slave device ID is the default device, the next verification process can be performed, that is, verifying whether the login information is valid. At this time, the token creation time and duration in the primary token information carried in the automatic login request are obtained, so as to determine whether the primary token information is valid based on the token creation time and duration.

If both the token creation time and the duration satisfy corresponding time conditions, it is determined that the token creation time and duration are both valid. For example, if the token creation time has not expired, and the generation time of the main token information does not exceed the time length threshold, it can be determined that both the token creation time and the time length are valid.

After the server verifies that the slave device is allowed to log in automatically based on the master token information, the slave device is a default device, and the login information (master token information) is valid, it can determine the slave device ID and Primary token information verification passed.

After the server verifies that the slave device ID and master token information carried in the automatic login request sent by the slave device pass, it will generate slave token information for realizing the login of the specified account by the slave device. Since the slave token information is updated based on the master token information sent by the master device, the role of the slave token information is for the slave device to log in with an account, and the slave device needs to log in with the same specified account as the master device. Therefore, the user ID and account ID in the master token information sent by the master device do not need to be updated, but the device ID used to identify the slave device and the token creation time and duration for the slave device to log in within the time limit need to be updated.

After the slave device receives the slave token information returned by the server, the slave token information includes the user ID, slave device ID, account ID, new token creation time, and new duration, and can then log in to the designated account ID corresponding to the slave token information. Account, the specified account is the same as the specified account logged in on the main device.

The server provided by the embodiment of the present application, when multiple display devices perform cooperative login account, based on the account login request sent by the main device, generates the main token information after the verification of the user login information is passed, so as to facilitate the main device to log in and the user to log in The specified account corresponding to the information, and automatically send the master token information to the slave device with the pairing relationship. Based on the automatic login request generated by the slave device based on the master token information, the slave token information is generated after the verification of the master token information is passed and the slave device is allowed to log in to the account, so that the slave device can log in to the same designated account as the master device based on the slave token information . After the user first logs in the account on one of the display devices, the other display devices can log in automatically, without the need for the user to log in manually, achieving the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.

In the multi-device cooperative login method, display device, and server provided by the embodiments of the present application, the master device that first logs in to the account sends an account login request to the server based on the user login information input by the user, and the receiving server verifies the user login information and passes the verification. After the master token information is generated, log in to the specified account, and at the same time, automatically send the master token information to the slave device with the pairing relationship. The slave device generates an automatic login request based on the master token information and sends it to the server. After the server verifies the master token information, it generates the slave token information and allows the slave device to log in to the account, so that the slave device is based on the slave token information and the master device. Log in to the same designated account. The method, the display device and the server can realize that after the user logs in the account preferentially on one of the display devices, the other display devices can automatically log in, without the need for the user to log in manually, and realize the purpose of multi-device cooperative login account. While providing convenience to users, account login efficiency is high, which improves user interaction experience.

According to some embodiments of the present application, the multiple terminal devices can also control the corresponding terminal devices to perform corresponding operations by voice after they log in to the account cooperatively.

With the wide application of home furnishing, controlling the corresponding terminal equipment through voice commands has won the favor of the majority of users. Fig. 10 is an application scenario diagram of a terminal device according to some embodiments. As shown in Figure 10, terminal equipment such as TV 103, air conditioner 104, refrigerator 105, and washing machine 106 in the home can be connected to mobile terminal 101 and server 102 through the Internet of Things module, and mobile terminal 101 and terminal equipment can be connected through LAN or WAN for data transmission to realize the control and management of terminal equipment. Typically, IoT modules can be built into individual end devices.

In related technologies, a voice recognition device is generally a device with certain functions. For the received voice, if no device is specified, it is considered that the device responds to the voice, and if a device to be operated is specified, a control command is sent to the specified device. In some embodiments, taking the speaker as an example, after receiving "fast forward", first judge whether there is an operation corresponding to the instruction on the speaker, if there is, perform the fast forward operation, if not, then feedback represents an unrecognized prompt information. If "TV fast forward" is received, the speaker first judges whether there is an operation corresponding to the corresponding TV corresponding to the mapping relationship, and if so, sends an operation command to trigger TV fast forward to the TV; if not, the feedback indicates that there is no Recognized prompt information. In the existing operation, the user must specify the target device to execute the operation intention, otherwise the speaker will input the operation intended to be performed by the speaker by default.

In some embodiments, if the user does not specify a target device, the speech recognition device may play a prompt to the user representing several devices for which mappings exist.

In some embodiments, the determination of the mapping relationship between the speech recognition device and other devices can be performed in a local module or in the cloud. In some embodiments, the local modules may be housed within the speech recognition device or fixed to each other. They can also be separate objects.

In some embodiments, taking cloud execution as an example, when a user adds a terminal device to a home, the terminal device can be divided into regions and the terminal device can be set in a fixed region. For example, TVs, speakers, air conditioners, etc. are installed in the living room area, and refrigerators are installed in the kitchen area. Fig. 11 is another application scenario diagram of a terminal device according to some embodiments. When the user wants to listen to XXX songs, after the user inputs the wake-up word “Hi! Voice message, the voice recognition device converts the voice message into a voice command through the voice application and transmits it to the server 102. After receiving the voice command, the server 102 inquires about the terminal equipment currently configured by the user. After finding out that there are TVs, refrigerators and air conditioners, It will give feedback to the user through the voice recognition device, "You have 3 devices, which one do you want to use?" The server 102 will further determine that there are two terminal devices, a TV and an air conditioner, in the living room at this time, and feed back to the user through the voice recognition device "You have 3 devices in the living room, which one do you want to use?" Let's play", so far, the server 102 will control the TV to play the XXX song. Wherein, the speech recognition device may be a device with a radio function such as a remote controller and a speaker. The voice recognition device may also be a display device with voice recognition and control functions, such as a TV or a mobile device.

Based on the fact that there are multiple terminal devices in one family, the server 102 needs to perform multiple rounds of voice interaction with the user, so that the user can actively make various prompts, and finally determine the terminal device to execute the command. The above illustrates the process of multiple interactions with the user.

In some embodiments, in order to improve user experience, the present application provides a server in some embodiments, and the server is configured to perform a voice interaction process. The voice interaction process will be described below with reference to the accompanying drawings.

In some embodiments, methods performed in server 102 may be performed in one other device or voice recognition device. Follow-up uses server execution as an example.

Fig. 12 is a schematic flowchart of a voice control method according to some embodiments. As shown in Figure 12, the method includes the following steps:

In the user's actual environment, there are speech recognition devices, as well as other devices such as TVs, refrigerators, and air conditioners. The speech recognition devices and the other devices can be connected to the server through the network, and the user can log in to the server through a pre-operation on a certain device. The same account further enables the server to store a mapping relationship between a user identifier (that is, a user ID) for identifying a user and a device. The speech recognition device can receive the speech input by the user at any moment.

In some embodiments, the user may send a voice command to the voice recognition device based on his own needs, and control the corresponding terminal device to execute the voice command by voice. For example, when the user wants to use the target device to watch a XXX movie, he can input the wake-up word "Hi! XX" to the voice recognition device, and then input a voice message of "Play XXX movie" to the voice recognition device. The target device and the voice recognition device are two independent devices. The target device can have its own voice receiving device for voice control. The target device and the voice recognition device are not the relationship between the TV and the TV remote control that is only used for the TV. Not the relationship between the TV processor and the audio receiver on the TV.

In some embodiments, the wake word and voice command can be entered continuously. The voice recognition device can recognize voice commands contained in sentences containing wake words.

In some embodiments, the voice recognition device can integrate sound collection and voice analysis functions. Among them, the radio function is to receive the voice message sent by the user, and the voice analysis function refers to extracting the key part of the voice message sent by the user. The key part can reflect the user's intention or the content to be done, and analyze the user's intention Afterwards, it is converted into a voice command, and the voice command may be an executable command format agreed between the voice recognition device and the server 102 . Wherein, the command format may include command (command) and parameters (parameter). After being converted into a voice command, the voice command is sent to the server 102 . Wherein, adding the corresponding user identifier to the voice command facilitates the server 102 to identify which user sent the voice command, and also facilitates the user to search for all terminal devices configured by the user.

In some embodiments, the speech recognition device may have an independent audio-visual display system, which may decode and play audio and video streams.

In some embodiments, the target device may have an independent audio-visual display system capable of decoding and playing audio and video streams.

S301: Receive a voice instruction including a user identifier sent by a voice recognition device.

In some embodiments, the speech recognition device may use the text converted from the speech as a speech instruction and send it to the server.

In some embodiments, the voice conversion server is in the server, so the voice recognition device can send the received voice to the server according to the agreed encapsulation structure, and the server decapsulates the received data and obtains voice instructions.

S302: Find all terminal devices related to the user identifier.

In some embodiments, after receiving the voice command, the server 102 analyzes the voice command to obtain the user identifier and the user's intention conveyed by the voice command.

In some embodiments, the server determines the device type corresponding to the command by analyzing the voice command and according to the keywords in the parsed text, where the device type refers to the functional authority of the terminal device.

In some embodiments, the mapping relationship between keywords and device types may be cached in the server, and in some embodiments, a trained keyword-device type neural network model may also be stored.

Take the device type corresponding to the cache key in the server as an example, as shown in Table 1:

语音指令voice command	设备类型Equipment type	终端设备Terminal Equipment
播放电影play movie	视频播放video playback	电视、冰箱屏TV, refrigerator screen
播放歌曲play song	音频播放Audio Player	音箱speakers
开始送风start blowing	室温调整room temperature adjustment	空调、电风扇air conditioner, electric fan

At the same time, after obtaining the user ID, the server 102 searches the database for all terminal devices related to the user ID, that is, searches for all terminal devices configured by the user. This is because, if device discovery is based on near-field communication, the device can discover all the devices it scans, including its own and/or devices of other users in close proximity, but some of these devices do not belong to the user , even if it is judged by the LAN, it is possible to initially select the scenario where a guest device is connected to the LAN. By pre-establishing the association between the user ID and its own equipment in a server with a management function, the server 102 can determine the equipment actually owned by the user according to the user ID.

In some embodiments, the terminal device refers to other devices associated with the user identifier except the speech recognition device.

In some embodiments, the recognition of the voice command needs to determine the type of the voice command, and then compare it with the types that can be executed by each terminal device, and then determine the terminal device that can execute the voice command.

In some embodiments, the execution types of each terminal device may be pre-calibrated, for example, the kitchen refrigerator corresponds to freezing, refrigeration, recipe recommendation, and ingredient identification. It may also be determined according to the device identifier when a newly added device is scanned. For example, when the newly added device identifier indicates that the device is a refrigerator, an association between the device identifier and freezing, refrigeration, recipe recommendation, food material identification, etc. is established.

In some embodiments, after acquiring the user ID, the server 102 may also load all terminal devices and device attributes related to the user ID from the database into the cache. Subsequently, when the server 102 searches for the terminal device again, it can directly search in the cache, thereby speeding up the speed at which the server 102 searches for the terminal device. The device attributes include inherent attributes such as the location of the terminal device, the name of the terminal device, and the ID of the terminal device.

S303: When there is no terminal device related to the user identifier, feed back a parameter representing the absence of a terminal device, so that the voice recognition device broadcasts that there is no terminal device executing the voice command.

In some embodiments, if the server 102 does not find a terminal device related to the user identifier, it means that the user has not configured a terminal device. At this time, the server 102 needs to feed back parameters indicating that there is no terminal device to the speech recognition device. The voice recognition device broadcasts that there is no terminal device according to the received parameter representing the absence of the terminal device.

S304: When there is a terminal device related to the user identifier, use the preset filtering rules to filter out the most matching terminal device, and feed back the parameters representing the most matching terminal device, so that the voice recognition device broadcasts the existence of the execution voice The terminal device with the instruction, and controls the most matching terminal device to execute the voice instruction.

In some embodiments, the preset filtering rule filter refers to a rule for filtering out a preset rule conforming to the user's intention/a device capable of executing the voice instruction. For example, when the user inputs the voice of "play folk music", the voice recognition device sends the instruction of playing folk music to the server, and the server recognizes that the type of user's intention is to play music, and then determines the terminal device that can perform this type of operation according to the type .

In some embodiments, when there is a terminal device related to the user identifier, the terminal device related to the user identifier is used to execute the voice instruction.

In some embodiments, when there is a terminal device related to the user identifier, it is also necessary to determine the number of terminal devices and/or the functions that the terminal device can perform. Here, the function that the terminal device can perform is the terminal Device function permissions. In some embodiments, when the voice instruction is an instruction to play a video, it is necessary to determine the device that can play the video, and the device that cannot perform video playback will be screened out. When the voice instruction is an instruction for cooling at 26°, cooling cannot be performed. The devices that are screened out, use the voice command that can be executed by the cooling device.

In some embodiments, the server 102 is configured with preset filtering rules. The filtering rule characterizes the mapping relationship between the voice instruction and the terminal device. The filter rules include the first set of rules and the second set of rules, or only the first set of rules do not include the second set of rules, wherein the first set of rules refers to the necessary rules, and the second group of rules refers to the rules that are superimposed and utilized one by one when the best matching terminal device is not screened out.

The first group of rules includes the mapping relationship between the device type and each device, such as Table 1.

The first group of rules may be a judgment on the number corresponding to user identifiers.

In some embodiments, after the devices are determined according to the first set of rules, if there are still two or more devices, the second rule needs to be further used for further screening. In some embodiments, after the optional device is determined according to the device type and the mapping relationship of each device, if the number of the device is 1, the voice command is directly sent to the device; if it is 0, the feedback is irrelevant. The device to execute is given to the voice recognition device. If there are more than one, it can be based on one or any combination of location/installation time/usage frequency/device that executed this type of command last/start time/signal strength/priority, etc. to filter as the second set of rules.

Fig. 13 is a schematic flow diagram of screening terminal devices according to some embodiments. With reference to FIG. 13 , in some embodiments of the present application, the process of screening terminal devices according to filtering rules is as follows:

S401: Use the first set of rules to screen terminal devices related to the user identifier of the current terminal device.

In some embodiments, when there is a terminal device related to the user identifier, the server 102 first screens the terminal device according to the necessary rules for screening out the most matching terminal device. In some embodiments, the first group of rules includes terminal device function authority sub-rules. Here, the terminal device function authority sub-rules refer to the functions that each terminal device has respectively, such as a TV having the function of playing media assets, an air conditioner, etc. It has the functions of cooling and heating, etc. That is, when the server 102 screens out the terminal devices actually owned by the user, it further uses the sub-rule of the terminal device function permission to screen the terminal devices, and excludes the terminal devices that do not have corresponding permission functions.

In some embodiments, when terminal devices are screened by using the sub-rules of terminal device function rights, the server 102 respectively detects function rights of multiple terminal devices related to the user identifier. When the function authority is suitable for executing the voice instruction, the server 102 selects the corresponding terminal device as the candidate terminal device. When the function authority is not suitable for executing the voice instruction, the server 102 excludes the corresponding terminal device.

S402: If the terminal device related to the user identifier does not have the authority to execute the voice command, feed back a parameter representing the absence of a terminal device, so that the voice recognition device broadcasts that there is no terminal device that executes the voice command.

S403: If the terminal device related to the user identifier has the right to execute the voice command, confirm the number of terminal devices that have the right to execute the voice command.

In some embodiments, when using the first set of rules to filter the terminal devices related to the user identifier, if none of the terminal devices actually owned by the user has the corresponding functional authority, the server 102 needs to feed back to the voice recognition device There are parameters for the end device. The voice recognition device broadcasts that there is no terminal device according to the received parameter representing the absence of the terminal device.

In some embodiments, when using the first set of rules to filter terminal devices related to the user identifier, if the terminal device actually owned by the user has the corresponding functional authority, the server 102 also needs to confirm that the terminal device with the corresponding functional authority quantity.

In some embodiments, when the server 102 detects that there is only one terminal device with the authority to execute the voice command, it feeds back to the voice recognition device the parameters representing the terminal device with the corresponding authority, and the voice recognition device exists according to the token. The parameters of the terminal equipment with the corresponding authority are broadcast to the terminal equipment with the corresponding authority.

The server 102 also needs to control the current terminal device to perform corresponding operations according to the user's intention in the voice command.

In some embodiments, the server 102 sends the voice instruction to the current terminal device, and the current terminal device receives the voice instruction and performs a corresponding operation according to the voice instruction.

For example, the user is currently only equipped with a sweeping robot. If the user inputs a voice message of "sweeping the floor", the voice device will transmit the voice message to the server 102. After querying, the server 102 finds that the current user is only equipped with a sweeping robot. With cleaning function. Therefore, the server 102 controls the voice recognition device to broadcast "the sweeping robot starts to sweep the floor", and controls the sweeping robot to perform the sweeping function. Wherein, when the server 102 controls the speech recognition device to broadcast, it may feed back the parameters representing the terminal device with corresponding authority to the speech recognition device through a long link.

In some embodiments, when there are multiple terminal devices that have the authority to execute the voice command, the sub-rules in the second set of rules are used to screen the multiple terminal devices that have the authority to execute the voice command one by one in order of priority from high to low. Terminal devices with the above-mentioned voice command permissions until a terminal device that best matches is screened out.

In some embodiments, the second set of rules preset in the server 102 includes a user frequency sub-rule, a distance from a speech recognition device sub-rule, and a terminal device priority sub-rule. Here, the user frequency sub-rule refers to the number of times the user has executed similar voice commands through the terminal device, for example, the number of times the user plays video data such as A movie and B variety show through the TV, or the user plays C song and D song through the speaker. The frequency of audio data such as A movie, B variety show, C song, D song, etc. can be divided into media data. "song" and "play D song" can be regarded as similar voice commands, both of which are play-type voice commands. "Play A movie" and "play B variety show" can be further divided into video-play voice commands, and "play C "Song" and "play D song" are divided into audio playback type voice commands. The distance sub-rule to the speech recognition device refers to the distance between each terminal device and the speech recognition device. The terminal device priority sub-rule refers to the priority of each terminal device set by the user when executing a voice command. For example, corresponding to the voice command for playing media assets, the priority of the TV is higher than that of the refrigerator screen. Among them, the terminal device The priority of the mobile terminal 101 can be set by the user through the application program.

In some embodiments, a plurality of sub-rules in the above-mentioned second group of rules can be set with corresponding priorities by the user, for example, the priority of setting the priority of the sub-rule of user usage frequency is higher than the priority of the sub-rule of the distance from the speech recognition device, The priority of the terminal device priority sub-rule is higher than that of the user frequency sub-rule.

In some embodiments, the server 102 uses filtering rules to filter terminal devices to filter out one of the most suitable terminal devices for executing the current voice command, that is to say, at the end of the screening, the number of the most matching terminal devices for 1 pc. After screening by the first set of rules, as long as the number of corresponding terminal devices is not unique, the server 102 will use the sub-rules in the second set of rules to screen the remaining terminal devices one by one according to the priority.

For example, after the user inputs "play song A", the server 102 finds out that the user is equipped with a TV, sound box, air conditioner, refrigerator, and washing machine. If the TV, sound box and refrigerator have the playback function, the TV, sound box and refrigerator need to be further screened through the second set of rules. The server 102 further filters through the distance sub-rule with the speech recognition device, and through the device location information in the device attributes in the terminal device, the server 102 judges that the TV, the sound box and the speech recognition device are all in the living room, that is, the distance is relatively close, and the refrigerator In the kitchen, the distance from the speech recognition equipment is relatively long, and based on this, the refrigerator is excluded. Since there are still two candidate terminal devices left, i.e. TV and sound box, further screening is required, and the server 102 continues to filter through the terminal device priority sub-rules. The priority of the speaker, so the server 102 selects the TV as the most matching terminal device.

For another example, after the user inputs "lower indoor temperature", the server 102 finds out that the user is equipped with a TV, a speaker, an air conditioner, a refrigerator, and an electric fan. Obtaining that the air conditioner and the electric fan have the function of adjusting the room temperature, the server 102 needs to further screen the air conditioner and the electric fan through the second set of rules. The server 102 continues to filter through the terminal device priority sub-rule. For voice commands such as room temperature adjustment, the priority of the air conditioner set by the user is higher than that of the electric fan, so the server 102 selects the air conditioner as the most matching terminal device. Of course, users can also set the priority of the electric fan higher than that of the air conditioner according to their own needs.

In some embodiments, when using the user frequency sub-rule to screen terminal devices, the server 102 respectively detects the execution frequencies of multiple terminal devices related to the user identifier, wherein the execution frequency refers to the terminal device The number of times similar voice commands have been executed in historical behaviors. The server 102 will retain the terminal device with the highest execution frequency.

In some embodiments, when terminal devices are screened using the sub-rule of distance from a voice recognition device, the server 102 respectively detects the distances between a plurality of terminal devices related to the user identifier and the voice recognition device. The server 102 will reserve the terminal device with the closest distance to the speech recognition device.

In some embodiments, when using the terminal device priority to screen terminal devices, the server 102 reserves the terminal device with the highest priority according to the terminal device priority set by the user.

The above embodiments are not only applicable to home scenarios, but also to office scenarios.

In this application, when the server responds to the voice command issued by the user and inquires that there are currently multiple terminal devices, it can select the terminal device that will finally execute the command based on the filtering rules, so as to avoid multiple voice interactions between the user and the voice recognition device process to improve user experience.

Based on the same inventive concept, the present application also provides a voice control method in some embodiments, the method includes: the server 102 receives the voice command containing the user ID sent by the voice recognition device, and searches for the voice command related to the user ID. all terminal equipment. When there is no terminal device related to the user identifier, the server 102 feeds back parameters representing the absence of the terminal device, so that the voice recognition device broadcasts that there is no terminal device executing the voice command. . When there is a terminal device related to the user identifier, the server 102 uses preset filtering rules to filter out the most matching terminal device, and feeds back parameters representing the most matching terminal device, so that the voice recognition device broadcasts the most matching terminal device. terminal device, and control the most matching terminal device to execute the voice command.

For convenience of explanation, the above description has been made in conjunction with specific implementation manners. However, the discussion of some of the above is not intended to be exhaustive or to limit the implementations to the specific forms disclosed above. Many modifications and variations are possible in light of the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various modified embodiments suitable for specific use considerations.

Claims

A terminal device comprising:

Controller, configured as:

receiving user login information input by the user to make the terminal device function as a master device, the user login information corresponding to a designated account;

Send an account login request to the server based on the user login information to request primary token information for login, the primary token information is generated by the server based on passing the verification of the user login information, the primary The token information includes an account ID for identifying the specified account;

receiving the primary token information from the server, and logging into a corresponding designated account based on the account ID in the primary token information;

Send the master token information to the slave device, so that the slave device sends an automatic login request to the server based on the master token information to request slave token information for logging in and log in based on the account ID in the slave token information The specified account of the slave device is in a pairing relationship with the master device.
The terminal device according to claim 1, wherein the controller is further configured to:

Start the bluetooth module configured on this end;

Establish a pairing relationship with the slave device based on the bluetooth module, wherein the slave device is in a token monitoring state, so that the slave device has the function of receiving token information and does not have the function of sending token information; the master device It is in the state of canceling token monitoring, so that the master device has the function of receiving and sending token information.
The terminal device according to claim 1, wherein the controller is further configured to:

receiving a monitoring cancellation instruction generated when the slave device logs in to a second account different from the specified account;

In response to the monitoring cancel instruction, stop sending master token information to the slave device.
The terminal device according to claim 1, wherein the controller is further configured to: generate the account login request based on user login information and a master device ID corresponding to the device ID used to identify the terminal device and send it to a server to request master token information from said server,

Wherein, when the server determines that the username and password in the login information are consistent with the stored registered username and password corresponding to the master device ID, the verification of the user login information passes, and the master The token information also includes the master device ID; and

Wherein, the slave device generates the automatic login request based on the master token information and the slave device ID corresponding to the slave device and sends it to the server to request the slave token information, and the slave token information is the The server generates it based on the slave device ID and the master token information based on the verification of the master token information, the slave token information includes the account ID and the slave device ID, and the slave device is based on the According to the slave device ID in the token information, use the account ID to log in the corresponding designated account.
A terminal device comprising:

Controller, configured as:

Receiving the master token information sent by the master device after logging in to the specified account based on the user login information input by the user to make the terminal device act as a slave device, the master token information is sent by the server based on the master device The verification of the user login information is generated and sent to the main device, and the main token information includes an account ID for identifying the specified account;

Send an automatic login request to the server based on the primary token information to request secondary token information for logging in, the secondary token information is generated by the server based on the verification of the primary token information , the slave token information includes the account ID;

receiving the slave token information from the server, and logging into a corresponding specified account based on the account ID in the slave token information.
The terminal device according to claim 5, wherein the controller is further configured to: generate the automatic login request based on the master token information and the slave device ID corresponding to the device ID of the terminal device to request The slave token information,

Wherein, the master device generates an account login request based on the user login information and the master device ID corresponding to the master device, and requests the server for the master token information, and the master token information is obtained by the server based on the The main device ID is generated by passing the verification of the user login information, and the main token information also includes the main device ID. When the server determines that the user name and password in the login information are consistent with the stored When the registered user name corresponding to the master device ID and the registered password are consistent, the verification of the user's login information is passed; and

Wherein, the secondary token information is generated by the server according to the primary token information and the secondary device ID based on the verification of the primary token information, and the secondary token information includes the account ID and the The slave device ID, the slave device uses the account ID to log in to the corresponding specified account according to the slave device ID in the slave token information.
The terminal device according to claim 6, wherein the controller is further configured to:

After logging in with the same specified account as the master device, the slave token information is cached.
The terminal device according to claim 7, wherein the controller is further configured to:

In response to the received second primary token information, determine whether there is second secondary token information required for logging in to the second account in the local cache, wherein the second primary token information includes information for identifying the A second account ID of a second account, the second slave token information includes the second account ID and the slave device ID.
The terminal device according to claim 8, wherein the controller is further configured to:

In response to the fact that the second secondary token information does not exist in the local cache, a corresponding target account is logged in based on a user's selection operation, and the target account includes one of the specified account and the second account.
The terminal device according to claim 8, wherein the controller is further configured to:

In response to the existence of the second slave token information in the local cache, determine whether the second slave token information is within the creation time;

In response to determining that the second slave token information is within the creation time, log in the second account with a second account ID based on the second slave token information; and

In response to determining that the second slave token information has expired, requesting updated second slave token information from the server based on the second slave token information, and utilizing the second account ID based on the updated second slave token information Log in to the second account.
The terminal device according to claim 9, wherein the terminal device further comprises a display configured to present a user interface,

The controller is further configured to:

In response to the absence of the second secondary token information in the local cache, a prompt interface is generated and displayed on the user interface, the prompt interface includes the specified account option corresponding to the secondary token information and the second primary token information The corresponding second account option;

In response to the user triggering the login instruction of the designated account option, logging in the corresponding designated account based on the slave token information;

In response to the user triggering the login instruction of the second account option, the corresponding second account is logged in based on the second primary token information.
The terminal device according to claim 10 or 11, wherein the controller is further configured to:

generating a monitoring cancellation instruction in response to the slave device logging into the second account;

In response to the monitoring cancellation instruction, stop receiving the master token information sent by the master device.
The terminal device according to claim 10 or 11, wherein the controller is further configured to:

generating a monitoring cancellation instruction in response to the slave device logging into the second account;

Sending the monitoring cancel instruction to the master device, so that the master device stops sending master token information to the slave device.
A server comprising:

Controller, configured as:

In response to an account login request carrying user login information sent by the master device, verifying the user login information, and,

After the verification is passed, the main token information is generated and sent to the main device, so that the main device logs in the corresponding designated account based on the account ID in the main token information and sends the main token information To the slave device, the account ID is used to identify the corresponding specified account after the user login information verification is passed;

In response to the automatic login request generated by the slave device receiving the primary token information sent by the master device, the primary token information carried in the automatic login request is verified, and the primary token information is generated after the verification is passed. Slave token information, and send it to the slave device, so that the slave device logs in the specified account based on the account ID in the slave token information, and the slave device is in a pairing relationship with the master device.
The server according to claim 14, wherein the controller is further configured to: when verifying the primary token information,

In response to the automatic login request generated by the slave device receiving the master token information sent by the master device, the slave device ID and the master token information are obtained, and the master token information includes token creation time and duration;

Responding to the presence of an account ID in the server for identifying a specified account for logging in by the master device, allowing the slave device to perform automatic login, and checking whether the terminal device corresponding to the slave device ID is a preset device, and the preset device is Refers to terminal devices that are in the same Internet of Things for Bluetooth pairing and connection;

In response to the terminal device corresponding to the slave device ID being a preset device, verifying whether the token creation time and duration are valid;

In response to the valid token creation time and duration, it is determined that the slave device ID and master token information carried in the automatic login request sent by the slave device pass the verification.
The server according to claim 14, wherein the server is further configured to:

Receive the voice command including the user ID sent by the voice recognition device;

Find all terminal devices associated with said user ID;

In response to the absence of a terminal device associated with the user ID, feeding back parameters representing the absence of a terminal device, so that the voice recognition device broadcasts that there is no terminal device executing a voice command;

In response to the existence of a terminal device related to the user ID, use a preset filter rule to filter out the most matching terminal device, and feed back parameters representing the most matching terminal device, so that the voice recognition device broadcasts that there is a user who executes the voice command. terminal device, and control the most matching terminal device to execute the voice command.
The server according to claim 16, wherein the filtering rule represents the mapping relationship between the voice command and the terminal device, and the filtering rule includes the first group of rules and the second group of rules, or only includes the first group of rules A set of rules does not include a second set of rules, wherein the first set of rules refers to the rules necessary to filter out the most matching terminal equipment, and the second set of rules refers to The rules are superimposed and utilized one by one when terminal devices are used.
The server according to claim 17, wherein the first set of rules includes terminal device function authority sub-rules, and the controller is further configured to: in response to the existence of a terminal device related to the user ID,

Using the first set of rules to screen the current terminal device and the terminal device related to the user ID;

In response to the fact that the terminal device associated with the user ID does not have the authority to execute the voice command, feeding back parameters representing the absence of the terminal device, so that the voice recognition device broadcasts that there is no terminal device that executes the voice command;

In response to the terminal device related to the user ID having the right to execute the voice command, confirm the number of terminal devices that have the right to execute the voice command.
The server according to claim 18, wherein the second set of rules includes a plurality of sub-rules, and a plurality of the sub-rules have priorities, and the controller is further configured to:

In response to the presence of a terminal device with the authority to execute the voice command, feed back parameters representing the terminal device with the corresponding authority, so that the voice recognition device broadcasts the terminal device with the corresponding authority, and controls the current terminal device to execute the voice command. instruction;

In response to the presence of multiple terminal devices with the authority to execute the voice command, use the sub-rules in the second set of rules to screen the multiple terminal devices with the authority to execute the voice command in descending order of priority , until a most matching terminal device is selected.
The server according to claim 17, wherein the second group of rules includes at least one of the user frequency sub-rule, the distance from the speech recognition device sub-rule, and the terminal device priority sub-rule.
The server according to claim 20, wherein, the server is configured to: in response to using the user usage frequency sub-rules to screen terminal devices,

Detecting the execution frequency of multiple terminal devices with the authority to execute the voice command, wherein the execution frequency refers to the number of times the terminal device has executed voice commands with the same function in historical behaviors;

The terminal device with the highest execution frequency is kept.
The server according to claim 20, wherein the server is configured to: in response to screening terminal devices using the distance from speech recognition device sub-rule,

Detecting the distance between a plurality of terminal devices having the authority to execute the voice command and the voice recognition device respectively;

Reserve the terminal device with the closest distance to the voice recognition device.
The server according to claim 20, wherein the server is configured to, in response to screening terminal devices using the terminal device priority,

According to the terminal device priority set by the user, the terminal device with the highest priority is reserved.
The server according to claim 16, wherein the server is further configured to: respond to the voice instruction sent by the receiving voice recognition device,

Loading all terminal devices and device attributes related to the user ID from the database into the cache, so that the server can search for the terminal devices in the cache.