WO2023016126A1 - Dispositif terminal, serveur et procédé de connexion de collaboration multi-dispositifs - Google Patents

Dispositif terminal, serveur et procédé de connexion de collaboration multi-dispositifs Download PDF

Info

Publication number
WO2023016126A1
WO2023016126A1 PCT/CN2022/102579 CN2022102579W WO2023016126A1 WO 2023016126 A1 WO2023016126 A1 WO 2023016126A1 CN 2022102579 W CN2022102579 W CN 2022102579W WO 2023016126 A1 WO2023016126 A1 WO 2023016126A1
Authority
WO
WIPO (PCT)
Prior art keywords
account
token information
slave
master
user
Prior art date
Application number
PCT/CN2022/102579
Other languages
English (en)
Chinese (zh)
Inventor
薛健
李金昆
Original Assignee
聚好看科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202110917713.4A external-priority patent/CN115910050A/zh
Priority claimed from CN202111093415.4A external-priority patent/CN115834095A/zh
Application filed by 聚好看科技股份有限公司 filed Critical 聚好看科技股份有限公司
Priority to CN202280050101.2A priority Critical patent/CN117652134A/zh
Publication of WO2023016126A1 publication Critical patent/WO2023016126A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40—Support for services or applications

Definitions

  • the present application relates to the technical field of device cooperative login, and in particular to a terminal device, a server and a multi-device cooperative login method.
  • terminal devices there may be multiple terminal devices in a user's home, such as remote controllers, televisions, refrigerators, and air conditioners.
  • various terminal devices in the home such as lighting equipment, home appliances, security equipment, etc.
  • the user when the user wants to control a terminal device, he can send a voice command to the smart speaker, and the smart speaker sends the voice command to the server, and the server controls the corresponding terminal device according to the voice command.
  • a terminal device including a controller configured to: receive user login information input by a user to make the terminal device act as a master device, the user login information corresponds to a specified account; based on User login information and send an account login request to the server to request the primary token information for login, the primary token information is generated by the server based on the verification of the user login information, the primary token
  • the card information includes an account ID for identifying the specified account; receiving the primary token information from the server, and logging into the corresponding specified account based on the account ID in the primary token information;
  • the card information is sent to the slave device, so that the slave device sends an automatic login request to the server based on the master token information to request the slave token information for login and the corresponding specified account based on the account ID in the slave token information, so
  • the slave device is in a pairing relationship with the master device.
  • a multi-device cooperative login method is provided, which is applied to a master device, wherein the method includes: receiving user login information input by a user, the user login information corresponding to a specified account; An account login request is sent to the server to request primary token information for login, the primary token information is generated by the server based on the verification of the user login information, and the primary token information includes An account ID for identifying the specified account; receiving the primary token information from the server, and logging into a corresponding specified account based on the account ID in the primary token information; sending the primary token information To the slave device, so that the slave device sends an automatic login request to the server based on the master token information to request the slave token information for login and log in the corresponding specified account based on the account ID in the slave token information, the slave device is in a pairing relationship with the master device.
  • a multi-device cooperative login method is provided, which is applied to a slave device, wherein the method includes: receiving the master token information sent by the master device after logging in to a specified account based on the user login information input by the user;
  • the terminal device is used as a slave device, the master token information is generated by the server based on the verification of the user login information sent by the master device and sent to the master device, the The primary token information includes an account ID for identifying the designated account; based on the primary token information, an automatic login request is sent to the server to request secondary token information for logging in, and the secondary token information is Generated by the server based on passing the verification of the primary token information, the secondary token information includes the account ID; receiving the secondary token information from the server, based on the secondary token information in the secondary token information Account ID, log in to the corresponding specified account.
  • a multi-device cooperative login method including: a first terminal device receives user login information input by a user to make the first terminal device act as a master device, and the user login information corresponds to a specified account; the main device sends an account login request to the server based on the user login information to request primary token information for login; the server responds to the account login request carrying the user login information sent by the main device, Verify the information, and generate primary token information after the verification is passed, the primary token information includes an account ID for identifying the specified account; the server sends the primary token information to the primary device; The master device receives the master token information from the server, and logs in the corresponding designated account based on the account ID in the master token information; the master device sends the master token information; the second terminal device acts as a slave device And receive the master token letter from the master device; the slave device sends an automatic login request to the server based on the master token information to request the slave token information for logging in; the server responds
  • Fig. 1 is a schematic diagram of an operation scene between an intelligent control device and a control device according to some embodiments
  • FIG. 2 is a block diagram of a hardware configuration of an intelligent control device 200 according to some embodiments.
  • FIG. 3 is a block diagram of a hardware configuration of a control device 100 according to some embodiments.
  • Fig. 5 is an interactive diagram of establishing a pairing relationship between multiple devices according to some embodiments.
  • Fig. 6 is a flowchart of a multi-device coordinated login method performed by a master device according to some embodiments
  • Fig. 7 is an interaction diagram of a multi-device cooperative login method according to some embodiments.
  • Fig. 8 is a flowchart of a multi-device cooperative login method performed by a slave device according to some embodiments.
  • Fig. 9 is a flowchart of a multi-device cooperative login method performed by a server according to some embodiments.
  • Fig. 10 is an application scenario diagram of a terminal device according to some embodiments.
  • Fig. 11 is another application scenario diagram of a terminal device according to some embodiments.
  • Fig. 12 is a schematic flowchart of a voice control method according to some embodiments.
  • Fig. 13 is a schematic flow diagram of screening terminal devices according to some embodiments.
  • Fig. 1 is a usage scenario of an intelligent control device according to some embodiments.
  • the intelligent control device 200 may be a display device, and also communicates with the server 400 , and the user may operate the intelligent control device 200 through the smart device 300 or the control device 100 .
  • the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a notebook computer, an AR/VR device, and the like.
  • the smart device 300 can also be used to control the smart control device 200 .
  • the smart control device 200 is controlled using an application program running on the smart device.
  • the smart device 300 and the smart control device may also be used for data communication.
  • the intelligent control device 200 can also be controlled in a manner other than the control device 100 and the intelligent device 300, for example, the voice instruction module configured inside the intelligent control device 200 can directly receive the user's voice
  • the command control can also be controlled by receiving the user's voice command through the voice control device installed outside the intelligent control device 200 .
  • the intelligent control device 200 also performs data communication with the server 400 .
  • the intelligent control device 200 can be allowed to communicate via local area network (LAN), wireless local area network (WLAN) and other networks.
  • the server 400 may provide various contents and interactions to the smart control device 200 .
  • Fig. 2 is a block diagram of hardware configuration of the control device 100 according to some embodiments.
  • the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply.
  • the control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the intelligent control device 200 can recognize and respond to, and acts as an intermediary interaction between the user and the intelligent control device 200 .
  • the communication interface 130 is used for communicating with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module.
  • the user input/output interface 140 includes at least one of a microphone, a touch pad, a sensor, a button, or an alternative module.
  • Fig. 3 is a block diagram of a hardware configuration of an intelligent control device 200 according to some embodiments.
  • the intelligent control device 200 includes a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 285, a memory, and a power supply , at least one of the user input interface 280 .
  • the controller includes a CPU, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input/output.
  • the display 260 includes a display screen component for presenting images, and a drive component for driving image display, for receiving image signals output from the controller, and displaying video content, image content, and menu manipulation interface. Components and user manipulation of the UI interface, etc.
  • the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.
  • the controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box wait.
  • the controller 250 controls the work of the intelligent control device and responds to user operations by storing various software control programs in the memory.
  • the controller 250 controls the overall operation of the smart control device 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.
  • the user input interface 280 is an interface that can be used to receive control input (such as: physical buttons on the body of the smart control device, or others).
  • Fig. 4 is a software configuration diagram in the intelligent control device 200 according to some embodiments.
  • the system is divided into four layers, from top to bottom are respectively the application (Applications) layer (abbreviated as “application layer”), application framework (Application Framework) layer (abbreviated as “framework layer”) "), Android runtime (Android runtime) and system library layer (referred to as “system runtime layer”), and the kernel layer.
  • application layer application layer
  • Application Framework Application Framework
  • Android runtime Application runtime
  • system library layer referred to as “system runtime layer”
  • kernel layer the kernel layer.
  • an operating system is configured in the display device.
  • an operating system In order to obtain a richer functional experience from the operating system, it is usually necessary to log in with a system account first.
  • each device needs to log in with a user account.
  • the commonly used login method is that each device needs to log in with an account separately, for example, the user actively scans the QR code on each device to log in.
  • This SSO (single sign-on) method needs to be based on the user's initiative to trigger different devices to log in to the account. This login method is not convenient enough, resulting in low login efficiency and poor user experience.
  • a terminal device and a server in which multiple devices can log in to an account cooperatively, that is, after a terminal device (such as a display device, or a non-display device) logs in to an account, other devices also automatically log in.
  • a terminal device such as a display device, or a non-display device
  • other devices also automatically log in.
  • the same account can not only provide convenience to users, but also improve user experience.
  • other devices may also be display devices, such as various devices with display screens, such as televisions, mobile phones, refrigerators, etc., or may also be non-display devices. Users can only log in to an account on one device, and according to the habits defined by the user, they can automatically log in to multiple devices on the device they want to log in to, achieving smart login and improving user interaction experience.
  • the embodiment of the present application provides a display device, such as the display device with the configurations shown in FIG. 1 to FIG. 4 , where the display device is a master device, that is, a device that performs account login first.
  • the display device may serve as a slave device.
  • the display device includes: a display configured to present a user interface; and a controller connected to the display.
  • a pairing relationship needs to be established between each display device in the home environment.
  • the controller is further configured to perform the following steps:
  • Step 1 Start the Bluetooth module configured on the local end.
  • the bluetooth module is started along with the startup process, and the user login is performed after the system startup is completed.
  • the bluetooth module configured at the local end may also be activated after the user logs in.
  • some terminal display devices will disable the Bluetooth function by default, so the terminal display device can choose to enable the Bluetooth function through the setting menu at any time before or after the user logs in, and then activate the Bluetooth module configured locally.
  • Step 2 Establish a pairing relationship with the slave device based on the Bluetooth module to transmit data with the slave device through the Bluetooth module.
  • the Bluetooth module configured from the device is in the startup state, and the slave device is registered with a monitoring token (token) event, and monitors the token The event is used to monitor the token event sent by the slave device to the master device.
  • token token
  • Each display device for collaborative login in the home environment is equipped with a Bluetooth module.
  • the Bluetooth module is configured to be turned on by default or must be turned on forcibly. Therefore, when each display device is initialized, start Each display device has its own bluetooth module, and each display device realizes the pairing relation connection between itself and other devices through the bluetooth module. After the pairing connection is made, data transmission can be performed between each device. Data transmission includes communication data transmission and token data transmission. Communication data transmission is used to ensure communication between devices. Token data transmission can receive master token information sent by the master device from the device.
  • the display device can also manually turn on or turn off the Bluetooth function according to requirements. After the Bluetooth function is turned off, it cannot be paired with other devices through the Bluetooth module. After the Bluetooth function is turned on, it can be paired with other devices through the Bluetooth module. .
  • Multi-device collaborative login is applicable not only to various display devices in the home environment, but also to various display devices in classrooms, shopping malls and other places. Each display device meets the distance connection requirements, or is located in the same local area network.
  • Fig. 5 is an interaction diagram for establishing a pairing relationship between multiple devices according to some embodiments. See the example shown in part (a) of Figure 5.
  • the main device is a TV
  • the slave devices are a Rubik's Cube screen and a refrigerator screen
  • the Bluetooth modules of the TV, the Rubik's Cube screen, and the refrigerator screen are activated in sequence.
  • the Rubik's Cube screen is a portable terminal device with a display screen and a Bluetooth module
  • the refrigerator screen refers to a refrigerator equipped with a display screen and a Bluetooth module.
  • each device After completing the pairing relationship connection among the multiple devices, each device registers the monitoring token event for each connected display device in response to the establishment of the pairing connection.
  • Monitoring the token event is used to monitor the token event sent by the master device, and the token event can be the master token information sent by the master device.
  • the display device that performs account login based on user input is the master device, and the remaining display devices that implement the automatic login function through the login of the master device are slave devices.
  • only the monitoring of the token event of the master device by the slave device is reserved. For example, if the user selects the TV as the master device to log in the first account, the monitor token event of the TV is canceled, and the Rubik’s Cube screen
  • the monitoring token event of the refrigerator screen is reserved, that is, the master device is not in the monitoring state, and other slave devices are in the monitoring state.
  • the master device that is not in the listening state has the function of receiving token data and sending token data
  • the slave device in the listening state only has the function of receiving token data, but not the function of sending token data, and the communication data can be normal transmission.
  • the slave device in the listening state is only configured to receive token data and send/receive communication data functions, but not to send tokens
  • the data function can prevent one of the slave devices from sending the master token information to other slave devices after receiving the master token information sent by the master device to achieve account login, thereby avoiding data transmission confusion and affecting other slave devices.
  • Account login efficiency That is to say, each slave device can respond to the master token information and communication data sent by the master device that logged in the account first, and respond to the communication data sent by other slave devices, but cannot send token information to other devices.
  • the master device has the function of transmitting communication data and token data at the same time.
  • the monitoring token events of the other two devices are reserved, and they are determined to be slave devices, and both continue to be in the monitoring state.
  • the two slave devices only have the function of communication data transmission. Therefore, the server can determine which device is the master device of the priority login account based on monitoring the state change of the token event, and inform other devices that there is already a master device of the login account.
  • the listening status of each display device can be switched in real time according to the identity of the display device itself. For example, when the display device is the master device, the display device is not in the listening state; when the display device is the slave device, the display device is in the listening state.
  • each display device after each display device is paired and connected through its own Bluetooth module, it can automatically log in to other display devices according to the account login information on one display device, so as to realize the purpose of multi-device cooperative login account. For example, when each display device does not turn on the Bluetooth module, that is, when no pairing connection is established, if a certain display device has a priority login account, and then turns on the Bluetooth module to establish a pairing connection with other devices, the main device of the priority login account can still be based on The account login information sends token information to other slave devices, so that each slave device can automatically log in. That is to say, in the scenario where a certain display device logs in the account first and then turns on Bluetooth pairing, other slave devices that have not logged in the account can still achieve automatic login.
  • Fig. 6 is a flowchart of a multi-device coordinated login method performed by a master device according to some embodiments
  • Fig. 7 is an interaction diagram of a multi-device coordinated login method according to some embodiments. Specifically, after the pairing connection is completed, each display device realizes the multi-device cooperative login account, that is, when performing the multi-device cooperative login method shown in FIG. 6 and FIG. 7, the controller in the main device is configured to perform the following steps: :
  • a home environment there are usually multiple display devices, and the user can select one of the display devices as the primary device for first logging in to the account.
  • a feasible way for the main device to implement account login is to display a login QR code on the display of the main device, and after the user scans the login QR code on the display through the smart terminal, a login interface is displayed on the smart terminal.
  • the login interface presents a user name input box and a password input box.
  • the user enters a user name in the user name input box and a password in the password input box to obtain user login information for account login of the master device.
  • the smart terminal may be a device with a scanning function such as a mobile phone and a Pad.
  • a successful login interface may pop up on the display, prompting, for example, that account A has successfully logged in.
  • Another feasible way for the main device to implement account login is to directly present the login interface on the display of the main device.
  • the user login information is obtained to log in the account of the main device.
  • the two ways for the user to input the user login information may be determined according to actual conditions, and other input ways may also be used, which are not limited here.
  • the user login information includes a user name and password, etc., and the user login information corresponds to a designated account.
  • the main device requests the server to verify the user login information, and the account login can only be realized after the verification is passed.
  • the main device generates an account login request based on the user login information input by the user, and sends it to the server to access a login (login) interface of the server.
  • the server After receiving the account login request sent by the master device, the server can verify the user login information carried in the account login request.
  • the account login request carries user login information, a user ID and a device ID (that is, the main device ID), the user ID is used to identify the user, and the device ID is used to identify the device that implements account login.
  • the database in the server stores account information generated when the user registers an account, and the account information includes a registered user name and a registered password. Therefore, when verifying, the server compares the user name and password in the user login information with the registered user name and password stored in the database configured by the server. If they are consistent, it means that the user name and password entered by the user are correct and the account can be logged in. .
  • Token information is a string of numbers, including user ID, device ID, account ID, and token creation. time and duration.
  • the master device and the slave device can realize the specified account corresponding to the login user login information, where the user ID is used to identify the user, the device ID is used to identify the device that implements the account login, and the account ID is used to identify the account (user name and password), the token creation time is the starting point when the token becomes effective, and the duration is the effective duration of the token.
  • the token creation time is calculated from the time when the server verifies the user login information.
  • the server After verifying and passing the user login information based on the account login request sent by the master device, the server obtains the user ID and device ID carried in the account login request, obtains the account ID corresponding to the account information from the server database, and Generate the token creation time and duration, based on the user ID, device ID, account ID, token creation time and duration, then generate the main token information, and return it to the main device to realize the account login of the main device.
  • the main device if there is a new device with a newly turned on Bluetooth module for pairing connection after the main device logs in with the account, then after pairing and connection with the new device, in response to the Bluetooth connection of the new device, the main device according to the established Login account information, and request the server to return the primary token information. At this time, the primary token information only updates the token creation time and duration, and the rest remains unchanged.
  • the server After the server receives the request from the master device, it can request to return the master token information to the master device through the HTTP interface. Due to the uniqueness of the HTTP request, only the requester can receive the returned information.
  • the master device receives the master token information, and when it is determined that the device ID included in the master token information is consistent with the device ID of the master device, the master device can perform account login based on the account ID in the master token information.
  • the master device can log in with the specified account or establish a new Bluetooth connection with other slave devices.
  • the main token information used to realize the login of the specified account is automatically sent to each slave device through the Bluetooth module.
  • the slave device refers to other display devices in the home environment except for the first login to the specified account.
  • display device A If there are five display devices in the home environment, and the user selects display device A as the main device to log in with an account, then after display device A completes the account login, display device A sends display devices to display device B, display device C, and display device respectively through the Bluetooth module.
  • Device D and display device E send master token information, wherein display device B, display device C, display device D, and display device E are all slave devices.
  • each slave device After each slave device receives the master token information sent by the master device through the bluetooth module, it can request the slave token information from the server based on the master token information. Specifically, the slave device generates an automatic login request based on the master token information and the slave device ID corresponding to the slave device and sends it to the server to request the slave token information; after the server receives the master token information sent by the slave device Verify the primary token information, and generate secondary token information according to the primary token information and the secondary device ID and send it to the secondary device after the verification is passed. The slave device logs in the specified account corresponding to the user login information based on the account ID in the slave token information.
  • the master device and the slave device log in with the same designated account, so that after the user logs in on one device, he can automatically log in on other devices in the family, which is not only convenient for the user to log in, but also improves the user experience.
  • the process for the server to verify the primary token information includes: after the server receives the primary token information sent from the device, decrypts the primary token information according to the salt value; Obtain the main device ID, account ID, token creation time, and valid duration; determine whether the main token information has expired, if it is determined that the main token information has expired, then determine that the verification of the main token information has failed; if it is determined that the main token information has not expired, Then it is determined that the verification of the primary token information is passed.
  • the contents of the master token information and the slave token information are different except that the user ID and account ID are the same, that is, the device ID, token creation time and duration will be updated.
  • the slave device automatically logging in to the same designated account based on the master token information sent by the master device, reference may be made to the contents of steps S21 to S23 provided in subsequent embodiments.
  • the account after each display device completes an account login, the account will usually log out automatically after a period of time. For example, if the validity period of the token expires, then the user needs to log in to the account again on the corresponding display device during subsequent use.
  • the account for re-login of the display device may be the specified account for the previous login, or the second account that needs to be logged in currently. For example, if a slave device has logged in a specified account, and in the subsequent login, the slave device switches to log in to the second account, it is necessary to notify the master device that logs in to the specified account, and inform the master device that it does not need to send the specified account to the slave device. Corresponding primary token information.
  • the controller is further configured to perform the following steps:
  • the slave device When the login account of the slave device is switched from the specified account to the second account, the slave device will cancel the token event monitoring of the master device that logs in to the specified account, that is, the slave device generates a monitoring cancel command and sends it to the master device that logs in to the specified account .
  • the master device After the master device receives the monitoring cancellation instruction sent by the slave device, it will no longer send the master token information for logging in to the specified account to the slave device, so that the slave device can receive the master token information sent by the new master device logging in to the second account .
  • the new master device (the original first-round slave device) logged in with the switching account may generate a monitoring cancellation instruction.
  • the second round of new master devices (the original first round of slave devices) that switch accounts to log in can send a monitoring cancel command to the first round of master devices, so that Inform the master device of the first round that there is no need to send the master token information of the first round to the new master device.
  • the new master device may refuse to receive the master token information sent by the master device in the first round.
  • a prompt interface can be displayed on the display screen of the slave device for The user chooses to continue to log in to the original designated account or switch to log in to the second account. If a slave device generates an instruction to log in to the second account, the slave device generates a monitoring cancellation command and sends it to the first round of the master device to cancel the monitoring of the token information of the first round of the master device. At this time, the The slave device will monitor the token information of the new master device in the second round to realize the automatic login of the second account.
  • a slave device If a slave device generates an instruction to continue to log in to the original designated account status, the slave device generates a monitoring cancellation command and sends it to the new master device in the second round, canceling the monitoring of the token information of the new master device in the second round , at this time, the slave device will continue to monitor the token information of the master device in the first round, and continue to maintain the login status of the specified account.
  • the display device logged into the second account may be other display devices in the home environment.
  • display device A is the master device
  • display device B and display device C are slave devices
  • the three display devices are all registered with the specified account.
  • each device may have to re-login to the account
  • the user selects display device B to log in to the second account, and the identity of display device B changes from the original slave device to the master device, while the display device A and display device C are slave devices.
  • display device B belonging to the identity of the master device sends token information for logging in the second account to the slave devices (display device A and display device C), so that display device A and display device C log in to the second account.
  • the monitoring token event of the display device B is cancelled. , in the unlistened state, so that the display device B has the function of receiving/sending token data. Then restore the monitoring token event of the display device A, and it is in the monitoring state, so that the display device A only has the function of receiving token data and receiving/sending communication data, but not the function of sending token data. At this time, the display device B is not in the listening state, and both the display device A and the display device C are in the listening state.
  • the display device C as the slave device receives the master token information sent by the display device B as the master device in the second round, if the account logged in by operating the display device C is also switched from the original designated account to the second account, the login After the account is switched, the display device C correspondingly generates a monitoring cancel command, and sends it to the display device A that originally logged in with the specified account, and informs the display device A that was originally the master device of the first round, to stop sending the specified account to the original slave device (display device C) Corresponding primary token information.
  • the display device logged in with the second account may also be the original main device, that is, the main device originally logged in with the specified account.
  • the identities of the master device and the slave device are the original configuration.
  • Each slave device does not need to cancel the monitoring token event, that is, there is no need to generate a monitoring cancel command. .
  • the master device is always in a non-monitoring state, and each slave device is always in a listening state.
  • a display device provided in an embodiment of the present application generates an account login request based on the user login information input by the user when the master device that first logs in to the account performs multi-device collaborative login, sends it to the server, and receives the server's request for the user to log in to the account.
  • the master token information generated after the login information verification is passed logs in the specified account corresponding to the account ID in the master token information, and at the same time, automatically sends the master token information to the slave device with a pairing relationship, so that the slave device is based on the master token
  • the information requests the slave device information from the server, log in the same specified account based on the account ID in the slave device information.
  • the other display devices can log in automatically, without the need for the user to log in manually, achieving the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.
  • a multi-device cooperative login method performed by a slave device is provided.
  • a display device provided in an embodiment of the present application is a slave device, that is, a device that performs automatic account login based on the master token information sent by the master device.
  • the display device includes: a display configured to present a user interface; a controller connected to the display, and when the slave device executes a multi-device cooperative login method, the controller is configured to perform the following steps:
  • the main token information refers to the information generated by the server after the main device requests the server to pass the verification based on the user login information.
  • the primary token information includes user ID, primary device ID, account ID, token creation time and duration.
  • each display device is paired and connected through its own bluetooth module.
  • account login if the user selects one of the display devices as the first device to log in the account, then the device is the master device.
  • the main device logs in to the specified account (such as account A) based on the user login information entered by the user, it will request verification from the server and receive the primary token information generated by the server after the verification is passed, and then the primary device logs in to the specified account based on the primary token information , while automatically sending the master token information to each slave device.
  • the specified account such as account A
  • each slave device After each slave device receives the master token information sent by the master device, it can automatically log in the corresponding designated account based on the master token information, so that the slave device and the master device log in to the same account.
  • an automatic login request is generated and sent to the server.
  • the automatic login request is used to instruct the server to verify the slave device ID Check with the primary token information, and generate secondary token information based on the primary token information after the verification is passed.
  • the secondary token information includes user ID, account ID, secondary device ID, updated token creation time and duration.
  • the initial Bluetooth password of many Bluetooth devices is generally 0000 or 1234, and even some Bluetooth pairing does not require a password, just long press it, this will appear
  • the display devices of other users are also connected to the Internet of Things of the local user's home environment through their own Bluetooth modules, causing the display devices of other users to automatically log in to the account used by the local user, resulting in information leakage.
  • the slave device when the slave device receives the master token information sent by the master device and performs account login, it needs to verify the information with the server, and the specified account can be logged in only after the verification is passed. At this time, the slave device will generate an automatic login request with its own slave device ID and the user ID, master device ID, account ID, token creation time and duration in the master token information, and send it to the server.
  • the server responds to the automatic login request sent by the slave device, and verifies the slave device ID and master token information carried in the automatic login request.
  • the verification content includes checking whether the slave device allows automatic login, whether the slave device to perform automatic login is a preset device, and whether the master token information is valid.
  • Preset devices refer to display devices in the same home environment, such as display devices in the same Internet of Things. This is because TV devices in the same Internet of Things generally belong to the same user.
  • the preset device can also be a device bound to an account, for example, according to the account information in the slave device ID and the master token information, query the slave device ID in the pre-stored mapping relationship between the account and the device Whether there is a binding relationship with the account information, if it exists, the validity of the primary token is judged, and if it does not exist, the automatic login of the device is rejected.
  • the user device can bind the account and the device ID through the initial login or other terminal settings, and then in the local area communication network connected by Bluetooth, when there is a non-user device, it can also prevent the non-user device from automatically logging in to the user. account.
  • the main token information includes user ID, main device ID, account ID, token creation time and duration, and the main device ID is used to identify the source of the main token information. Therefore, it is possible to check whether the slave device allows automatic login based on the user ID and account ID, judge whether the slave device currently requiring automatic account login is a preset device based on the slave device ID, and judge the master token based on the token creation time and duration information is valid.
  • the above three judging processes need to pass the verification of the previous judging process before the next judging process can be executed, and the server will generate slave token information only after the verification of the three judging processes is passed.
  • the server checks whether the slave device currently requesting automatic account login is a preset device
  • another method can also be used: when the slave device uploads the master token information, the slave device ID is also required, and the server After verifying that the slave device can realize the automatic account login function, the server parses out the account information based on the account ID from the master token information, and then obtains a list of user devices bound to the account information or logged in. Judging whether the slave device ID is in the device list, if it is, then the verification is passed, and then the verification of the validity of the master token information is performed; if not, the slave device is rejected to perform the automatic login process.
  • the slave device Based on the comparison between the slave device ID and the device list that has logged in to the account, if the slave device is another user's device, since the other user's device cannot log in to the account, there will be no other user's device identification in the device list , Therefore, if the comparison fails, it can avoid the automatic login account caused by the bluetooth connection of the outsider's device.
  • the user in order to make the identification of each display device in the home Internet of Things exist in the device list, in some embodiments, the user can actively log in to the account once in the new device, so that the device ID of the new device can be added to the device corresponding to the account in the server. list, or the user can select the device ID of a new device to add to the user device list by operating the main device.
  • the server After the server checks that the slave device can perform an automatic login process based on the master token information, that the slave device is a preset device, and that the master token information is valid, it can be determined that the verification of the master token information passes.
  • the server performs an automatic login process based on the master token information, that the slave device is a preset device, and that the master token information is valid, it can be determined that the verification of the master token information passes.
  • the server After verifying that the primary token information carried in the automatic login request sent by the slave device is passed, the server will generate secondary token information for realizing the login of the specified account by the slave device. Since the slave token information is updated based on the master token information sent by the master device, the role of the slave token information is for the slave device to log in with an account, and the slave device needs to log in with the same specified account as the master device. Therefore, the user ID and account ID in the master token information sent by the master device do not need to be updated, but the device ID used to identify the slave device and the token creation time and duration for the slave device to log in within the time limit need to be updated.
  • the update content includes replacing the device ID of the original master device in the master token information with the device ID of the slave device, and updating the token creation time and duration, so that the updated slave token information includes user ID, slave device ID, Account ID, new token creation time and new duration.
  • the starting point of the new token creation time is calculated from the time when the server verifies the primary token information.
  • the server After the server verifies that the master token information carried in the automatic login request sent by the slave device has passed, it will return the synchronously generated slave token information to the slave device, so that the slave device can log in to the specified account based on the slave token information.
  • the TV sends the main token information required for logging in to the specified account (such as account A) to the Rubik's Cube screen and the refrigerator screen respectively through the Bluetooth module.
  • the Rubik's Cube screen (or refrigerator screen) requests the server through the primary token information and the secondary device ID in exchange for the secondary token information required by its own device to log in to the account.
  • the Rubik's Cube screen (or refrigerator screen) can log in the specified account.
  • the TV, Rubik’s Cube and Refrigerator are all logged into the same account A, the difference is that the TV is manually logged in by the user, while the Rubik’s Cube and Refrigerator are automatically logged in without the user’s manual login.
  • the slave device when the slave device as an automatic login account performs multi-device cooperative login account, the slave device receives the master token information sent by the master device, generates an automatic login request and sends it to the server, and the server receives Check the master token information and generate slave token information based on the slave device information after passing the verification.
  • the slave device logs in the same designated account as the master device based on the slave token information.
  • the other display devices can log in automatically, without the need for the user to log in manually, realizing the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.
  • both the master device and the slave device will cache the corresponding token information. That is, after the master device implements the specified account login based on the user login information entered by the user, it will store the corresponding master token information in the local cache of the master device; the slave device will automatically log in the designated account based on the master token information sent by the master device After that, the corresponding slave token information will be stored in the local cache of the slave device.
  • the cached token information can facilitate the login of the corresponding account by directly using the token information in the cache in the subsequent login process. Therefore, after completing an account login, the controller is further configured to: cache the slave token information returned by the server for implementing the login of the specified account after logging in with the master device with the same specified account.
  • the slave token information returned by the server for realizing the login of the designated account is stored in the local cache of the slave device.
  • the TV when the master device (TV) logs into account A, the TV sends the master token information corresponding to account A to the slave devices (the Rubik's Cube screen and the refrigerator screen). After receiving the primary token information corresponding to account A, the Rubik’s cube screen and refrigerator screen exchange their own secondary token information from the server, and then realize automatic login of account A based on the secondary token information.
  • each display device After each display device completes an account login, usually the account will automatically log out after a period of time. Then, when the user uses it subsequently, the corresponding display device needs to log in to the account again.
  • the account for re-login of the display device may be the specified account for the previous login, or a second account that needs to be logged in currently, and the second account is different from the specified account.
  • the slave device In order to judge whether the master token information received from the master device when the slave device automatically logs in to the account is the master token information corresponding to the original login to the specified account, the slave device needs to receive the new master token information when it receives the new master token information. Compare with the slave token information used to log in the specified account in the local cache, and determine whether the current account that needs to be logged in is the specified account for the original login through the comparison result, and determine the target account that needs to be logged in currently.
  • the controller when re-logging into the account subsequently, the controller is further configured to perform the following steps:
  • slave token information does not exist in the local cache, log in the corresponding second account based on the second master token information.
  • the corresponding target account is logged in based on the user's selection operation, and the target account includes one of the specified account or the second account.
  • the slave device When the slave device receives the second master token information sent by the master device, it determines whether the slave token information corresponding to the previous login account is stored in the local cache.
  • the second primary token information refers to the Nth primary token information sent by the master device, N ⁇ 2, which can be the same as the primary token information sent by the aforementioned master device, and is used for login correspondence
  • the specified account it can also be different, it is used to log in to other accounts.
  • the slave token information is not obtained in the local cache of the slave device, it means that the current account login process of the slave device is the first account login process, then the received second master token information is The master token information sent by the master device described earlier.
  • the process of judging whether there is slave token information in the local cache can be applied in the process of step S21, that is, when the slave device receives the master token information sent by the master device, it judges whether there is a specified account for logging in in the local cache. required slave token information.
  • the process of steps S22 to S23 is continued to complete the first account login process. That is to say, the process of logging in the second account by the slave device based on the second primary token information can refer to the implementation content of steps S21 to S23.
  • the slave token information used to log in to the specified account can be obtained in the local cache of the slave device, it means that the current account login process of the slave device is the Nth login process, that is, the slave device has performed N -1 account login process, which belongs to the post-login account process. Then the received second primary token information is the Nth primary token information sent by the master device.
  • the account ID in the slave token information is consistent with the account ID in the second master token information, it means that the account to be logged in corresponding to the second master token information currently received from the slave device is the same as the account that has been logged in before. the same one.
  • the slave device does not need to exchange for a new token from the server based on the second master token information again. From the token information, the designated account can be directly logged in based on the slave token information in the local cache, thereby improving account login efficiency. If it is confirmed that the slave token information is invalid according to the token creation time and duration, the slave device still needs to request the server to update the slave token information based on the slave token information before logging in with the specified account.
  • the account ID in the secondary token information is inconsistent with the account ID in the second primary token information, it means that the account that needs to be logged in has not been logged in before, and account switching and login are required.
  • the second account that the slave device currently needs to log in is different from the account in the local cache, the corresponding target account can be logged in based on the user's selection operation, and the target account includes one of the specified account or the second account.
  • the controller is further configured to:
  • a prompt interface is generated and displayed in the user interface.
  • the prompt interface includes the specified account option corresponding to the secondary token information and the second account option corresponding to the second primary token information.
  • the corresponding specified account is logged in based on the secondary token information.
  • the corresponding second account is logged in based on the second primary token information.
  • the slave device When the slave device needs to perform account switching and login, the user can perform a selection operation.
  • the slave device can automatically log in to the corresponding second account based on the second primary token information, or it can also be set so that the slave device can automatically log in to the corresponding second account based on the slave token information.
  • the specified account option and the second account option are displayed on the prompt interface.
  • the specified account option is used to implement the login of the specified account corresponding to the secondary token information
  • the second account option is used to implement the login of the second account (such as account B) corresponding to the second primary token information.
  • the slave device logs in the corresponding specified account according to the slave token information in the local cache. Since the slave token information in the local cache of the slave device is the slave token information required by the slave device to request the server for its own login account based on the master token information, if the token creation time and duration in the slave token information After confirming that the slave token information is still within the creation time, the slave token information can directly realize the login of the specified account. Therefore, the slave device can directly log in to the specified account based on the slave token information in the local cache without requesting to the server. information from the token.
  • the slave device still needs to exchange for a new token from the server based on the user ID, account ID and slave device ID in the slave token information. After logging in with the specified account after the secondary token information, the updated content of the new secondary token information is the token creation time and duration.
  • the slave device Based on the login instruction, the slave device exchanges brand new slave token information from the server according to the second master token information, and logs in the corresponding second account (such as account B) with the second master token information.
  • the corresponding second account such as account B
  • the slave device For the implementation process of the slave device logging into the corresponding second account based on the second master token information, reference may be made to the content of the foregoing steps S22 to S23.
  • the slave device changes its login account during the subsequent account login process, that is, it switches from the original designated account to the second account, it needs to notify the master device that logged in the designated account to inform the master The device no longer needs to send the master token information corresponding to the specified account to the slave device.
  • the controller is further configured to perform the following steps:
  • a monitoring cancel instruction is generated, and the monitoring cancel instruction is used to instruct the slave device to stop receiving the master token information sent by the master device.
  • the slave token information used to log in to the specified account can be obtained from the local cache of the slave device, if the received second master token information is consistent with the account ID of the slave token information , it means that the master device that generates the second master token information can be the master device that originally logged in with the specified account.
  • the master device that originally logged into the designated account switches to log in to the second account, so that the master device that generates the second master token information is still the master device that originally logged into the designated account. At this point, there is no need to cancel token monitoring.
  • the second is to log in the second account as another master device. At this time, the master device switches, so that the master device that generates the second master token information is switched to another display device, which is different from the master device that originally logged in with the specified account. At this point, the token monitoring needs to be canceled.
  • the process of the slave device judging whether there is slave token information in the local cache occurs in the scenario where the slave device receives the second master token information, by judging whether there is slave token information in the local cache card information, and the result of comparing the second master token information with the account ID of the slave token information to determine the target token information required for account login, and to determine the target account to be logged in from the slave device, and then realize the slave device The account of the device is automatically logged in.
  • the process of the slave device judging whether there is slave token information in the local cache can also occur in any scenario where the slave device is in the monitoring state, such as the re-login process of subsequent account login automatically. At this time, there is no need for the slave device to trigger the judgment process after receiving the second master token information sent by the master device. As long as the slave device needs to log in to the account again, it can read the slave token information in the local cache in real time to realize the account login.
  • the slave token information can be obtained directly from the local cache of the slave device to log in to the specified account. If the secondary token information in the local cache is invalid, it is necessary to exchange new secondary token information from the server to realize the login of the specified account.
  • the specific implementation process may include: during the automatic account login process of the slave device, if the slave device needs to log in to the account again, at this time, the slave device obtains the login information from the local cache.
  • the login information refers to the information required for logging in to the previous account. information from the token.
  • the corresponding account login process is automatically performed based on the master token information sent by the master device. If there is login information in the local cache, the corresponding account can be directly logged in based on the login information to realize automatic account login.
  • an implementation scenario can be: if there is no login information in the local cache, monitor the The event of the arrival of the token will automatically log in after receiving it. If there is login information in the cache and it has not expired, it will be automatically logged in directly. If you receive new token information and the account ID is inconsistent with the cached one, you can switch the login information through the interface prompt.
  • Another implementation scenario may be: when the slave device receives the second primary token information sent by the master device, if the login information does not exist in the local cache, log in the corresponding account based on the second primary token information.
  • the slave device can automatically log in the account based on the local cache, so as to log in the same account as the master device, without the need for the user to log in manually, and realize the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.
  • Fig. 9 is a flowchart of a multi-device cooperative login method executed by a server according to some embodiments.
  • a server provided in an embodiment of the present application includes: a controller. When executing the multi-device cooperative login method shown in FIG. 7 and FIG. 9 , the controller is configured to perform the following steps:
  • the user wants to log in to multiple display devices with an account.
  • the multi-device collaborative login method provided by this application, when performing account login, the user can only perform manual account login on one display device, while the other display devices All devices can realize automatic account login.
  • the display device that the user performs manual account login is the master device, and the other display devices that automatically perform account login are slave devices.
  • the master device generates an account login request based on the user login information input by the user, and sends it to the server to access a login (login) interface of the server.
  • the server can verify the user login information carried in the account login request, and the user login information includes a user name and a password.
  • the account login request carries user login information, user ID and device ID, the user ID is used to identify the user, and the device ID is used to identify the device that implements account login.
  • the server compares the user name and password in the user login information with the user name and password stored in the database configured by the server. If they are consistent, it means that the user name and password entered by the user are correct and the account can be logged in.
  • Token information is a string of numbers, including user ID, device ID, account ID, and token creation. time and duration.
  • the master token information is used to realize the specified account corresponding to the user login information of the master device and the slave device, the user ID is used to identify the user, the device ID is used to identify the device that realizes the account login, and the account ID is used to identify the account after verification (username and password), the token creation time is the valid period of the token, and the duration is the valid period of the token.
  • the server After verifying and passing the user login information based on the account login request sent by the master device, the server obtains the user ID and device ID carried in the account login request, obtains the account ID corresponding to the account information from the server database, and generates a token Creation time and duration, based on user ID, device ID, account ID, and token creation time and duration, the main token information is generated immediately and returned to the main device to realize the account login of the main device.
  • the master device After the master device receives the master token information generated by the server after verifying the user login information and passing it, the master device can log in to the corresponding account based on the account ID in the master token information, that is, realize the designated account based on the user name and password entered by the user. login.
  • the master device sends the master token information to the slave device, and the slave device is used to request the slave token information from the server based on the master token information to
  • the account ID in the information logs into the specified account corresponding to the user login information, and the slave device refers to the device that is in a pairing relationship with the master device.
  • the master device and the slave device log in with the same designated account, so that after the user logs in on one device, he can automatically log in on other devices in the family, which is not only convenient for the user to log in, but also improves the user experience.
  • the slave device ID and the master token information carried in the automatic login request are verified. Then generate slave token information based on the master token information and send it to the slave device.
  • the master token information includes user ID, account ID, token creation time and duration
  • the slave token information includes user ID, account ID, slave The token ID, the updated token creation time and duration, so that the slave device can log in to the specified account based on the account ID in the slave token information, and the slave device is a device in a pairing relationship with the master device.
  • each slave device After each slave device receives the master token information sent by the master device, it can request the slave token information from the server based on the master token information, and automatically log in to the specified account corresponding to the specified ID in the slave token information, so that the slave device and the master The devices log in with the same account.
  • the slave device When the slave device automatically logs in to the same designated account based on the master token information sent by the master device, it needs to verify the information authority with the server, and the designated account can only be logged in after the verification is passed. At this point, the slave device generates an automatic login request based on the primary token information, and sends it to the server.
  • the slave device In order to prevent the display devices of other users from connecting to the Internet of Things of the local user's home environment through their own Bluetooth modules, causing the display devices of other users to automatically log in to the account used by the local user, resulting in information leakage, it is necessary to request automatic login.
  • the slave device performs verification. For this reason, the slave device ID must be carried in the automatic login request generated by the slave device to identify the slave device. At this time, the slave device will generate an automatic login request with its own slave device ID and the user ID, account ID, token creation time and duration in the master token information, and send it to the server.
  • the server responds to the automatic login request sent by the slave device, and verifies the slave device ID and master token information carried in the automatic login request.
  • the verification content includes checking whether the slave device allows automatic login, whether the slave device to perform automatic login is a preset device, and whether the master token information is valid.
  • Preset devices refer to display devices in the same home environment, such as display devices in the same Internet of Things.
  • the main token information includes user ID, main device ID, account ID, token creation time and duration, and the main device ID is used to identify the source of the main token information. Therefore, it is possible to check whether the slave device allows automatic login based on the user ID and account ID, judge whether the slave device currently requiring automatic account login is a preset device based on the slave device ID, and judge the master token based on the token creation time and duration information is valid.
  • the above three judging processes need to pass the verification of the previous judging process before the next judging process can be executed, and the server will generate slave token information only after the verification of the three judging processes is passed.
  • the controller executes the automatic login request generated based on the slave device ID after receiving the master token information sent by the master device in response to the slave device, and the slave device ID and the master token information carried in the automatic login request For verification, is further configured to perform the following steps:
  • the account ID, token creation time and duration in the master token information are acquired.
  • the slave device If there is a registered account ID used to identify the specified account of the master device login in the server, the slave device is allowed to perform an automatic login process, and it is checked whether the display device corresponding to the slave device ID is a preset device, and the default device refers to Display devices that are in the same Internet of Things for Bluetooth pairing connection;
  • the display device corresponding to the slave device ID is a preset device, check whether the token creation time and duration are valid.
  • the server After the server receives the automatic login request sent by any slave device, it obtains the slave device ID, user ID, account ID, token creation time and duration carried in the automatic login request to check whether the slave device allows automatic login Check if the login information is valid.
  • the account ID in the primary token information carried in the automatic login request sent by the slave device is compared with the registered account ID stored on the server side, and the registered account ID is used to identify the master device Logged in account. If an account is logged in by the main device, the server will store the corresponding account information, such as the registered account ID. Therefore, if the server stores the registered account ID, the account ID is consistent with the registered account ID, indicating that the specified account corresponding to the account ID is logged in by the master device, and the slave device is allowed to respond to the token information of the master device to realize the corresponding account ID. automatic login. If the account ID is not stored on the server, the account ID is inconsistent with the registered account ID, indicating that the specified account corresponding to the account ID has not been logged in by the master device, and the slave device is not allowed to perform the automatic login account process.
  • the conditions for the slave device to perform the automatic login process include: responding to the master token information sent by the master device (that is, the master device triggers automatic account login of the slave device), and the master device has logged into the corresponding account.
  • the preset device refers to a display device that is in the same Internet of Things for Bluetooth pairing connection .
  • the slave device ID of the slave device can be connected with the Bluetooth pairing connection with the slave device stored on the server side in the same Internet of Things as the master device If the slave device ID is consistent with one of the display device IDs in the same Internet of Things, it can be determined that the slave device is a preset device, and the automatic account login process can be performed. If the comparisons are inconsistent, it means that the slave device is the display device of another user, and cannot log in with the same account as the display device of the local user to avoid information leakage, that is, the subsequent automatic account login process cannot be performed.
  • the server checks whether the slave device currently requesting automatic account login is a preset device
  • another method can also be used: when the slave device uploads the master token information, the slave device ID is also required, and the server After verifying that the slave device can realize the automatic account login function, the server parses out the account information based on the account ID from the master token information, and then obtains a list of devices bound to the account information or logged in. Determine whether the slave device ID is in the device list. If it is, the verification is passed, and then the master token verification is performed; if not, the slave device does not perform the automatic login process.
  • the slave device Based on the comparison between the slave device ID and the device list that has logged in to the account, if the slave device is another user's device, since the other user's device cannot log in to the account, there will be no other user's device identifier in the device list. Therefore, if the comparison fails, it can prevent the account from being automatically logged in on the outsider's device after the Bluetooth connection of the outsider's device. In this scenario, in order to have the IDs of each display device in the home IoT in the device list, the user needs to actively log in to the account once in the new device, so that the device ID of the new device can be added to the device list corresponding to the account in the server.
  • the next verification process can be performed, that is, verifying whether the login information is valid.
  • the token creation time and duration in the primary token information carried in the automatic login request are obtained, so as to determine whether the primary token information is valid based on the token creation time and duration.
  • both the token creation time and the duration satisfy corresponding time conditions, it is determined that the token creation time and duration are both valid. For example, if the token creation time has not expired, and the generation time of the main token information does not exceed the time length threshold, it can be determined that both the token creation time and the time length are valid.
  • the server After the server verifies that the slave device is allowed to log in automatically based on the master token information, the slave device is a default device, and the login information (master token information) is valid, it can determine the slave device ID and Primary token information verification passed.
  • the server After the server verifies that the slave device ID and master token information carried in the automatic login request sent by the slave device pass, it will generate slave token information for realizing the login of the specified account by the slave device. Since the slave token information is updated based on the master token information sent by the master device, the role of the slave token information is for the slave device to log in with an account, and the slave device needs to log in with the same specified account as the master device. Therefore, the user ID and account ID in the master token information sent by the master device do not need to be updated, but the device ID used to identify the slave device and the token creation time and duration for the slave device to log in within the time limit need to be updated.
  • the update content includes replacing the device ID of the original master device in the master token information with the device ID of the slave device, and updating the token creation time and duration, so that the updated slave token information includes user ID, slave device ID, Account ID, new token creation time and new duration.
  • the starting point of the new token creation time is calculated from the time when the server verifies the primary token information.
  • the server After the server verifies that the master token information carried in the automatic login request sent by the slave device has passed, it will return the synchronously generated slave token information to the slave device, so that the slave device can log in to the specified account based on the slave token information.
  • the slave token information After the slave device receives the slave token information returned by the server, the slave token information includes the user ID, slave device ID, account ID, new token creation time, and new duration, and can then log in to the designated account ID corresponding to the slave token information.
  • Account the specified account is the same as the specified account logged in on the main device.
  • the server when multiple display devices perform cooperative login account, based on the account login request sent by the main device, generates the main token information after the verification of the user login information is passed, so as to facilitate the main device to log in and the user to log in The specified account corresponding to the information, and automatically send the master token information to the slave device with the pairing relationship.
  • the slave token information is generated after the verification of the master token information is passed and the slave device is allowed to log in to the account, so that the slave device can log in to the same designated account as the master device based on the slave token information .
  • the other display devices can log in automatically, without the need for the user to log in manually, achieving the purpose of multi-device collaborative login account. While providing convenience to users, improve user interaction experience.
  • the master device that first logs in to the account sends an account login request to the server based on the user login information input by the user, and the receiving server verifies the user login information and passes the verification.
  • the master token information After the master token information is generated, log in to the specified account, and at the same time, automatically send the master token information to the slave device with the pairing relationship.
  • the slave device generates an automatic login request based on the master token information and sends it to the server.
  • the server verifies the master token information, it generates the slave token information and allows the slave device to log in to the account, so that the slave device is based on the slave token information and the master device. Log in to the same designated account.
  • the method, the display device and the server can realize that after the user logs in the account preferentially on one of the display devices, the other display devices can automatically log in, without the need for the user to log in manually, and realize the purpose of multi-device cooperative login account. While providing convenience to users, account login efficiency is high, which improves user interaction experience.
  • the multiple terminal devices can also control the corresponding terminal devices to perform corresponding operations by voice after they log in to the account cooperatively.
  • FIG. 10 is an application scenario diagram of a terminal device according to some embodiments.
  • terminal equipment such as TV 103, air conditioner 104, refrigerator 105, and washing machine 106 in the home can be connected to mobile terminal 101 and server 102 through the Internet of Things module, and mobile terminal 101 and terminal equipment can be connected through LAN or WAN for data transmission to realize the control and management of terminal equipment.
  • IoT modules can be built into individual end devices.
  • the speaker If "TV fast forward" is received, the speaker first judges whether there is an operation corresponding to the corresponding TV corresponding to the mapping relationship, and if so, sends an operation command to trigger TV fast forward to the TV; if not, the feedback indicates that there is no Recognized prompt information.
  • the user In the existing operation, the user must specify the target device to execute the operation intention, otherwise the speaker will input the operation intended to be performed by the speaker by default.
  • the speech recognition device may play a prompt to the user representing several devices for which mappings exist.
  • the determination of the mapping relationship between the speech recognition device and other devices can be performed in a local module or in the cloud.
  • the local modules may be housed within the speech recognition device or fixed to each other. They can also be separate objects.
  • Fig. 11 is another application scenario diagram of a terminal device according to some embodiments.
  • the voice recognition device converts the voice message into a voice command through the voice application and transmits it to the server 102.
  • the server 102 inquires about the terminal equipment currently configured by the user.
  • the server 102 After finding out that there are TVs, refrigerators and air conditioners, It will give feedback to the user through the voice recognition device, "You have 3 devices, which one do you want to use?" The server 102 will further determine that there are two terminal devices, a TV and an air conditioner, in the living room at this time, and feed back to the user through the voice recognition device "You have 3 devices in the living room, which one do you want to use?" Let's play", so far, the server 102 will control the TV to play the XXX song.
  • the speech recognition device may be a device with a radio function such as a remote controller and a speaker.
  • the voice recognition device may also be a display device with voice recognition and control functions, such as a TV or a mobile device.
  • the server 102 needs to perform multiple rounds of voice interaction with the user, so that the user can actively make various prompts, and finally determine the terminal device to execute the command.
  • the above illustrates the process of multiple interactions with the user.
  • the present application provides a server in some embodiments, and the server is configured to perform a voice interaction process.
  • the voice interaction process will be described below with reference to the accompanying drawings.
  • methods performed in server 102 may be performed in one other device or voice recognition device.
  • follow-up uses server execution as an example.
  • the speech recognition devices and the other devices can be connected to the server through the network, and the user can log in to the server through a pre-operation on a certain device.
  • the same account further enables the server to store a mapping relationship between a user identifier (that is, a user ID) for identifying a user and a device.
  • the speech recognition device can receive the speech input by the user at any moment.
  • the user may send a voice command to the voice recognition device based on his own needs, and control the corresponding terminal device to execute the voice command by voice. For example, when the user wants to use the target device to watch a XXX movie, he can input the wake-up word "Hi! XX" to the voice recognition device, and then input a voice message of "Play XXX movie" to the voice recognition device.
  • the target device and the voice recognition device are two independent devices.
  • the target device can have its own voice receiving device for voice control.
  • the target device and the voice recognition device are not the relationship between the TV and the TV remote control that is only used for the TV. Not the relationship between the TV processor and the audio receiver on the TV.
  • the wake word and voice command can be entered continuously.
  • the voice recognition device can recognize voice commands contained in sentences containing wake words.
  • the voice recognition device can integrate sound collection and voice analysis functions.
  • the radio function is to receive the voice message sent by the user
  • the voice analysis function refers to extracting the key part of the voice message sent by the user.
  • the key part can reflect the user's intention or the content to be done, and analyze the user's intention Afterwards, it is converted into a voice command, and the voice command may be an executable command format agreed between the voice recognition device and the server 102 .
  • the command format may include command (command) and parameters (parameter).
  • the voice command is sent to the server 102 .
  • adding the corresponding user identifier to the voice command facilitates the server 102 to identify which user sent the voice command, and also facilitates the user to search for all terminal devices configured by the user.
  • the speech recognition device may have an independent audio-visual display system, which may decode and play audio and video streams.
  • the target device may have an independent audio-visual display system capable of decoding and playing audio and video streams.
  • S301 Receive a voice instruction including a user identifier sent by a voice recognition device.
  • the speech recognition device may use the text converted from the speech as a speech instruction and send it to the server.
  • the server 102 analyzes the voice command to obtain the user identifier and the user's intention conveyed by the voice command.
  • the server determines the device type corresponding to the command by analyzing the voice command and according to the keywords in the parsed text, where the device type refers to the functional authority of the terminal device.
  • mapping relationship between keywords and device types may be cached in the server, and in some embodiments, a trained keyword-device type neural network model may also be stored.
  • the server 102 searches the database for all terminal devices related to the user ID, that is, searches for all terminal devices configured by the user. This is because, if device discovery is based on near-field communication, the device can discover all the devices it scans, including its own and/or devices of other users in close proximity, but some of these devices do not belong to the user , even if it is judged by the LAN, it is possible to initially select the scenario where a guest device is connected to the LAN. By pre-establishing the association between the user ID and its own equipment in a server with a management function, the server 102 can determine the equipment actually owned by the user according to the user ID.
  • the terminal device refers to other devices associated with the user identifier except the speech recognition device.
  • the recognition of the voice command needs to determine the type of the voice command, and then compare it with the types that can be executed by each terminal device, and then determine the terminal device that can execute the voice command.
  • the execution types of each terminal device may be pre-calibrated, for example, the kitchen refrigerator corresponds to freezing, refrigeration, recipe recommendation, and ingredient identification. It may also be determined according to the device identifier when a newly added device is scanned. For example, when the newly added device identifier indicates that the device is a refrigerator, an association between the device identifier and freezing, refrigeration, recipe recommendation, food material identification, etc. is established.
  • the server 102 may also load all terminal devices and device attributes related to the user ID from the database into the cache. Subsequently, when the server 102 searches for the terminal device again, it can directly search in the cache, thereby speeding up the speed at which the server 102 searches for the terminal device.
  • the device attributes include inherent attributes such as the location of the terminal device, the name of the terminal device, and the ID of the terminal device.
  • the server 102 if the server 102 does not find a terminal device related to the user identifier, it means that the user has not configured a terminal device. At this time, the server 102 needs to feed back parameters indicating that there is no terminal device to the speech recognition device. The voice recognition device broadcasts that there is no terminal device according to the received parameter representing the absence of the terminal device.
  • S304 When there is a terminal device related to the user identifier, use the preset filtering rules to filter out the most matching terminal device, and feed back the parameters representing the most matching terminal device, so that the voice recognition device broadcasts the existence of the execution voice The terminal device with the instruction, and controls the most matching terminal device to execute the voice instruction.
  • the preset filtering rule filter refers to a rule for filtering out a preset rule conforming to the user's intention/a device capable of executing the voice instruction. For example, when the user inputs the voice of "play folk music", the voice recognition device sends the instruction of playing folk music to the server, and the server recognizes that the type of user's intention is to play music, and then determines the terminal device that can perform this type of operation according to the type .
  • the terminal device related to the user identifier when there is a terminal device related to the user identifier, the terminal device related to the user identifier is used to execute the voice instruction.
  • the voice instruction when there is a terminal device related to the user identifier, it is also necessary to determine the number of terminal devices and/or the functions that the terminal device can perform.
  • the function that the terminal device can perform is the terminal Device function permissions.
  • the voice instruction when the voice instruction is an instruction to play a video, it is necessary to determine the device that can play the video, and the device that cannot perform video playback will be screened out. When the voice instruction is an instruction for cooling at 26°, cooling cannot be performed. The devices that are screened out, use the voice command that can be executed by the cooling device.
  • the server 102 is configured with preset filtering rules.
  • the filtering rule characterizes the mapping relationship between the voice instruction and the terminal device.
  • the filter rules include the first set of rules and the second set of rules, or only the first set of rules do not include the second set of rules, wherein the first set of rules refers to the necessary rules, and the second group of rules refers to the rules that are superimposed and utilized one by one when the best matching terminal device is not screened out.
  • the first group of rules includes the mapping relationship between the device type and each device, such as Table 1.
  • the first group of rules may be a judgment on the number corresponding to user identifiers.
  • the second rule needs to be further used for further screening.
  • the optional device is determined according to the device type and the mapping relationship of each device, if the number of the device is 1, the voice command is directly sent to the device; if it is 0, the feedback is irrelevant.
  • the device to execute is given to the voice recognition device. If there are more than one, it can be based on one or any combination of location/installation time/usage frequency/device that executed this type of command last/start time/signal strength/priority, etc. to filter as the second set of rules.
  • S401 Use the first set of rules to screen terminal devices related to the user identifier of the current terminal device.
  • the server 102 when terminal devices are screened by using the sub-rules of terminal device function rights, the server 102 respectively detects function rights of multiple terminal devices related to the user identifier.
  • the server 102 selects the corresponding terminal device as the candidate terminal device.
  • the server 102 excludes the corresponding terminal device.
  • the server 102 when using the first set of rules to filter the terminal devices related to the user identifier, if none of the terminal devices actually owned by the user has the corresponding functional authority, the server 102 needs to feed back to the voice recognition device There are parameters for the end device.
  • the voice recognition device broadcasts that there is no terminal device according to the received parameter representing the absence of the terminal device.
  • the server 102 when using the first set of rules to filter terminal devices related to the user identifier, if the terminal device actually owned by the user has the corresponding functional authority, the server 102 also needs to confirm that the terminal device with the corresponding functional authority quantity.
  • the server 102 when the server 102 detects that there is only one terminal device with the authority to execute the voice command, it feeds back to the voice recognition device the parameters representing the terminal device with the corresponding authority, and the voice recognition device exists according to the token.
  • the parameters of the terminal equipment with the corresponding authority are broadcast to the terminal equipment with the corresponding authority.
  • the server 102 also needs to control the current terminal device to perform corresponding operations according to the user's intention in the voice command.
  • the server 102 sends the voice instruction to the current terminal device, and the current terminal device receives the voice instruction and performs a corresponding operation according to the voice instruction.
  • the user is currently only equipped with a sweeping robot. If the user inputs a voice message of "sweeping the floor", the voice device will transmit the voice message to the server 102.
  • the server 102 finds that the current user is only equipped with a sweeping robot. With cleaning function. Therefore, the server 102 controls the voice recognition device to broadcast "the sweeping robot starts to sweep the floor", and controls the sweeping robot to perform the sweeping function.
  • the server 102 controls the speech recognition device to broadcast, it may feed back the parameters representing the terminal device with corresponding authority to the speech recognition device through a long link.
  • the sub-rules in the second set of rules are used to screen the multiple terminal devices that have the authority to execute the voice command one by one in order of priority from high to low. Terminal devices with the above-mentioned voice command permissions until a terminal device that best matches is screened out.
  • the second set of rules preset in the server 102 includes a user frequency sub-rule, a distance from a speech recognition device sub-rule, and a terminal device priority sub-rule.
  • the user frequency sub-rule refers to the number of times the user has executed similar voice commands through the terminal device, for example, the number of times the user plays video data such as A movie and B variety show through the TV, or the user plays C song and D song through the speaker.
  • the frequency of audio data such as A movie, B variety show, C song, D song, etc. can be divided into media data.
  • “song" and "play D song” can be regarded as similar voice commands, both of which are play-type voice commands.
  • the distance sub-rule to the speech recognition device refers to the distance between each terminal device and the speech recognition device.
  • the terminal device priority sub-rule refers to the priority of each terminal device set by the user when executing a voice command. For example, corresponding to the voice command for playing media assets, the priority of the TV is higher than that of the refrigerator screen. Among them, the terminal device The priority of the mobile terminal 101 can be set by the user through the application program.
  • a plurality of sub-rules in the above-mentioned second group of rules can be set with corresponding priorities by the user, for example, the priority of setting the priority of the sub-rule of user usage frequency is higher than the priority of the sub-rule of the distance from the speech recognition device, The priority of the terminal device priority sub-rule is higher than that of the user frequency sub-rule.
  • the server 102 uses filtering rules to filter terminal devices to filter out one of the most suitable terminal devices for executing the current voice command, that is to say, at the end of the screening, the number of the most matching terminal devices for 1 pc. After screening by the first set of rules, as long as the number of corresponding terminal devices is not unique, the server 102 will use the sub-rules in the second set of rules to screen the remaining terminal devices one by one according to the priority.
  • the server 102 finds out that the user is equipped with a TV, sound box, air conditioner, refrigerator, and washing machine. If the TV, sound box and refrigerator have the playback function, the TV, sound box and refrigerator need to be further screened through the second set of rules.
  • the server 102 further filters through the distance sub-rule with the speech recognition device, and through the device location information in the device attributes in the terminal device, the server 102 judges that the TV, the sound box and the speech recognition device are all in the living room, that is, the distance is relatively close, and the refrigerator In the kitchen, the distance from the speech recognition equipment is relatively long, and based on this, the refrigerator is excluded. Since there are still two candidate terminal devices left, i.e. TV and sound box, further screening is required, and the server 102 continues to filter through the terminal device priority sub-rules. The priority of the speaker, so the server 102 selects the TV as the most matching terminal device.
  • the server 102 finds out that the user is equipped with a TV, a speaker, an air conditioner, a refrigerator, and an electric fan. Obtaining that the air conditioner and the electric fan have the function of adjusting the room temperature, the server 102 needs to further screen the air conditioner and the electric fan through the second set of rules. The server 102 continues to filter through the terminal device priority sub-rule. For voice commands such as room temperature adjustment, the priority of the air conditioner set by the user is higher than that of the electric fan, so the server 102 selects the air conditioner as the most matching terminal device. Of course, users can also set the priority of the electric fan higher than that of the air conditioner according to their own needs.
  • the server 102 when using the user frequency sub-rule to screen terminal devices, the server 102 respectively detects the execution frequencies of multiple terminal devices related to the user identifier, wherein the execution frequency refers to the terminal device The number of times similar voice commands have been executed in historical behaviors. The server 102 will retain the terminal device with the highest execution frequency.
  • the server 102 when terminal devices are screened using the sub-rule of distance from a voice recognition device, the server 102 respectively detects the distances between a plurality of terminal devices related to the user identifier and the voice recognition device. The server 102 will reserve the terminal device with the closest distance to the speech recognition device.
  • the server 102 when using the terminal device priority to screen terminal devices, reserves the terminal device with the highest priority according to the terminal device priority set by the user.
  • the server when the server responds to the voice command issued by the user and inquires that there are currently multiple terminal devices, it can select the terminal device that will finally execute the command based on the filtering rules, so as to avoid multiple voice interactions between the user and the voice recognition device process to improve user experience.
  • the present application also provides a voice control method in some embodiments, the method includes: the server 102 receives the voice command containing the user ID sent by the voice recognition device, and searches for the voice command related to the user ID. all terminal equipment. When there is no terminal device related to the user identifier, the server 102 feeds back parameters representing the absence of the terminal device, so that the voice recognition device broadcasts that there is no terminal device executing the voice command. . When there is a terminal device related to the user identifier, the server 102 uses preset filtering rules to filter out the most matching terminal device, and feeds back parameters representing the most matching terminal device, so that the voice recognition device broadcasts the most matching terminal device. terminal device, and control the most matching terminal device to execute the voice command.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Telephonic Communication Services (AREA)

Abstract

La présente demande concerne un dispositif terminal, un serveur et un procédé de connexion de collaboration multi-dispositifs. Le dispositif terminal comprend un dispositif de commande configuré pour : recevoir des informations de connexion utilisateur entrées par un utilisateur de façon à ce que le dispositif terminal soit utilisé comme dispositif maître, les informations de connexion utilisateur correspondant à un compte spécifié ; envoyer une demande de connexion au compte à un serveur d'après les informations de connexion de l'utilisateur pour demander des informations de jeton maître utilisées pour la connexion, les informations de jeton maître étant générées par le serveur d'après le succès de la vérification des informations de connexion utilisateur, et les informations de jeton maître comprenant un identifiant de compte utilisé pour identifier le compte spécifié ; recevoir les informations de jeton maître du serveur, puis se connecter au compte spécifié correspondant d'après l'identifiant de compte dans les informations de jeton maître ; et envoyer les informations de jeton maître à un dispositif esclave de façon à ce que le dispositif esclave envoie une demande de connexion automatique au serveur d'après les informations de jeton maître pour demander des informations de jeton esclave utilisées pour la connexion, puis se connecte au compte spécifié correspondant d'après l'identifiant de compte dans les informations de jeton esclave, le dispositif esclave étant en relation d'appariement avec le dispositif maître.
PCT/CN2022/102579 2021-08-11 2022-06-30 Dispositif terminal, serveur et procédé de connexion de collaboration multi-dispositifs WO2023016126A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202280050101.2A CN117652134A (zh) 2021-08-11 2022-06-30 终端设备、服务器及多设备协同登录方法

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110917713.4 2021-08-11
CN202110917713.4A CN115910050A (zh) 2021-08-11 2021-08-11 服务器及语音控制方法
CN202111093415.4 2021-09-17
CN202111093415.4A CN115834095A (zh) 2021-09-17 2021-09-17 一种多设备协同登录方法及显示设备、服务器

Publications (1)

Publication Number Publication Date
WO2023016126A1 true WO2023016126A1 (fr) 2023-02-16

Family

ID=85200535

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/102579 WO2023016126A1 (fr) 2021-08-11 2022-06-30 Dispositif terminal, serveur et procédé de connexion de collaboration multi-dispositifs

Country Status (2)

Country Link
CN (1) CN117652134A (fr)
WO (1) WO2023016126A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103118032A (zh) * 2013-03-01 2013-05-22 北京奇艺世纪科技有限公司 一种终端登录账号同步的方法、终端与服务器
CN104079565A (zh) * 2014-06-13 2014-10-01 小米科技有限责任公司 授权方法和装置
CN106131047A (zh) * 2016-08-12 2016-11-16 乐视控股（北京）有限公司 账号登录方法及相关设备、账号登录系统
US20170195429A1 (en) * 2015-12-30 2017-07-06 Symantec Corporation Systems and methods for facilitating single sign-on for multiple devices
CN110035305A (zh) * 2019-03-08 2019-07-19 佛山市云米电器科技有限公司 基于智能家居的多屏共享账号信息的视频跟随方法及系统
CN113225188A (zh) * 2020-01-19 2021-08-06 华为技术有限公司 登录认证方法、装置与系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103118032A (zh) * 2013-03-01 2013-05-22 北京奇艺世纪科技有限公司 一种终端登录账号同步的方法、终端与服务器
CN104079565A (zh) * 2014-06-13 2014-10-01 小米科技有限责任公司 授权方法和装置
US20170195429A1 (en) * 2015-12-30 2017-07-06 Symantec Corporation Systems and methods for facilitating single sign-on for multiple devices
CN106131047A (zh) * 2016-08-12 2016-11-16 乐视控股（北京）有限公司 账号登录方法及相关设备、账号登录系统
CN110035305A (zh) * 2019-03-08 2019-07-19 佛山市云米电器科技有限公司 基于智能家居的多屏共享账号信息的视频跟随方法及系统
CN113225188A (zh) * 2020-01-19 2021-08-06 华为技术有限公司 登录认证方法、装置与系统

Also Published As

Publication number Publication date
CN117652134A (zh) 2024-03-05

Similar Documents

Publication Publication Date Title
CN108022590B (zh) 语音接口设备处的聚焦会话
WO2018039814A1 (fr) Procédé, appareil et système de commande domestiques intelligents
US7024256B2 (en) Method, system, and computer program product for automatically managing components within a controlled environment
EP3206362B1 (fr) Procédé et appareil d'ouverture de session de compte
US20190304448A1 (en) Audio playback device and voice control method thereof
US9793962B2 (en) Processing near field communications between active/passive devices and a control system
WO2018010533A1 (fr) Système, procédé et appareil de commande de dispositif, et passerelle
JP2018531404A6 (ja) ホームオートメーションシステムの音声制御のための履歴ベースのキーフレーズの提案
JP2018531404A (ja) ホームオートメーションシステムの音声制御のための履歴ベースのキーフレーズの提案
US20120146918A1 (en) Remote Control of Electronic Devices Via Mobile Device
CN107976911B (zh) 设备控制方法、装置及系统
CN107635214B (zh) 基于蓝牙遥控器的响应方法、装置、系统及可读存储介质
EP2412107A1 (fr) Agencement pour gérer une communication sans fil entre dispositifs
WO2023138285A1 (fr) Procédé de liaison pour un dispositif domestique intelligent et terminal
CN111372222A (zh) 设备连接方法、系统以及相应的终端设备和服务器
CN109599100A (zh) 交互式电子设备控制系统、交互式电子设备及其控制方法
KR20200024813A (ko) 홈 네트워크 시스템에서 홈 디바이스를 원격으로 제어하는 방법 및 장치
WO2023016126A1 (fr) Dispositif terminal, serveur et procédé de connexion de collaboration multi-dispositifs
WO2022268136A1 (fr) Dispositif terminal et serveur pour commande vocale
CN111010322B (zh) 信息配置方法、装置、电子设备及存储介质
CN113296415A (zh) 智能家电控制方法、智能家电控制装置与系统
CN114301628A (zh) 智能家居设备认证方法、装置、设备、系统和存储介质
CN113300920A (zh) 基于家电控制群组的智能家电控制方法与控制设备
CN114286144A (zh) 一种控制装置和设备的配对方法及控制装置
WO2023284870A1 (fr) Procédé de commande et dispositif de commande

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22855110

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280050101.2

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE