WO2021238230A1

WO2021238230A1 - Smart home system and control method and device thereof

Info

Publication number: WO2021238230A1
Application number: PCT/CN2021/070632
Authority: WO
Inventors: 王宝军; 钱莉; 杨宇庭
Original assignee: 华为技术有限公司
Priority date: 2020-05-28
Filing date: 2021-01-07
Publication date: 2021-12-02
Also published as: CN111522250A; CN111522250B

Abstract

A smart home system and a control method and device thereof, applied to the technical field of smart home. The method comprises: receiving a screen paging indication sent by a user terminal; sending a paging request to at least one monitoring device according to the screen paging indication, wherein the monitoring device and a screen terminal have a mapping relationship; receiving monitoring data sent by at least one monitoring device, the monitoring data being used for indicating whether a target user is monitored; selecting a target screen terminal from the at least one screen terminal according to the monitoring data; and transferring real-time running data of a target application on the user terminal to the target screen terminal for displaying. When the user leaves the user terminal and transfers to other areas, a control center can transfer the real-time running data of the target application on the user terminal to the screen terminal of the area where the user is located, so that the user can continuously use the target application in a seamless connection manner, and the user is helped to get rid of constraints of terminals such as a mobile phone.

Description

Smart home system and its control method and device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on May 28, 2020, the application number is 202010464921.9, and the application name is "Smart Home System and Its Control Method and Device", the entire content of which is incorporated herein by reference Applying.

Technical field

This application relates to the field of smart home technology, and in particular to a smart home system and its control method and device.

Background technique

With the development of artificial intelligence (AI) technology and the improvement of people's living standards in recent years, people are paying more and more attention to the comfort, safety and convenience of their living environment. In order to meet the needs of the public, smart home systems (smart home) have been developed rapidly. Among them, the purpose of the smart home system is to use the Internet of Things technology to enable the various devices in the user’s home to serve more according to the user’s lifestyle, so as to create a more comfortable, more environmentally friendly, more convenient and smarter residence for the user. environment.

At present, when users are in the target environment (such as home environment, office environment, etc.), they often use terminal devices such as smartphones and personal computers (PC) for entertainment or office; but when users are far away from the corresponding terminal devices, users It is often impossible to continue to process events on the corresponding terminal device, for example, the user cannot continue to have a video conference with others, cannot process new email notifications, and so on. In other words, the current smart home system cannot free users from the shackles of corresponding terminal devices (such as entertainment terminals or office terminals), and the degree of intelligence is low, resulting in poor user experience.

Summary of the invention

The embodiments of the present application provide a smart home system and its control method and device, which can enable users to seamlessly continue to use target applications in different areas, help users get rid of the shackles of mobile phones and other terminals, and enable users to enjoy smart homes. The convenience brought by the system has improved the degree of intelligence of the smart home system.

In the first aspect, an embodiment of the present application provides a control method of a smart home system, which is applied to a control center, and the method includes: receiving a screen paging instruction sent by a user terminal; sending to at least one monitoring device according to the screen paging instruction Paging request, so that at least one monitoring device monitors the target user, and there is a mapping relationship between the monitoring device and the screen terminal; receiving monitoring data sent by at least one monitoring device, the monitoring data is used to indicate whether the target user is monitored; according to the monitoring data, Select a target screen terminal from at least one screen terminal, where the target screen terminal refers to the screen terminal corresponding to the first monitoring device, and the first monitoring device refers to the monitoring device that monitors the target user; based on the received target application sent by the user terminal The running status information of the target application is transferred to the target screen terminal for display, where the running status information includes the real-time running data of the target application, and/or the identification and progress of the target application.

That is to say, in this method, the control center transfers the real-time running data of the target application on the user terminal to the target screen terminal, so that when the user leaves the user terminal and transfers to the area where the target screen terminal is located, the user also The ability to seamlessly continue to use the target application helps users get rid of the shackles of mobile phones and other terminals, enables users to enjoy the convenience brought by the smart home system, and improves the degree of intelligence of the smart home system.

In a possible implementation manner, the monitoring data includes user characteristic information of the user collected by at least one monitoring device; according to the monitoring data, selecting a target screen terminal from at least one screen terminal includes: according to the user characteristic information, from Screen target users among users monitored by at least one monitoring device; according to the screening results, use the monitoring device that monitors the target user as the first monitoring device; use the first monitoring device according to the mapping relationship between the monitoring device and the screen terminal, Select the target screen terminal from at least one screen terminal.

In other words, when the monitoring data includes the user characteristic information of the user collected by the monitoring device, the control center can filter out the monitoring devices that have monitored the target user based on the user characteristic information, and based on the mapping between the monitoring device and the screen terminal Relationship, the target screen terminal is selected, thereby achieving the purpose of determining the target screen terminal on the side of the control center based on user characteristic information.

In a possible implementation manner, filtering target users from users monitored by at least one monitoring device according to user characteristic information includes: obtaining pre-stored first characteristic information of the user, and the first characteristic information includes the target user The characteristic information of the user; the user characteristic information is compared with the first characteristic information, and if the comparison is successful, the target user is screened.

In other words, when the control center screens target users based on user characteristic information, it can compare the user characteristic information with the pre-stored first characteristic information that includes characteristic information of the target user, so as to screen out the target user.

In a possible implementation manner, filtering target users from users monitored by at least one monitoring device according to user characteristic information includes: receiving second characteristic information of the user sent by the user terminal, where the second characteristic information is the user terminal It is collected before sending the screen paging instruction, and the second characteristic information includes characteristic information of the target user; the user characteristic information is compared with the second characteristic information, and if the comparison is successful, the target user is screened.

That is to say, when the control center screens target users based on user characteristic information, it can compare the user characteristic information with the second user characteristic information sent by the received user terminal and including the characteristic information of the target user, so as to filter out the target. user.

In a possible implementation manner, the monitoring data includes the monitoring result of at least one monitoring device, and the monitoring result is used to indicate whether the monitoring device has monitored the target user; according to the monitoring data, the target screen terminal is selected from the at least one screen terminal, Including: according to the monitoring result, the screen terminal corresponding to the first monitoring device is selected as the target screen terminal.

That is, when the monitoring data includes an indication of whether the monitoring device has monitored the target user, the screen terminal corresponding to the monitoring device that has monitored the target user can be directly selected as the target screen terminal.

In a possible implementation, there are multiple first monitoring devices and multiple target users. Selecting the target screen terminal from at least one screen terminal includes: obtaining each target user’s priority according to the monitored target user’s priority. The priority of the first monitoring device, and the screen terminal corresponding to the first monitoring device with the highest priority is selected as the target screen terminal.

In other words, when there are multiple monitoring devices for the target user, and there are multiple target users monitored (at this time, there are multiple users registered in the system, and at least two users are monitored by different screen terminals. ), the priority of the monitoring device is determined based on the priority of the target user; after that, the screen terminal corresponding to the monitoring device with the highest priority is selected as the target screen terminal to avoid the situation of erroneously transferring the real-time running data of the target application.

In a possible implementation, there are multiple first monitoring devices, and there is one target user, and selecting the target screen terminal from at least one screen terminal includes: monitoring the first monitoring device that is closest to the target user The screen terminal corresponding to the device is selected as the target screen terminal. That is to say, when there are multiple monitoring devices for the target user, and only one target user is monitored, the screen terminal corresponding to the monitoring device closest to the target user is selected as the target screen terminal to facilitate the target user. The user's manipulation and/or observation improves the intelligence of the smart home system.

In a possible implementation, before transferring the real-time running data of the target application to the target screen terminal for display, it also includes: sending a confirmation instruction to the target screen terminal to display the transfer prompt information on the target screen terminal. It is used to prompt whether to transfer the real-time operating data to the target screen terminal; to receive the first confirmation information that the target screen terminal responds to the transfer prompt information feedback, and the first confirmation information is used to instruct to transfer the real-time operating data to the target screen terminal. After receiving the first confirmation information feedback from the target screen terminal, the real-time operating data is transferred to the target screen terminal, so that the user can independently choose whether to perform the transfer operation, which improves the convenience of the smart home system.

In a possible implementation manner, before receiving the screen paging instruction sent by the user terminal, the method further includes: receiving a registration request sent by the user terminal, where the registration request is used to request to display real-time running data of the target application on the target screen terminal; Detect whether the target application adapts to at least one screen terminal; if the target application adapts to at least one screen terminal, add the target application to the registration list; if the target application fails to adapt to at least one screen terminal, send a prompt message to the user terminal , So that the user terminal will remind the user who initiated the registration request. Through the user's independent choice to add a target application, and a prompt is issued when the target application cannot be adapted to the screen terminal, the user experience is prompted.

In a possible implementation manner, based on the received operating status data of the target application sent by the user terminal, before transferring the real-time operating data of the target application to the target screen terminal for display, the method further includes: feeding back the selected data to the user terminal The information of the target screen terminal, so that the user terminal can send the running status information of the target application; receive the running status information of the target application sent by the user terminal.

In a possible implementation manner, the screen paging indication carries the running status information of the target application.

In a possible implementation manner, there are multiple monitoring devices, and the multiple monitoring devices perform user monitoring at the same time or in time sharing. By performing user monitoring at the same time, the accuracy of monitoring is improved; by time-sharing user monitoring, unnecessary monitoring is avoided, and the energy consumption of the smart home system is reduced.

In the second aspect, the embodiments of the present application provide a control method of a smart home system, which is applied to a control center, and the method includes: receiving a screen paging instruction sent by a user terminal; and sending a paging instruction to at least one screen terminal according to the screen paging instruction. Call request so that the screen terminal displays the transfer prompt information, which is used to prompt whether to transfer the real-time running data of the target application on the user terminal to the screen terminal; receive the second confirmation message fed back by at least one screen terminal, and send the feedback first 2. The screen terminal of the confirmation message is selected as the target screen terminal, and the second confirmation information is used to instruct the transfer of real-time running data to the screen terminal; based on the received running status information of the target application sent by the user terminal, the real-time running data is transferred to The target screen terminal displays, where the running status information includes real-time running data of the target application, and/or the identification and progress of the target application.

That is to say, in this method, the control center sends a paging request to the screen terminal whether to transfer the real-time running data of the target application to the screen terminal by means of broadcasting, etc., and transfers the real-time running data to the screen terminal that confirms the transfer , So that when leaving the user terminal, the user can independently select the screen terminal he needs to use, and can continue to use the target application seamlessly, helping users get rid of the shackles of mobile phones and other terminals, allowing users to enjoy the smart home system The convenience of coming has improved the degree of intelligence of the smart home system.

In a possible implementation manner, before sending a paging request to at least one screen terminal according to the screen paging instruction, the method further includes: sending a self-check instruction to at least one monitoring device, so that the monitoring device detects the target hardware on the monitoring device Whether there is a failure, the mapping relationship between the monitoring device and the screen terminal, the target hardware is used to monitor the target user; receiving self-check data fed back by at least one monitoring device; according to the self-check data, it is concluded that the target hardware on at least one monitoring device is present Fault.

That is to say, when the target hardware on the monitoring device fails, the control center can directly send a paging request to the screen terminal whether to transfer real-time running data, so that the user can choose independently, so that the target hardware on the monitoring device fails At the time, the target user can continue to use the target application seamlessly.

In a possible implementation manner, before receiving the screen paging instruction sent by the user terminal, the method further includes: receiving a registration request sent by the user terminal, where the registration request is used to request to display real-time running data of the target application on the target screen terminal; Detect whether the target application adapts to at least one screen terminal; if the target application adapts to at least one screen terminal, add the target application to the registration list; if the target application fails to adapt to at least one screen terminal, send a prompt message to the user terminal , So that the user terminal will remind the user who initiated the registration request.

In a possible implementation manner, based on the received running status information of the target application sent by the user terminal, before transferring the real-time running data to the target screen terminal for display, it also includes: feeding back the selected target screen terminal to the user terminal To enable the user terminal to send the running status information of the target application; to receive the running status information of the target application sent from the user terminal.

In a third aspect, an embodiment of the present application provides a control method of a smart home system, which is applied to a user terminal. The method includes: identifying that a trigger condition for screen paging is met, and the trigger condition includes detecting that the target user is not visible to the user terminal. At least one of the duration within the range reaches the first preset duration, and the duration of detecting that the target application is activated reaches the second preset duration; sends a screen paging instruction to the control center so that the control center can monitor at least one The device or the screen terminal sends a paging request to monitor the mapping relationship between the device and the screen terminal.

That is to say, when the user terminal recognizes that the screen paging trigger condition is met, the method sends a screen paging instruction to the control center, so that the control center sends a paging request to at least one monitoring device or screen terminal to facilitate control The center determines the screen terminal in the area where the target user is located, and then transfers the real-time running data of the target application on the user terminal to the screen terminal, so that the target user can continue to use the target application seamlessly.

In a possible implementation, the trigger condition includes detecting that the target user is not within the visible range of the user terminal for a first preset time period, and detecting that the target application is activated for a second preset time period. ; Before identifying that the trigger condition of the screen paging is met, it also includes: obtaining the type of the target application; according to the type, selecting the trigger condition associated with the type. By determining the trigger condition according to the type of the target application, the purpose of determining different trigger conditions according to different applications is achieved, and the control accuracy is improved.

In a possible implementation manner, before identifying that the trigger condition of the screen paging is satisfied, the method further includes: collecting second characteristic information of the user who uses the user terminal within a preset time, and sending the second characteristic information to the control center, So that the control center can screen out the target user from the monitoring data sent by the monitoring device according to the second characteristic information.

In a possible implementation manner, before identifying that the trigger condition of the screen paging is satisfied, the method further includes: collecting third characteristic information of the user who uses the user terminal within a preset time, and sending the third characteristic information to the monitoring device, So that the monitoring device judges whether the target user is monitored according to the third characteristic information.

In a possible implementation manner, before recognizing that the trigger condition of the screen paging is satisfied, it further includes: recognizing that the user terminal is in a target environment to avoid misdirection, where the target environment includes a home environment and an office environment.

In a possible implementation, the method further includes: receiving a registration request sent by the target user, where the registration request is used to request to display real-time running data of the target application on the target screen terminal; and sending the registration request to the control center to Make the control center detect whether the target application is compatible with at least one screen terminal. Through the user's independent choice to add the target application, the user experience is prompted

In a possible implementation manner, the screen paging instruction carries the operating status information of the target application, so that the control center transfers the real-time operating data of the target application to the target screen terminal for display based on the operating status information. The status information includes real-time running data of the target application, and/or the identification and progress of the target application.

In a possible implementation manner, after sending the screen paging instruction to the control center, it further includes: receiving the information that the control center feeds back to determine the target screen terminal, and sending the running status information of the target application to the control center to control Based on the running status information, the center transfers the real-time running data of the target application to the target screen terminal for display, where the running status information includes the real-time running data of the target application, and/or the identification and progress of the target application.

In a possible implementation manner, after sending the screen paging instruction to the control center, it further includes: receiving the information that the control center feeds back to determine the target screen terminal, and sending the running status information of the target application to the target screen terminal to The target screen terminal is made to display the real-time running data of the target application, where the running state information includes the real-time running data of the target application, and/or the identification and progress of the target application.

In a fourth aspect, an embodiment of the present application provides a control method of a smart home system, which is applied to a monitoring device, and the mapping relationship between the monitoring device and the screen terminal. The method includes: receiving a paging request sent by a control center, and a paging request After the control center receives the screen paging instruction sent by the user terminal; according to the paging request, monitor the target user, and send the monitoring data to the control center, so that the control center selects the target from at least one screen terminal according to the monitoring data Screen terminal, and transfer the real-time running data of the target application on the user terminal to the target screen terminal, and the monitoring data is used to indicate whether the target user is monitored.

In other words, after the monitoring device in this method receives the paging request, it starts to monitor the target user and feeds back the monitoring data to the control center. After that, the control center selects the target screen terminal based on the monitoring data and sets the target on the user terminal. The real-time running data of the application is transferred to the target screen terminal, so that when the user leaves the user terminal, the user can continue to use the target application seamlessly, helping the user to get rid of the shackles of the mobile phone and other terminals, allowing the user to enjoy the smart home The convenience brought by the system has improved the degree of intelligence of the smart home system.

In a possible implementation, the monitoring data includes monitoring results, which are used to indicate whether the monitoring device has monitored the target user; monitoring the target user includes: collecting user characteristic information of users within the monitoring range of the monitoring device, and according to the user The characteristic information, the monitoring result is obtained, and it can be concluded whether the target user is detected by the monitoring device.

In a possible implementation manner, obtaining the monitoring result according to the user characteristic information includes: obtaining the pre-stored first characteristic information of the user, the first characteristic information includes the characteristic information of the target user; and the user characteristic information is combined with the first characteristic information. The feature information is compared, and if the comparison is successful, the monitoring result is that the target user is monitored.

In a possible implementation manner, obtaining the monitoring result according to the user characteristic information includes: receiving the second characteristic information of the user sent by the control center, the second characteristic information is sent from the user terminal to the control center, and the second characteristic information is the user The terminal is collected before sending the screen paging instruction to the control center, and the second characteristic information includes the characteristic information of the target user; the user characteristic information is compared with the second characteristic information, if the comparison is successful, the monitoring result is that the target is monitored user.

In a possible implementation manner, obtaining the monitoring result according to the user characteristic information includes: receiving the third characteristic information of the user sent by the user terminal, and the third characteristic information is collected before the user terminal sends a screen paging instruction to the control center, And the third characteristic information includes characteristic information of the target user; the user characteristic information is compared with the third characteristic information, and if the comparison is successful, the monitoring result is that the target user is monitored. .

In the fifth aspect, the embodiments of the present application provide a method for controlling a smart home system, which is applied to a screen terminal. The method includes: receiving a paging request sent by a control center, and displaying a transfer prompt message, the paging request is received by the control center After the screen paging instruction sent by the user terminal, the transfer prompt message is used to prompt whether to transfer the real-time running data to the screen terminal; receive the transfer confirmation message issued by the user, and feedback the second confirmation message to the control center, the second confirmation The information is used to indicate the transfer of real-time running data to the screen terminal; to receive the real-time running data of the target application on the user terminal transferred from the control center, and to display the real-time running data.

In a sixth aspect, an embodiment of the present application provides a control device for a smart home system. The control device includes at least one processor configured to execute instructions stored in a memory, so that the control center executes the method in the first aspect , Or execute the method in the second aspect.

In a seventh aspect, an embodiment of the present application provides a control device for a smart home system. The control device includes at least one processor configured to execute instructions stored in a memory, so that a user terminal executes the method in the third aspect .

In an eighth aspect, an embodiment of the present application provides a control device for a smart home system. The control device includes at least one processor configured to execute instructions stored in a memory, so that the monitoring device executes the method in the fourth aspect .

In a ninth aspect, an embodiment of the present application provides a control device for a smart home system. The control device includes at least one processor configured to execute instructions stored in a memory, so that the screen terminal executes the method in the fifth aspect .

In a tenth aspect, an embodiment of the present application provides a control center for executing the method in the first aspect or the method in the second aspect.

In an eleventh aspect, an embodiment of the present application provides a user terminal for executing the method in the third aspect.

In the twelfth aspect, an embodiment of the present application provides a monitoring device for executing the method in the fourth aspect.

In the thirteenth aspect, an embodiment of the present application provides a screen terminal for executing the method in the fifth aspect.

In a fourteenth aspect, an embodiment of the present application provides a smart home system, including a control center, a user terminal monitoring device, and a screen terminal. The control center is used to execute the method in the first aspect, and the user terminal is used to execute the third In the method in the aspect, the monitoring device is used to perform the method in the fourth aspect.

In a fifteenth aspect, an embodiment of the present application provides a smart home system, including a control center, a user terminal, and a screen terminal. The control center is used to execute the method in the second aspect, and the user terminal is used to execute the third aspect. The screen terminal is used to execute the method in the fifth aspect.

In a sixteenth aspect, an embodiment of the present application provides a computer storage medium that stores instructions in the computer storage medium, and when the instructions run on a computer, the computer executes the method provided in any of the foregoing aspects.

In the seventeenth aspect, the embodiments of the present application provide a computer program product containing instructions, which when run on a computer, cause the computer to execute the method provided in any of the foregoing aspects.

Description of the drawings

FIG. 1 is a system architecture diagram of a smart home system provided by an embodiment of the present application;

2 is a schematic diagram of the hardware structure of a terminal provided by an embodiment of the application;

FIG. 3 is a schematic diagram of the hardware structure of a control center provided by an embodiment of the application;

4A and 4B jointly constitute a schematic diagram of an application scenario of the control method of the smart home system provided by the embodiment of the present application;

FIG. 5 is a schematic diagram of steps for adding a target application provided by an embodiment of the present application;

6A is a schematic diagram of a login interface of a smart home system provided by an embodiment of the present application;

6B is a schematic diagram of a user interface for adding a target application provided by an embodiment of the present application;

FIG. 7A is a schematic diagram of a user interface when a target application is successfully added according to an embodiment of the present application;

FIG. 7B is a schematic diagram of a user interface when a target application fails to be added according to an embodiment of the present application;

FIG. 8 is a communication schematic diagram of a control method of a smart home system provided by an embodiment of the present application;

FIG. 9A is a schematic diagram of a user interface of a user terminal provided by an embodiment of the present application;

FIG. 9B is a schematic diagram of a user interface of a user terminal provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of a user interface of a user terminal provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of a scenario in which a user terminal determines that a target user is within a visible range according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a scenario where there are multiple target users and multiple target screen terminals according to an embodiment of the present application;

FIG. 13 is a schematic diagram of a scenario in which there are one target user and multiple target screen terminals according to an embodiment of the present application;

FIG. 14 is a schematic diagram of steps for a user terminal to determine a trigger condition corresponding to a target application according to an embodiment of the present application;

15 is a schematic diagram of the steps of screening target users based on user characteristic information provided by an embodiment of the present application;

16 is a schematic diagram of another step of screening target users based on user characteristic information provided by an embodiment of the present application;

FIG. 17 is a communication schematic diagram of a control center inquiring a target screen terminal according to an embodiment of the present application; FIG.

FIG. 18 is a schematic diagram of a user interface of a screen terminal provided by an embodiment of the present application;

19A and 19B together constitute a schematic diagram of an application scenario of the control method of the smart home system provided by an embodiment of the present application;

FIG. 20 is a schematic diagram of an application scenario of a method for controlling a smart home system provided by an embodiment of the present application;

FIG. 21 is a communication schematic diagram of another method for controlling a smart home system according to an embodiment of the present application;

Fig. 22 is a schematic structural diagram of a control device of a smart home system provided by an embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to represent examples, illustrations, or illustrations. Any embodiment or design solution described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be construed as being more preferable or advantageous than other embodiments or design solutions. To be precise, words such as "exemplary", "for example" or "for example" are used to present related concepts in a specific manner.

In the description of the embodiments of the present application, the term "and/or" is merely an association relationship describing associated objects, indicating that there can be three types of relationships, for example, A and/or B can indicate: A alone exists, and A alone exists. There are B, and there are three situations of A and B at the same time. In addition, unless otherwise specified, the term "plurality" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals.

In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features. The terms "including", "including", "having" and their variations all mean "including but not limited to" unless otherwise specifically emphasized.

Fig. 1 is a system architecture diagram of a smart home system provided by an embodiment of the present application. As shown in Figure 1, the smart home system includes a user terminal 11, a control center 12, at least one screen terminal (such as screen terminals 13, 14), and at least one monitoring device (not shown in the figure); wherein, the monitoring device and the screen There is a mapping relationship between the terminals, and each screen terminal corresponds to at least one monitoring device; the

screen terminals

13, 14 are set in different positions. The user terminal 11 is a terminal used by the user in the first time period, and the screen terminal 13 or the screen terminal 14 is a terminal that the user needs to use in the second time period, wherein the first time period is earlier than the second time period. The user terminal 11 and at least one screen terminal (for example, screen terminals 13, 14) may be in the same area or in different areas. For example, the user terminal 11 may be in the living room, and the screen terminal 13 may be in the living room or bedroom.

In some examples, both the user terminal 11 and the screen terminal (such as screen terminals 13, 14) may be mobile phones, tablet computers, digital cameras, personal digital assistants (personal digital assistants, PDAs), wearable devices, laptop computers (laptop computers). ), smart TVs, Huawei smart screens and other electronic devices with display screens. Exemplary embodiments of electronic devices include, but are not limited to, electronic devices equipped with iOS, android, Windows, Harmony OS or other operating systems. The above-mentioned electronic device may also be other electronic devices, such as a laptop computer with a touch-sensitive surface (such as a touch panel). The embodiment of the present application does not specifically limit the type of electronic device.

Both the user terminal 11 and the screen terminal (for example, the screen terminal 13, 14) can be connected to the control center 12 through a network such as a wired network (Wired network) or a wireless network (Wireless network). For example, the network may be a local area network (LAN) or a wide area network (WAN) (such as the Internet). The network between the user terminal 11 and the screen terminal and the control center 12 can be implemented using any known network communication protocol. The above-mentioned network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus ( universal serial bus, firewire, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access, CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new air interface ( New radio, NR), bluetooth (bluetooth), wireless fidelity (Wi-Fi) and other communication protocols.

In addition, the user terminal 11 and the screen terminal (for example, the screen terminal 13, 14) may also be connected through a network such as a wired network (Wired network) or a wireless network (wireless network). For detailed network types, refer to the above description, which will not be repeated here.

It should be understood that the user terminal 11 may be equipped with an information collection device. For example, the information collection device can be used to collect user characteristic information in the environment, where the information collection device can be an image collection device, a sound collection device, and the like. In some examples, the image collection device may be a camera, and the sound collection device may be a receiver, a microphone, or the like. In other examples, the user terminal 11 may be configured with a positioning system for locating the location of the user terminal 11, such as the Global Positioning System (GPS) and the BeiDou Navigation Satellite System (BDS).

In some examples, the monitoring device may be an image acquisition device, a sound acquisition device, etc., where the image acquisition device may be a camera, and the sound acquisition device may be a receiver, a microphone, or the like. The monitoring device may be connected to the control center, the screen terminal and/or the user terminal through a wired network (Wired network) or a wireless network (Wireless network) and other networks. For detailed network types, refer to the above description, which will not be repeated here. Among them, the monitoring device is integrated or independent of the screen terminal, and there is a mapping relationship between the monitoring device and the screen. For the mapping relationship, it can be understood that: the monitoring device A corresponds to the screen terminal a, and the monitoring device B corresponds to the screen terminal b. In addition, when the monitoring device is independent of the screen terminal, the monitoring device can be set on a support such as a wall or a bracket that is the same or opposite to the screen terminal.

The following describes a schematic diagram of the hardware structure of a terminal in an embodiment of the present application, where the terminal is a user terminal 11 and/or a screen terminal (for example, screen terminals 13, 14).

Figure 2 shows a schematic diagram of the hardware structure of the terminal. As shown in FIG. 2, the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, and a battery 142, Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, A display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.

It is understandable that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem (modem), a graphics processing unit (GPU), and an image signal processor ( image Signal processor, ISP, controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU), etc. One or more. Among them, the different processing units may be independent devices or integrated in one or more processors.

A memory may also be provided in the processor 110 to store instructions and data. In some examples, the memory in the processor 110 is a cache memory. The memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory to avoid repeated access, reduce the waiting time of the processor 110, and improve the efficiency of the system. In some examples, the processor 110 may be used to parse the image data collected by the camera 193, analyze user characteristic information (such as facial features, posture characteristics, etc.) in the image, and identify and/or determine whether to monitor according to the user characteristic information When the target user or other users appear in the coverage area of the screen terminal; or determine whether the user is within the visible range of the user terminal 11 or the screen terminal according to user characteristic information and/or profile information of the user in the image data.

In some examples, the processor 110 may include one or more interfaces. The interface can include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (Pulse Code Modulation, PCM) interface, and a universal asynchronous receiver/transmitter (Universal Asynchronous Receiver/ Transmitter, UART) interface, mobile industry processor interface

(Mobile Industry Processor Interface, MIPI), General Purpose I/O Ports (GPIO), Subscriber Identity Module (SIM) interface, and/or Universal Serial Bus (Universal Serial Bus, USB) interface, etc.

The charging management module 140 is used to receive charging input from the charger. Among them, the charger can be a wireless charger or a wired charger. In some wired charging examples, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some examples of wireless charging, the charging management module 140 may receive a wireless charging input through the wireless charging coil of the terminal 100. While the charging management module 140 charges the battery 142, it can also supply power to other electronic devices through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other examples, the power management module 141 may also be provided in the processor 110. In other examples, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the terminal 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem, and a baseband processor.

The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other examples, the antenna can be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the terminal 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves by at least two antennas including the antenna 1, and perform processing such as filtering, amplifying and transmitting the received electromagnetic waves to a modem for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem, and convert it into electromagnetic waves through the antenna 1 and radiate it out. In some examples, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110. In some examples, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.

The modem may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194. In some examples, the modem may be a stand-alone device. In other examples, the modem may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules. In other examples, the mobile communication module 150 may be a module in a modem.

The wireless communication module 160 can provide applications on the terminal 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation satellite systems. (Global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be sent from the processor 110, perform frequency modulation, amplify it, and convert it into electromagnetic waves to radiate through the antenna 2.

In some examples, the antenna 1 of the terminal 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), fifth generation, new air interface ( new radio, NR), BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), and the quasi-zenith satellite system (quasi). -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).

The terminal 100 implements a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and is used for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode). emitting diode (AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc. In some examples, the terminal 100 may include one or more display screens 194.

The terminal 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

The ISP is used to process the data fed back from the camera 193. For example, when shooting, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transfers the electrical signal to the ISP for processing and is converted into an image visible to the naked eye. ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some examples, the ISP may be provided in the camera 193.

The camera 193 is used to capture still images or videos, for example, to capture facial feature information and posture feature information of the user. The object passes through the lens to generate an optical image and project it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some examples, the terminal 100 may include one or more cameras 193.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the terminal 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The terminal 100 may support one or more video codecs. In this way, the terminal 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.

The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by running instructions stored in the internal memory 121. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function. The data storage area can store data (such as audio data, phone book, etc.) created during the use of the terminal 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

The terminal 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some examples, the audio module 170 may be provided in the processor 110, or part of the functional modules of the audio module 170 may be provided in the processor 110.

The speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. The terminal 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal 100 answers a call or voice message, it can receive the voice by bringing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through the human mouth, and input the sound signal into the microphone 170C. The terminal 100 may be provided with at least one microphone 170C. In other examples, the terminal 100 may be provided with two microphones 170C, which can implement noise reduction functions in addition to collecting sound signals. In other embodiments, the terminal 100 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions.

The earphone interface 170D is used to connect wired earphones. The earphone interface 170D may be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and an ambient light sensor 180L, bone conduction sensor 180M, etc.

Among them, the pressure sensor 180A is used to sense a pressure signal, and can convert the pressure signal into an electrical signal. In some examples, the pressure sensor 180A may be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors and so on. The capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The terminal 100 determines the strength of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal 100 detects the intensity of the touch operation according to the pressure sensor 180A. The terminal 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some examples, touch operations that act on the same touch location but have different touch operation strengths may correspond to different operation instructions. For example: when a touch operation whose intensity is less than the first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the motion posture of the terminal 100. In some examples, the angular velocity of the terminal 100 around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the terminal 100 is used to collect user characteristic information in the environment, the gyroscope sensor 180B detects the jitter angle of the terminal 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the terminal 100's loss through reverse motion. Shake, realize anti-shake.

The air pressure sensor 180C is used to measure air pressure. In some examples, the terminal 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal 100 in various directions (generally three-axis). When the terminal 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to recognize the posture of the terminal, and it can be used in applications such as horizontal and vertical screen switching, pedometer and so on.

Distance sensor 180F, used to measure distance. The terminal 100 can measure the distance by infrared or laser. In some examples, when using the terminal to collect user characteristic information of the user in the environment, the terminal 100 may use the distance sensor 180F to measure distance to achieve rapid focusing.

The ambient light sensor 180L is used to sense the brightness of the ambient light. The terminal 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light.

The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photographs, fingerprint answering calls, and so on.

The temperature sensor 180J is used to detect temperature. In some examples, the terminal 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the terminal 100 executes to reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal 100 heats the battery 142 to avoid abnormal shutdown of the terminal 100 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the terminal 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch device". The touch sensor 180K may be provided on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”. The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal 100, which is different from the position of the display screen 194.

The button 190 includes a power-on button, a volume button, an input keyboard, and so on. The button 190 may be a mechanical button. It can also be a touch button. The terminal 100 may receive key input, and generate key signal input related to user settings and function control of the terminal 100.

The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration notification, and can also be used for touch vibration feedback. For example, touch operations on different applications (such as video playback, audio playback, etc.) can correspond to different vibration feedback effects. Acting on touch operations in different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminding, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, or to indicate messages, missed calls, notifications, and so on.

The following describes a schematic diagram of the hardware structure of a control center in an embodiment of the present application.

Figure 3 shows a schematic diagram of the hardware structure of the control center. As shown in FIG. 3, the control center 12 may include a processor 210, a memory 220, a wireless communication module 230, a transceiver 240, a detection module 250, a wired communication module 260, a power switch 270, and so on.

The processor 210 can be used to read and execute computer readable instructions. In specific implementation, the processor 111 may mainly include a controller, an arithmetic unit, and a register. Among them, the controller is mainly responsible for instruction decoding, and sends out control signals for the operation corresponding to the instruction. The arithmetic unit is mainly responsible for saving the register operands and intermediate operation results temporarily stored during the execution of the instruction. In some examples, the processor 210 may be used to analyze the signals received by the wireless communication module 230 and/or the wired communication module 270, such as analyzing image data to obtain an image, and analyzing user characteristic information (such as facial features, posture, etc.) in the image. Features, etc.), and screen out target users based on user feature information, and then select the screen terminal corresponding to the monitoring device that monitors the target user as the target screen terminal. In the embodiment of the present application, the image data may include video data sent by the monitoring device. Since video is composed of multiple frames of images, video data can also be understood as image data.

The processor 230 may be configured to perform corresponding processing operations according to the monitoring result sent by the monitoring device, and/or transfer real-time running data of the target application on the user terminal. For example, the screen terminal corresponding to the monitoring device that monitors the target user is selected as the target screen terminal, and/or real-time running data of the target application on the user terminal 11 is transferred to the target screen terminal.

The memory 220 is coupled with the processor 210, and is used to store various data and/or multiple sets of instructions. For example, storing pre-collected user characteristic information, storing installed applications, and so on.

The wireless communication module 230 may include a Bluetooth module 231 and a Wi-Fi module 232. The Bluetooth module 231 and/or the Wi-Fi module 232 can perform data interaction with the user terminal 11 and the screen terminal 13, for example, receive a screen paging instruction from the user terminal 11, and receive video data from the screen terminal 13. In a specific implementation, the Bluetooth module 231 may include a classic Bluetooth (for example, Bluetooth 2.1) module and/or a Bluetooth low energy (bluetooth low energy, BLE) module. The Wi-Fi module 232 may include one or more of a Wi-Fi direct module, a Wi-Fi LAN module, or a Wi-Fi softAP module. In some examples, the Bluetooth module 231 and/or the Wi-Fi module 232 can transmit signals, such as broadcast Bluetooth signals, beacon signals, etc., so that other devices (such as the user terminal 11 and the screen terminal 13) can discover the control center 12 and perform A wireless communication connection between the control center 12 and the user terminal 11 and the screen terminal 13 can be established for data interaction. In some examples, the control center 12 may access the Internet through the Wi-Fi module 232, so as to establish a communication connection with a server on the Internet (for example, a server of a video playback website). In some examples, the wireless communication module 230 may further include an infrared communication module (not shown). The infrared communication module can communicate with the remote control and other devices through infrared remote control technology.

The transceiver 240 is used to perform data receiving and/or sending operations. For example, receiving data sent by the user terminal 11 and the monitoring device, and/or sending data to the user terminal 11 and the screen terminal 13.

The detection module 250 is configured to detect whether the application requested by the user terminal 11 is adapted to at least one screen terminal, and/or detect whether the hardware used to monitor the user on the monitoring device is malfunctioning, and so on. For example, the application requested by the user terminal 11 is WhatsApp. If the permission requested by the user terminal 11 is to transfer voice, the detection module 250 will automatically detect the hardware information of the screen terminal 13 registered in the control center 12 to determine the screen terminal Whether 13 has a voice call capability, where, when the screen terminal 13 has a voice call capability, it indicates that WhatsApp is compatible with at least one screen terminal. The detection module 250 can also continuously or intermittently send a self-inspection instruction to the monitoring device through the transceiver 240, and then the self-inspection data fed back by the root monitoring device can determine whether the hardware used to monitor the user on the monitoring device is malfunctioning, for example, the detection module 250 sends one or more sets of self-check instructions to the monitoring device. If the monitoring device can run the self-check instructions normally, the detection module 250 determines that the hardware on the monitoring device for monitoring users is not faulty based on the information fed back by the monitoring device.

The wired communication module 260 may include a USB module 261, so that the control center 12 can communicate with other devices (for example, the user terminal 11, the screen terminal 13, and the monitoring device, etc.) through a USB data cable. The wired communication module 260 may also include a high definition multimedia interface (HDMI) (not shown). The control center 12 may communicate with devices such as HDMI and a set-top box (STB).

The power switch 270 can be used to control the power supply to the control center 12.

In some examples, the user terminal 11 is mainly used to run the target application, and when the user terminal 11 determines that the target application is being used and the user terminal 11 is separated from the target user, it sends a screen paging instruction to the control center 12, where: The screen paging instruction is an instruction to find the screen terminal closest to the target user.

The target user refers to a user who was using the user terminal 11 before being separated from the user terminal 11, or a user who turned on the user terminal 11, or a user registered in the smart home system. For example, when the user terminal 11 is a mobile phone, the target user may be a user who was using the mobile phone before being separated from the mobile phone; when the user terminal 11 is a smart TV, the target user may be a user who turns on the smart TV; When the user registered in the home system is user A, then user A is the target user. The target application refers to the application that can be displayed on the screen terminal (such as screen terminal 13, 14) added in the control center 12 by the user using the user terminal 11 or other terminal, and can also be understood as an application registered in the smart home system . Among them, the target application can be a video-on-demand application, such as YouTube, Tencent Video, iQiyi Video, etc.; it can also be a social application, such as Facebook, Twitter, WhatsApp, WeChat, etc.; it can also be an email application, such as Google Mail (gmail), NetEase mailbox, Huawei mailbox, etc.; it can also be a video conference software application, such as ZOOM, Huawei Cloud WeLink, etc.; it can also be a reminder application.

The control center 12 is mainly used to control and coordinate related equipment and resources. When initializing, it reads the related configuration of the related equipment and automatically monitors the related hardware resources. For example, it can detect whether the registered online monitoring devices (such as cameras, etc.) are operating normally . In addition, it is also used to send a paging request to one or more screen terminals (such as screen terminals 13, 14) after receiving a screen paging instruction sent by the user terminal 11, and to send a paging request to the target user on the user terminal 11 The real-time running data of the target application is transferred to the screen terminal. The control center 12 can be understood as a central control system in a smart home system, through which the smart devices in the user's home are connected together. In addition, the control center 12 can also be understood as a server, whereby the server connects the user terminal 11 and the screen terminal together.

Screen terminals (such as screen terminals 13, 14) are mainly used to display the real-time running data of the target application on the user terminal 11 transferred from the control center 12, and to display the real-time running data, so that the target user can continue to use seamlessly Target application.

The monitoring device is mainly used to monitor the target user according to the paging request after receiving the paging request sent by the control center 12. For example, when the monitoring device monitors a target user, it can collect user characteristic information of the user in the environment, etc., and then determine whether the user is the target user based on the collected user characteristic information. Among them, the user feature information includes one or more of facial feature information, voice feature information, and posture feature information.

It should be understood that taking the monitoring device as an example, when monitoring the target user, the monitoring device can determine whether the target user is monitored, or the control center 12 can determine whether the monitoring device has monitored the target user based on the monitoring data sent by the monitoring device. . For example, if the monitoring device determines whether the target user is monitored, the monitoring device can identify the user based on the user characteristic information it collects. When the target user is identified, it is determined that the target user has been monitored; if the control center 12 According to the monitoring data sent by the monitoring device to determine whether the monitoring device has monitored the target user, the control center 12 can identify the target user based on the user characteristic information of the user collected by the monitoring device contained in the monitoring data, and then determine whether the monitoring device is monitoring To the target user. Among them, when recognizing the user, the recognition can be performed based on the user's facial feature information, human posture feature information, voiceprint feature information, and the like.

For example, when facial feature information is used for recognition, a monitoring device (such as a camera or camera, etc.) is used to collect an image or video stream containing a human face, and automatically detect and track the human face in the image. After that, the monitoring device can directly Or indirectly call the user's facial feature information pre-stored in the database 15, and compare and identify the facial feature information collected in real time with the pre-stored facial feature information. When the similarity between the two reaches a preset threshold, the monitoring device can be determined The target user is detected.

When the human body posture feature information is used for recognition, a monitoring device (such as Kinect, etc.) is used to capture the user’s body posture feature information. After that, the monitoring device can directly or indirectly call the user’s body posture feature information pre-stored in the database 15 and combine The real-time collected and pre-stored human body posture feature information is compared and identified. When the similarity between the two reaches a preset threshold, it can be determined that the monitoring device has monitored the target user.

When the voiceprint feature information is used for identification, the user's voice information is collected by a monitoring device (such as a microphone, etc.), and the voiceprint feature information is extracted from the collected voice information. After that, the monitoring device can directly or indirectly call the database 15 The user’s voiceprint feature information is pre-stored in, and the extracted and pre-stored voiceprint feature information is compared and identified. When the similarity between the two reaches a preset threshold, it can be determined that the monitoring device has detected the target user. .

In addition, when there are multiple identified target users, the final target user can also be selected according to the similarity of the identified different target users, for example, the user with the highest similarity is the final selected target user.

It should be understood that when the user terminal 11 and the screen terminal (such as the screen terminal 13, 14) are registered in the control center 12, the control center 12 will store the user terminal 11, the monitoring device and the screen terminal (such as the screen terminal 13, 14) The address information (as shown in Table 1) is convenient for the control center 12 to search for the screen terminal, and/or for the control center 12 to identify the source of the received information.

Table I

设备名称Equipment name	设备地址Device address	设备型号Device model

用户设备11User equipment 11	192.168.253.1192.168.253.1	AC3205 AC3205

屏幕终端13Screen terminal 13	192.168.253.3192.168.253.3	EZ210 EZ210

屏幕终端14Screen terminal 14	192.168.253.3192.168.253.3	EZ210EZ210
监测装置20Monitoring device 20	192.168.253.9192.168.253.9	M24M24

In order to better understand the technical solutions provided by the embodiments of the present application, an application scenario of the embodiments of the present application is introduced below. FIG. 4 shows a schematic diagram of an application scenario of the control method of the smart home system provided by the embodiment of the present application. As shown in FIG. 4A, in the first time period (e.g., 8 to 9 in the evening), user A (that is, the human figure mark in the figure) is located in the living room and uses the Whatsapp application on the user terminal 11 to make a voice call. As shown in FIG. 4B, in the second time period (e.g., 9 to 10 in the evening), the user A moves from the living room to the bedroom, and the user A leaves the user terminal 11 in the living room. Among them, a control center 12 is arranged in the living room, and

screen terminals

13 and 14 are arranged in the bedroom and the kitchen respectively, wherein at least one monitoring device (such as a camera) is integrated on the

screen terminals

13 and 14.

In the second time period, since the user A and the user terminal 11 are separated, it will be difficult for the user A to continue to use the Whatsapp application on the user terminal 11 to make a voice call. At present, people generally use intelligent control terminals in other locations (such as the screen terminal 13 in the bedroom) to restart the Whatsapp application and perform corresponding control. In this way, although user A can continue the voice call, the entire continuation process is cumbersome and costly. Therefore, it is impossible to achieve seamless control on different terminals, resulting in low intelligence of the smart home system and poor user experience.

In the embodiment of the present application, in the second time period when the user A and the user terminal 11 are separated, the user terminal 11 sends the Whatsapp screen paging instruction to the control center 12 in the smart home system; then the control center 12 searches according to the screen. Paging instructions, send paging requests to monitoring devices on multiple screen terminals (such as screen terminals 13, 14) in the family; after the monitoring device receives the paging request, the monitoring device collects user characteristic information of users in the environment to The user A is monitored; later, when the control center 12 or the monitoring device recognizes the user A according to the user characteristic information, the control center 12 transfers the data of the voice call using Whatsapp on the user terminal 11 to the monitoring device where the user A is monitored. On the corresponding target screen terminal (such as the screen terminal 13 in the bedroom), so that user A can seamlessly continue to use the Whatsapp application to make voice calls, get rid of the shackles of mobile phones and other terminals, so that user A can enjoy the smart home system. The convenience of coming has improved the degree of intelligence of the smart home system. Among them, the target screen terminal refers to the screen terminal corresponding to the monitoring device of the user A. Since the monitoring device on the screen terminal 14 in the kitchen has not collected the user characteristic information of the user A, the control center 12 will not transfer the data of the voice call to the screen terminal 14.

The following is based on the system architecture shown in FIG. 1, the hardware structure of the terminal shown in FIG. 2, the hardware structure of the control center shown in FIG. 3, and an application scenario shown in FIG. The technical scheme is explained in detail.

In the embodiment of the present application, the user can pre-designate a certain application (such as Youtube) as the target application, so as to facilitate the user's independent choice and improve the user experience. Figure 5 shows a schematic flow chart of adding a target application. As shown in Figure 5, adding a target application includes the following steps:

Step 501: The user terminal receives a registration request sent by the target user.

The target user can open the client or web page of the smart home system on the user terminal, and then independently choose to add the target user of the target application in the control center through his self-registered account. When the target user chooses to add the target application, the user terminal receives a registration request, where the registration request is used to request to display real-time running data of the target application on the target screen terminal.

For example, as shown in Figure 6A, the target user uses his user name and password to log in to the client of the smart home system on the user terminal; afterwards, the login is successful, as shown in Figure 6B, the target user can click the Add button, and then from Select Add WhatsApp in the submenu; after that, the user terminal receives the registration request of the target user.

In some examples, the target user may choose to display the relevant permissions of the target application on the screen terminal, so as to increase the user's independent choice and improve the user experience. For example, when the target application is WhatsApp, the target user can choose audio permissions and video permissions.

Step 502: The user terminal sends a registration request to the control center.

Step 503: The control center receives the registration request, and detects whether the target application is adapted to at least one screen terminal.

When a device (such as the screen terminal 13) is added to the smart home system, the device sends its own hardware information to the control center, and the control center stores it, so that the control center knows the hardware information of the device . After the control center receives the registration request, it matches the target application type and/or the relevant permissions selected by the target user in its stored information; when the matching is successful, it determines that the target application is adapted to at least one screen terminal. When matching, the target application is added to the registration list; when the matching fails, it is determined that the target application cannot be adapted to at least one screen terminal. At this time, a prompt message is sent to the user terminal so that the user terminal reminds the user who initiated the registration request , So that the target user can re-select.

For example, the target application is WhatsApp, and the permission selected by the target user is audio call; if the information stored in the control center includes terminal information with audio call function, it is determined that WhatsApp is adapted to at least one screen terminal.

In some examples, after the control center determines that the target application is adapted to at least one screen terminal, the control center can establish an association relationship between the target application and the monitoring device corresponding to the corresponding screen terminal, so that the control center can interact with the target application according to the target application direction. The monitoring device with the associated relationship of the target application accurately sends the paging request to improve the processing efficiency. Among them, the corresponding screen terminal refers to a screen terminal adapted to the hardware required by the target application.

Step 504: The control center feeds back prompt information of the success or failure of the addition to the user terminal.

Step 505: The user terminal sends a prompt to the target user according to the prompt information.

If the prompt information fed back by the control center is that the addition is successful, the user terminal will send a prompt that the addition is successful to the target user; if the prompt information fed back by the control center is that the addition has failed, the user terminal will prompt the target user that the addition has failed. The user terminal can display the text information of "adding successfully" or "adding failed" on its user interface; it can also display the graphic information of "adding successful" or "adding failed" on its user interface, as shown in Figure 7A, adding When successful, a smiling face graphic can be displayed, as shown in Figure 7B, when the addition fails, a crying face graphic can be displayed.

In some examples, if the control center detects that the target application that needs to be added is adapted to at least one screen terminal, the control center can use the network to download and install the target application after adding the target application to the registration list, so that the target application can be downloaded in the control center. Run the target application in. In addition, the screen terminal adapted to the target application can also use the network to download and install the target application, so that the screen terminal can run the target application by itself.

After the target user successfully adds the target application, the control center can transfer the real-time running data of the target application on the user terminal to the target screen terminal based on the target user's use of the user terminal, as described below for details.

Fig. 8 is a communication schematic diagram of a control method of a smart home system provided by an embodiment of the present application. As shown in Figure 8, the control method of the smart home system provided by the present application includes the following steps:

In step 801, the user terminal recognizes that the trigger condition of the screen paging is satisfied.

The target user or other users can log in to the client or web page of the smart home system on the user terminal, and then preset the screen paging trigger condition in the client or web page of the smart home system. For example, taking the setting on the client of the smart home system as an example, as shown in Figure 9A, the user clicks the virtual button of "Settings"; after that, as shown in Figure 9B, the user can select trigger from the sub-menu of settings Condition, and then select one or more trigger conditions from the trigger condition sub-menu, for example, select manual trigger and/or automatic trigger. When the trigger condition is reached, the user terminal can determine that the trigger condition for screen paging is currently met.

In some examples, in order to improve the convenience of manipulation, the trigger condition is manual triggering. At this time, the target user is provided with physical buttons and/or virtual buttons on the user terminal, and the target user can press the physical buttons and/or virtual buttons to send After the screen paging instruction is turned on, the user terminal can determine that the trigger condition of the screen paging is currently met. For example, as shown in Figure 10, the user interface of the user terminal is provided with a virtual button of "enable screen paging". When the target user uses the user terminal to make a voice call, if the target user presses the virtual button, the user terminal recognizes the current The trigger condition of the screen paging has been met.

It should be understood that when the user terminal can be controlled by other devices (such as a remote control and other terminals), the target user can also issue an instruction to start screen paging to the user terminal through other devices.

In some examples, in order to improve the intelligence of the smart home system, the trigger condition is automatic trigger. At this time, if the user terminal autonomously recognizes that it is in a separate state from the target user, the user terminal can recognize that the screen search is currently satisfied. The trigger condition of the call. Among them, the situation where the user terminal autonomously recognizes that it is separated from the target user includes:

First, the user terminal detects that the target user is not within the visual range of the user terminal. For example, as shown in Figure 11, the user terminal is a smart TV, and the shooting angle β of the camera 3 on the smart TV is 90 degrees. When the target user is in area 1, the camera 3 can collect the user characteristic information of the target user. The TV can detect that the target user is within its visible range; when the target user is in area 2, the camera 3 cannot collect the user characteristic information of the target user, and the smart TV can detect that the target user is not within its visible range . In addition, if the image acquisition device such as the camera on the user terminal does not collect contour information such as the face and head, the user terminal can also determine that the target user is not detected within its visual range, which can be determined according to the actual situation. Not limited.

In addition, due to the phenomenon that the target user moves back and forth between the visible range and the invisible range of the user terminal, in order to improve the accuracy of control, you can also set the user terminal to not detect the target user within its visible range. The duration, when the duration reaches the first preset duration, the user terminal can detect that it is in a separated state from the target user.

Second, the user terminal detects that the target application is in the activated state for a period of time that reaches the second preset period of time. For example, the user terminal is a mobile phone, and when the mobile phone receives a new communication request (such as a call request), if the new communication request time reaches the second preset duration (such as 30 seconds), the mobile phone can detect that the target user is not present. Nearby, the mobile phone can detect that it is separated from the target user.

It should be understood that when the user terminal autonomously determines whether it is separated from the target user, it can determine that the target application on it is in an active state, so as to improve the accuracy of control, avoid unnecessary screen paging, and save resources . For example, if the target application is Youtube, the user terminal will determine whether it is separated from the target user only when Youtube is activated (ie, turned on), and then determine whether it is necessary to send a screen paging instruction to the control center.

In some examples, in order to facilitate the control center to quickly identify the source of the message, in addition to the instruction operation code, the address code of the user terminal may also be added to the screen paging instruction.

In addition, the screen paging instructions include not only the operation code of the screen paging, but also the running status information of the target application, so that the control center can transfer the real-time running data of the target application to the screen terminal according to the running status information of the target application. . In the embodiment of the present application, the running status information of the target application includes real-time running data of the target application, and/or the identification and progress of the target application. Among them, when the target application is a video-on-demand application such as Youtube, the real-time running data can be understood as the video content the target user is watching; when the target application is a remote call application such as WhatsApp, the real-time running data can be understood as communicating with the target user The identity information of other users (such as account information), etc.; when the target application is a mailbox application such as Gmail, the real-time running data can be understood as the content of the target user’s mailbox, which can be determined according to the actual situation. limited.

In some examples, since the smart home system often needs to run in the target environment, in this embodiment of the present application, before the user terminal recognizes that the trigger condition of the screen paging is satisfied, the user terminal can also identify whether it is in the target environment. Among them, the target environment includes home environment, office environment, etc. For example, the user terminal can use the positioning device on it to perform positioning, and when the user terminal is located in the residential area where the user is located, it can be determined that the user terminal is in a home environment. In addition, in order to improve the accuracy of the judgment, the user terminal can also use the camera and other devices on it to scan and photograph the environment in which it is located; afterwards, the scanned and photographed environmental image information is analyzed and processed to identify whether it is in In the target environment.

Step 802: The user terminal sends a screen paging instruction to the control center.

In some examples, the purpose of screen paging is to find the screen terminal closest to the target user.

Step 803: The control center receives the screen paging instruction sent by the user terminal.

In step 804, the control center sends a paging request to at least one monitoring device according to the on-screen paging instruction.

After receiving the screen paging instruction sent by the user terminal, the control center will respond to the screen paging instruction and send a paging request to at least one monitoring device so that the at least one monitoring device monitors the target user, where the monitoring device and the screen terminal Has a mapping relationship. Among them, the control center can send a paging request by broadcasting.

It should be understood that, in addition to adding the request operation code to the paging request, the address code of the monitoring device can also be added, so that the control center can accurately send the paging request to the monitoring device.

Step 805: The monitoring device receives the paging request sent by the control center.

Step 806: The monitoring device monitors the target user according to the paging request.

After the monitoring device receives the paging request sent by the control center, it starts monitoring the target user. For example, the monitoring device collects user characteristic information of users within its coverage area.

It should be understood that if there are multiple monitoring devices, the multiple monitoring devices can perform user monitoring at the same time or in time sharing. When performing user monitoring in time sharing, the monitoring device can use the distance detection unit on it to determine whether an object is detected. When an object is detected, it starts monitoring, thereby avoiding unnecessary monitoring and reducing the energy consumption of the smart home system.

Step 807: The monitoring device sends the monitoring data to the control center.

In some examples, the monitoring data includes user characteristic information of the user collected by at least one monitoring device; or includes the monitoring result of at least one monitoring device, and the monitoring result is used to indicate whether the monitoring device has detected the target user. Among them, the monitoring result is obtained by the monitoring device based on the user characteristic information collected by it.

Step 808: The control center receives the monitoring data sent by the monitoring device.

Step 809: The control center selects a target screen terminal from at least one screen terminal according to the monitoring data.

In some examples, if the monitoring data includes user characteristic information of the user collected by the monitoring device, the control center may screen the target users from the users monitored by at least one monitoring device according to the received user characteristic information; afterwards, According to the screening results, the monitoring device that monitors the target user is used as the first monitoring device; then, according to the mapping relationship between the monitoring device and the screen terminal, the first monitoring device is used to select the target screen terminal from at least one screen terminal . Among them, the target screen terminal refers to the screen terminal corresponding to the first monitoring device, and the first monitoring device refers to the monitoring device that monitors the target user. For example, the control center can determine whether the user characteristic information is the user characteristic information of the target user according to the user characteristic information sent by the monitoring device A. If it is, it is determined that the monitoring device A has detected the target user. At this time, the monitoring device A can be As the first monitoring device; afterwards, the first monitoring device is used to retrieve the mapping relationship between the monitoring device and the screen terminal, and then the screen terminal corresponding to the first monitoring device is obtained, and the screen terminal is selected as the target screen terminal.

In some examples, if the monitoring data includes the monitoring result of the monitoring device, the control center can select the screen terminal corresponding to the monitoring device of the target user as the target screen terminal according to the monitoring result. For example, the control center receives the monitoring results sent by the monitoring device A and the monitoring device B respectively. If the monitoring result sent by the monitoring device A indicates that it has monitored the target user, and the monitoring result sent by the monitoring device B indicates that it has not monitored the target user, Then the control center can select the screen terminal corresponding to the monitoring device A as the target screen terminal.

In some examples, when there are multiple target users registered in the control center through the user terminal or other terminals, there may be multiple monitoring devices that monitor the target user, which makes the target screen terminal determined by the control center may be Multiple. At this time, the control center can obtain the priority of each monitoring device that detects the target user based on the pre-stored priority of the target user; then, select the screen terminal corresponding to the monitoring device with the highest priority as the target screen terminal to avoid The real-time operating data of the target application may be transferred by mistake. For example, as shown in Figure 12, the target users include user A and user B, where the priority of user A is higher than the priority of user B, and the screen terminals include a first screen terminal and a second screen terminal. Both are integrated with a monitoring device; if the monitoring device in the first screen terminal detects user A, and the monitoring device in the second screen terminal detects user B, both can be used as the target screen terminal; but due to the priority of user A It is higher than the priority of user B, so the priority of the monitoring device in the first screen terminal is higher than the priority of the monitoring device in the second screen terminal; therefore, at this time, the control center will determine the first screen terminal as the final The target screen terminal. Among them, the dotted line in the figure represents the visible range of the monitoring device in the screen terminal.

In some examples, if multiple monitoring devices simultaneously monitor the same target user, this allows the control center to determine multiple target screen terminals. At this time, the control center can obtain the monitoring device that is closest to the target user based on the distance between the target user and each monitoring device, and select the screen terminal corresponding to the monitoring device as the final target screen terminal, so that Manipulate and/or observe the target user. For example, as shown in Figure 13, the target user is user A, and the screen terminal includes a first screen terminal and a second screen terminal; if the first screen terminal and the second screen terminal detect user A at the same time, the control center can connect both Both are used as the target screen terminal, and the monitoring device is integrated in the two screen terminals; but since the distance between the user A and the monitoring device in the second screen terminal is the closest, at this time, in order to facilitate user A's manipulation and/or observation, The control center selects the second screen terminal as the final target screen terminal. Among them, the dotted line in the figure represents the visible range of the monitoring device.

It should be understood that the distance between the monitoring device and the target user can be determined by a distance detection unit on the monitoring device or other locations (such as a screen terminal, etc.), where the distance detection unit can be a radar range sensor or an ultrasonic sensor. Distance sensor, laser distance sensor, etc. In addition, the monitoring device can also analyze and process the collected video data or image data to determine the distance between it and the target user; or the monitoring device can send the collected information to other equipment (such as user terminals, control Center, screen terminal, etc.), other equipment determines the distance between the monitoring device and the target user based on the information collected by the monitoring device.

In step 810, the control center transfers the real-time running data of the target application to the target screen terminal for display based on the received running state information of the target application sent by the user terminal.

After determining the target screen terminal, the control center transfers the real-time running data of the target application to the target screen terminal based on the received operating status information of the target application sent by the user terminal. Wherein, the running status information of the target application includes real-time running data of the target application, and/or the identification and process of the target application.

In some examples, if the operating status information of the target application is carried in the screen paging indication, where the operating status information includes the identification and process of the target application, the control center can start its system based on the operating status information of the target application. The installed virtual/physical target application; afterwards, the application process of the target application in the control center is adjusted to be consistent with the application process in the running status information. After that, the control center sends the real-time running data of the target application on it to the target screen terminal for display, thereby realizing the transfer of the real-time running data of the target application on the user terminal to the target screen terminal.

In some examples, if the screen paging instruction does not carry the operating status information of the target application, the control center will, after determining the target screen terminal, feedback the information of the target screen terminal that it has selected to the user terminal; After receiving the feedback, the user terminal sends the running status information of the target application to the control center. After that, the control center transfers the real-time running data of the target application on the user terminal to the target screen terminal in the transfer method described above.

In some examples, if the operating status information of the target application includes real-time operating data of the target application, the control center can be used as a relay station, which directly runs in real time after receiving the operating status information containing the real-time operating data of the target application The data is forwarded to the target screen terminal for display on the target screen terminal. For example, when the target application is a video-on-demand application such as Youtube, the user terminal can send the video stream file on the target application to the control center, and then the control center forwards it to the target screen terminal, so that the target screen terminal can display the video stream Real-time content in the file.

Step 811: The target screen terminal receives the real-time running data of the target application transferred from the control center, and displays the real-time running data.

The target screen terminal receives the real-time running data of the target application transferred from the control center, and displays the real-time running data, so that the target user can continue to use the target application seamlessly, helping the target user to get rid of the shackles of mobile phones and other terminals, so that the target user can enjoy The convenience brought by the smart home system has improved the degree of intelligence of the smart home system.

It should be noted that, because the target user continues to move around, in order to make the target user move from the determined target screen terminal position to the position of another screen terminal, the target user can also continue to use the target application. In the embodiment of the present application, the screen paging instruction issued by the user terminal may continue to be valid until the user terminal determines that it is no longer separated from the target user. And/or, when the monitoring device corresponding to the determined target screen terminal cannot detect the target user, the monitoring device will feed back to the control center; after that, the control center will resend the paging request to make at least one monitoring device The device monitors the target user again. And/or, when the target user is monitored by other monitoring devices, the other monitoring devices will feed back to the control center. After that, the control center will use the screen terminal corresponding to the monitoring device as the target screen terminal and collect the real-time operating data of the target application Transfer to the newly determined target screen terminal.

In addition, in the embodiment of the present application, the user terminal and the control center can share the kernel, so that the control center can transfer the real-time running data of the target application on the user terminal to the target screen terminal.

In some examples, when the user terminal determines whether it meets the trigger conditions for screen paging, different applications have different operating conditions, and different operating conditions often correspond to different trigger conditions. Therefore, in the embodiments of the present application, , The user terminal can determine the trigger condition corresponding to the target application before determining that the trigger condition of the screen paging is satisfied, so as to improve the accuracy of control and the degree of intelligence of the smart home system. Wherein, FIG. 14 shows a schematic diagram of the steps of determining the trigger condition corresponding to the target application, as shown in FIG. 14, specifically including the following steps:

Step 1401: The user terminal obtains the type of the target application.

After the target user downloads and installs the target application on the user terminal, the user terminal can determine the type of the target application according to the program name of the target application obtained in the process of parsing the installation package of the target application; Types include video type, social type, call type, etc. For example, when the name of the target application is Tencent Video, the user terminal can determine that the type of the target application is the video type.

Further, after determining the type of the target application, the user terminal stores the type of the target application in order to obtain the type of the target application later; and establish the association relationship between the type of the target application and the trigger condition, and store it . For example, when the type of the target application is a video type, the trigger condition can be that the user terminal does not detect the target user in its visual range; when the type of the target application is a call type, the trigger condition can be that the user terminal determines the target The duration during which the application is active reaches the first preset duration.

Step 1402: The user terminal selects a trigger condition associated with the type according to the type.

After the user terminal obtains the type of the target application, it queries the established association relationship between the type of the target application and the trigger condition, and then the trigger condition associated with the type of the target application can be selected.

In some examples, when the control center selects target users from the users monitored by at least one of the users monitored by the monitoring device according to the received user characteristic information, the control center may compare the user characteristic information with the user characteristic information of the target user for filtering. .

As a possible implementation manner, FIG. 15 shows a schematic diagram of steps for the control center to screen target users based on user characteristic information, as shown in FIG. 15, specifically including the following steps:

Step 1501: Obtain pre-stored first characteristic information of the user.

The first characteristic information of the user is pre-stored in the control center. Therefore, the control center can directly retrieve the first characteristic information, where the first characteristic information includes the characteristic information of the target user.

Step 1502: Compare the user characteristic information with the first characteristic information.

The control center may compare the user characteristic information with the first characteristic information, where the type of the user characteristic information is the same as the type of the first characteristic information.

Step 1503, it is judged whether the comparison is successful.

The control center can compare the similarity between the user feature information and the first feature information; then, the control center can determine whether the comparison is successful based on the similarity obtained by the comparison, and if the comparison is successful, perform step 1504 , Otherwise, go to step 1505. For example, if the similarity is greater than the preset similarity threshold, it can be judged that the comparison is successful; otherwise, it can be judged that the comparison has failed.

In step 1504, the target user is screened.

Step 1505, the target user is not screened.

As another possible implementation manner, FIG. 16 shows another schematic diagram of the steps in which the control center screens target users based on user characteristic information. As shown in FIG. 16, it specifically includes the following steps:

Step 1601: Receive second characteristic information of the user sent by the user terminal.

The control center receives the second characteristic information of the user sent by the user terminal, and the second characteristic information includes the characteristic information of the target user. The second characteristic information is characteristic information of a user who uses the user terminal within a preset time collected by the user terminal. For example, the second characteristic information may be collected before sending the screen paging instruction. Optionally, the second feature information may be carried in the screen paging instruction, or may be sent by the user terminal in other time periods.

Step 1602: Compare the user characteristic information with the second characteristic information.

The control center may compare the user characteristic information with the second characteristic information; wherein the type of the second characteristic information is the same as the type of the user characteristic information.

In step 1603, it is judged whether the comparison is successful.

The control center can compare the similarity between the user characteristic information and the second characteristic information; after that, the control center can determine whether the comparison is successful based on the similarity obtained by the comparison, and if the comparison is successful, perform step 1604 , Otherwise, go to step 1605. For example, if the similarity is greater than the preset similarity threshold, it can be judged that the comparison is successful; otherwise, it can be judged that the comparison has failed.

Step 1604: Screen the target user.

Step 1605, the target user is not screened.

It should be noted that when the monitoring device obtains the monitoring result based on the user characteristic information collected by the monitoring device, it can refer to the processing procedures in steps 1501-1505 or 1601-1605, which will not be repeated here.

In addition, the ways in which the monitoring device obtains the characteristic information required for the comparison can be divided into the following situations, as detailed in the following description:

Case 1: When sending a paging request, the control center can send the pre-stored first characteristic information of the user and/or the received user terminal to the monitoring device when sending the second characteristic information, so that the monitoring device can At the same time when the paging request is received, the first characteristic information and/or the second characteristic information are acquired.

In the second case, the monitoring device may actively send a request to the control center to request the control center to send the first characteristic information and/or the second characteristic information to it.

Case 3: When the user terminal and the monitoring device can directly communicate, the monitoring device can actively send a request to the user terminal to request the user terminal to directly send the user's characteristic information (hereinafter referred to as the third characteristic information) to the monitoring device; Or, the user terminal automatically sends the third characteristic information directly to the monitoring device. It should be understood that both the second feature information and the third feature information can be understood as the same information, and both are collected by the user terminal before sending the screen paging instruction, and both include the user feature information of the target user. Only for ease of description, the second and third are used to distinguish.

It should be understood that the first characteristic information and/or the second characteristic information may be stored in the control center or in a database. When stored in the database, the control center can communicate with the database to call the first characteristic information and/or the second characteristic information.

In some examples, before transferring the real-time running data of the target application on the user terminal to the target screen terminal, the control center may also ask the target screen terminal whether to approve the transfer. Among them, FIG. 17 shows a communication schematic diagram of the control center inquiring the target screen terminal, as shown in FIG. 17, which specifically includes the following steps:

Step 1701: The control center sends a confirmation instruction to the target screen terminal.

Step 1702, the target screen terminal receives the confirmation instruction and displays the transfer prompt message.

When the target screen terminal receives the confirmation instruction, it displays the transfer prompt information on its screen for the target user to make a selection, wherein the transfer prompt information is used to prompt whether to transfer the real-time running data to the target screen terminal.

Step 1703: The target screen terminal receives the first confirmation information issued by the target user, where the first confirmation information is used to instruct to transfer the real-time running data to the target screen terminal.

In step 1704, the target screen terminal feeds back the first confirmation message to the control center.

Step 1705, the control center receives the first confirmation message.

In other words, the control center needs to send a confirmation instruction whether to transfer the real-time running data of the target application on the user terminal to the target screen terminal. After the target screen terminal receives the confirmation instruction, it displays the transfer prompt message to the target user, for example, as shown in Figure 18. As shown, the target screen can display information on whether to agree to the transfer on its display screen, or can display it by using voice information. Continuing to refer to Figure 18, when the target user uses the virtual or physical buttons on the target screen terminal (ie "Yes" or "No" in the figure) to confirm the execution of the transfer operation, the target screen terminal sends the first confirmation message back to Control center; or, when the target user issues a voice confirmation to perform the transfer operation, the target screen terminal will feed back the first confirmation information to the control center. After that, after the control center confirms that it receives the first confirmation message, it starts to transfer the real-time running data of the target application on the user terminal to the target screen terminal.

It should be understood that when the target user prohibits the transfer operation, the control center does not transfer the real-time running data of the target application on the user terminal.

For ease of understanding, the following examples illustrate the control method of the smart home system provided in the embodiments of the present application.

In the first case, as shown in Figure 19A, when the target application is a voice call application, and user A is using a mobile phone (ie, user terminal) in the first area to make a voice call, if it is between 9:10 and 9:10 In the meantime, user A leaves the first area and moves to the second area, and the mobile phone can determine that it is separated from user A. After that, the mobile phone sends a screen paging instruction to the control center located in the fourth area; then, the control center respectively sends paging requests to the monitoring devices integrated on the screen terminal located in the second area and the third area. After that, the monitoring device integrated on the screen terminal in the second area and the third area monitors user A; since user A is in the second area, the monitoring device integrated on the screen terminal in the second area will monitor user A , And the monitoring device integrated on the screen terminal in the third area will not monitor user A; then, the monitoring devices on the two screen terminals will feed back the monitoring results to the control center. After that, the control center sends a confirmation instruction whether to perform the transfer operation to the screen terminal in the second area. After user A confirms the transfer operation, the control center transfers the voice call on the mobile phone in the first area to the second area On the screen terminal in the middle, so that user A can continue to make a voice call in the second area.

Further, as shown in FIG. 19B, after 9:10, the user A moves from the second area to the third area. When the user A is in the third area, the monitoring device integrated on the screen terminal in the third area will monitor the user A, and the monitoring device integrated on the screen terminal in the second area will no longer monitor the user A. After the control center receives the feedback from the monitoring devices on the two screen terminals, the control center transfers the voice call on the mobile phone in the first area to the screen terminal in the third area, so that user A is in the third area Voice calls can also be continued in the middle.

In the second case, as shown in Figure 20, when the target application is a mailbox application and the tablet computer (that is, the user terminal) in the first area receives a new email, if the time of receiving the new email exceeds If it has not been checked for 2 minutes, the tablet can determine that it is separated from user A. After that, the tablet computer sends a screen paging instruction to the control center located in the fourth area; then, the control center respectively sends paging requests to the monitoring devices integrated on the screen terminal located in the second area and the third area. After that, the monitoring device integrated on the screen terminal in the second area and the third area monitors the target user; since user A is in the second area, the monitoring device integrated on the screen terminal in the second area will monitor user A , And the monitoring device integrated on the screen terminal in the third area will not monitor user A; then, the monitoring devices on the two screen terminals will feed back the monitoring results to the control center. After that, the control center sends a confirmation instruction whether to perform the transfer operation to the screen terminal in the second area. After user A confirms the transfer operation, the control center transfers the content of the new mail on the tablet computer in the first area to On the screen terminal in the second area, so that the user A can also view the content of the new mail in the second area.

It should be understood that, for other types of applications, reference may be made to the descriptions in the above two cases, which will not be repeated here.

In the smart home system control method provided in this application, when the user leaves the user terminal and transfers to another area, the control center can transfer the real-time running data of the target application on the user terminal to the screen terminal in the area where the user is located, so that the user can Continue to use the target application seamlessly, helping users get rid of the shackles of mobile phones and other terminals.

In the following, another method for controlling a smart home system provided by an embodiment of the present application is introduced.

FIG. 21 is a communication schematic diagram of yet another method for controlling a smart home system according to an embodiment of the present application. As shown in Figure 21, the control method of the smart home system provided by the present application includes the following steps:

Step 2101: The user terminal recognizes that the trigger condition of the screen paging is satisfied.

Step 2102: The user terminal sends a screen paging instruction to the control center.

Step 2103: The control center receives the screen paging instruction sent by the user terminal.

Step 2104: The control center sends a self-check instruction to at least one monitoring device.

The control center may send a self-check instruction to the at least one monitoring device in real time or intermittently, so that the at least one monitoring device performs self-check.

Step 2105: The monitoring device receives the self-check instruction, and detects whether the target hardware on it is faulty.

Step 2106: The monitoring device feeds back the self-check data to the control center.

Step 2107: The control center receives the self-check data, and from the self-check data, it is concluded that the target hardware on at least one monitoring device is faulty.

If the control center finds from the self-inspection data that the target hardware on a certain monitoring device is not working properly, the control center can determine that the target hardware on the monitoring device is malfunctioning. Among them, the target hardware is used to monitor target users.

Because there are monitoring devices that cannot normally monitor the target user, the target user may be just in the coverage area of the monitoring device where the target hardware is malfunctioning, which results in the monitoring device not being able to normally monitor the target user. Therefore, at this time, in order for the target user to seamlessly use the real-time running data of the target application on the user terminal, the control center sends a paging request to at least one screen terminal by broadcasting or other methods, where the paging request is used for Make the screen terminal display the transfer prompt information, which is used to prompt whether to transfer the real-time running data of the target application on the user terminal to the screen terminal, that is, step 2108 is performed.

Step 2108: The control center sends a paging request to at least one screen terminal according to the screen paging instruction.

Step 2109: The screen terminal receives the paging request and displays the transfer prompt message.

Step 2110: The screen terminal receives the transfer confirmation information issued by the user.

In some examples, if the screen terminal receives the transfer confirmation information issued by the user, the screen terminal can determine that the user appears within its coverage area. If the screen terminal does not receive the confirmation transfer information issued by the user, the screen terminal can determine that the user does not appear in its coverage area. At this time, the screen terminal can choose not to give feedback to the control center, which can be determined according to the situation. This is not limited.

Step 2111: The screen terminal feeds back the second confirmation information to the control center, where the second confirmation information is used to instruct the real-time operation data to be transferred to the screen terminal

Step 2112: The control center receives the second confirmation message, and selects the screen terminal that feeds back the second confirmation message as the target screen terminal.

After receiving the second confirmation information fed back by the screen terminal, the control center selects the screen terminal that feeds back the second confirmation information as the target screen terminal.

Step 2113: The control center transfers the real-time running data of the target application on the user terminal to the target screen terminal.

Step 2114: The target screen terminal receives the real-time running data of the target application on the user terminal transferred from the control center, and displays the real-time running data.

In the control method of the smart home system provided in the present application, when the user terminal determines that it is separated from the target user, it initiates the screen paging instruction of the target application; after that, the control center determines that the target hardware of at least one monitoring device is faulty, and controls The center sends to at least one screen terminal a paging request indicating whether to transfer the real-time running data of the target application on the user terminal; then, the control center transfers the real-time running data of the target application on the user terminal to the screen terminal that feedbacks and confirms the transfer information In this way, when the target hardware fails to monitor the target user on the monitoring device, the target user can continue to use the target application seamlessly, helping the target user to get rid of the shackles of mobile phones and other terminals, so that the target user can enjoy the smart home The convenience brought by the system has improved the degree of intelligence of the smart home system.

It should be noted that in the embodiment provided in this application, in the process of transferring real-time running data of the target application on the user terminal to the target screen terminal, in addition to transferring with the control center, the user terminal can also directly transfer the target application's data. The running status information is sent to the target screen terminal so that the target screen terminal displays real-time running data of the target application, where the running status information includes the real-time running data of the target application, and/or the identification and progress of the target application. At this time, after the control center selects the target screen terminal, it feeds back the information of the target screen terminal to the user terminal; after that, the user terminal receives the information of the target screen terminal and sends the running status information of the target application to the target screen terminal , To display the real-time running data of the target application on the target screen terminal.

Wherein, when the running status information of the target application includes the identification and process of the target application, the target screen terminal can start the virtual/physical target application installed in its system according to the running status information of the target application; then, the target screen terminal Then adjust the application process of the target application on it to be consistent with the application process in the running status information, and the real-time running data of the target application can be displayed on the target screen terminal, thus realizing the transfer of the real-time running data of the target application on the user terminal to Target screen terminal.

When the running status information of the target application includes real-time running data of the target application, the target screen terminal can directly display the real-time running data of the target application. For example, when the target application is a video-on-demand application such as Youtube, the user terminal can send the video stream file on the target application to the target screen terminal, so that the target screen terminal can display the real-time content in the video stream file.

The following introduces a control device for a smart home system provided by an embodiment of the present application.

Fig. 22 is a schematic structural diagram of a control device of a smart home system provided by an embodiment of the present application. As shown in FIG. 22, the control device of the smart home system provided by the present application can be used to implement the control method executed by the control center, the control method executed by the user terminal, and the control method executed by the monitoring device described in the above method embodiment And the control method executed by the screen terminal.

The control device includes at least one processor 2201, and the at least one processor 2201 can support the control device to implement the control method executed by the control center, the control method executed by the user terminal, and the control method executed by the monitoring device described in the embodiments of this application. The control method executed by the screen terminal.

The processor 2201 may be a general-purpose processor or a special-purpose processor. For example, the processor 2201 may include a central processing unit (CPU) and/or a baseband processor. Among them, the baseband processor can be used to process communication data (for example, to determine the target screen terminal), and the CPU can be used to implement corresponding control and processing functions, execute software programs, and process data of the software programs.

Further, the control device may also include a transceiver unit 2205 to implement signal input (reception) and output (send). For example, the transceiver unit 2205 may include a transceiver or a radio frequency chip. The transceiver unit 2205 may also include a communication interface.

Optionally, the control device may further include an antenna 2206, which may be used to support the transceiver unit 2205 to implement the transceiver function of the control device.

Optionally, the control device may include one or more memories 2202, on which a program (or an instruction or code) 2204 is stored, and the program 2204 may be executed by the processor 2201, so that the processor 2201 executes the above method in the embodiment Described method. Optionally, data may also be stored in the memory 2202. Optionally, the processor 2201 may also read data stored in the memory 2202 (for example, pre-stored first characteristic information). The data may be stored at the same storage address as the program 2204, and the data may also be stored with the program 2204. In a different storage address.

The processor 2201 and the memory 2202 may be provided separately or integrated together, for example, integrated on a single board or a system-on-chip (SOC).

For a detailed description of the operations performed by the control device in the above-mentioned various possible designs, reference may be made to the description in the embodiment of the control method of the smart home system provided in this application, and the details are not repeated here.

It should be understood that each step of the foregoing method embodiment may be completed by a logic circuit in the form of hardware or instructions in the form of software in the processor 2201. The processor 2201 can be a CPU, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices , For example, discrete gates, transistor logic devices, or discrete hardware components.

It is understandable that the processor in the embodiments of the present application may be a central processing unit (central processing unit, CPU), or other general-purpose processors, digital signal processors (digital signal processors, DSP), and application-specific integrated circuits. (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

The method steps in the embodiments of the present application can be implemented by hardware, and can also be implemented by a processor executing software instructions. Software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (programmable rom) , PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), register, hard disk, mobile hard disk, CD-ROM or well-known in the art Any other form of storage medium. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in the ASIC.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be sent from a website site, computer, server, or data center to another website site via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) , Computer, server or data center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

It is understandable that the various numerical numbers involved in the embodiments of the present application are only for easy distinction for description, and are not used to limit the scope of the embodiments of the present application.

Claims

A control method of a smart home system, characterized in that it is applied to a control center, and the method includes:

Receiving the screen paging instruction sent by the user terminal;

Sending a paging request to at least one monitoring device according to the screen paging instruction, so that the at least one monitoring device monitors a target user, and there is a mapping relationship between the monitoring device and the screen terminal;

Receiving monitoring data sent by at least one monitoring device, where the monitoring data is used to indicate whether the target user is monitored;

According to the monitoring data, a target screen terminal is selected from at least one of the screen terminals, where the target screen terminal refers to the screen terminal corresponding to the first monitoring device, and the first monitoring device refers to the monitoring of the target user ’S monitoring device;

Based on the received running status information of the target application sent by the user terminal, the real-time running data of the target application is transferred to the target screen terminal for display, wherein the running status information includes the real-time running status of the target application Run data, and/or the identification and process of the target application.
The method according to claim 1, wherein the monitoring data includes at least one user characteristic information of the user collected by the monitoring device;

The selecting a target screen terminal from at least one screen terminal according to the monitoring data includes:

Screening the target user from the users monitored by at least one of the monitoring devices according to the user characteristic information;

According to the screening result, use the monitoring device that has monitored the target user as the first monitoring device;

According to the mapping relationship between the monitoring device and the screen terminal, the first monitoring device is used to select the target screen terminal from the at least one screen terminal.
The method according to claim 2, wherein the screening of the target user from the users monitored by at least one of the monitoring devices according to the user characteristic information comprises:

Acquiring pre-stored first characteristic information of the user, where the first characteristic information includes the characteristic information of the target user;

The user characteristic information is compared with the first characteristic information, and if the comparison is successful, the target user is screened.
The method according to claim 2, wherein the screening of the target user from the users monitored by at least one of the monitoring devices according to the user characteristic information comprises:

Receive second characteristic information of the user sent by the user terminal, where the second characteristic information is collected by the user terminal before sending the screen paging instruction, and the second characteristic information includes the target user’s Feature information

The user characteristic information is compared with the second characteristic information, and if the comparison is successful, the target user is screened.
The method according to claim 1, wherein the monitoring data includes at least one monitoring result of the monitoring device, and the monitoring result is used to indicate whether the monitoring device has monitored the target user;

The selecting a target screen terminal from at least one screen terminal according to the monitoring data includes:

According to the monitoring result, the screen terminal corresponding to the first monitoring device is selected as the target screen terminal.
The method according to any one of claims 1 to 5, wherein there are multiple first monitoring devices and multiple target users, and the target screen terminal is selected from at least one screen terminal ,include:

According to the monitored priority of the target user, the priority of each of the first monitoring devices is obtained, and the screen terminal corresponding to the first monitoring device with the highest priority is selected as the target screen terminal.
The method according to any one of claims 1-5, wherein there are multiple first monitoring devices, and the target user is one, and the target screen terminal is selected from at least one screen terminal, include:

In the first monitoring device, the screen terminal corresponding to the monitoring device closest to the target user is selected as the target screen terminal.
The method according to any one of claims 1-5, wherein before transferring the real-time running data of the target application to the target screen terminal for display, further comprising:

Sending a confirmation instruction to the target screen terminal to display transfer prompt information on the target screen terminal, where the transfer prompt information is used to prompt whether to transfer the real-time operating data to the target screen terminal;

Receiving first confirmation information fed back by the target screen terminal in response to the transfer prompt information, where the first confirmation information is used to instruct to transfer the real-time operating data to the target screen terminal.
The method according to any one of claims 1-5, wherein before the receiving the screen paging instruction sent by the user terminal, the method further comprises:

Receiving a registration request sent by the user terminal, where the registration request is used to request to display real-time running data of the target application on the target screen terminal;

Detecting whether the target application is adapted to at least one of the screen terminals;

If the target application adapts to at least one of the screen terminals, adding the target application to the registration list;

If the target application fails to adapt to at least one of the screen terminals, sending prompt information to the user terminal, so that the user terminal reminds the user who initiated the registration request.
The method according to any one of claims 1-5, wherein the real-time running data of the target application is transferred to the received running state data of the target application sent by the user terminal. Before the target screen terminal displays, it also includes:

Feeding back the selected information of the target screen terminal to the user terminal, so that the user terminal sends the running status information of the target application;

Receiving the running state information of the target application sent by the user terminal.
The method according to any one of claims 1 to 5, wherein the screen paging instruction carries running status information of the target application.
The method according to any one of claims 1 to 5, wherein there are multiple monitoring devices, and multiple monitoring devices perform user monitoring at the same time or in time sharing.
A control method of a smart home system, characterized in that it is applied to a control center, and the method includes:

Receiving the screen paging instruction sent by the user terminal;

According to the screen paging instruction, a paging request is sent to at least one screen terminal, so that the screen terminal displays transfer prompt information, and the transfer prompt information is used to prompt whether to use the real-time running data of the target application on the user terminal Transfer to the screen terminal;

At least one second confirmation message fed back by the screen terminal is received, and the screen terminal that feeds back the second confirmation message is selected as the target screen terminal, and the second confirmation information is used to instruct to transfer the real-time operating data to The screen terminal;

Based on the received running status information of the target application sent by the user terminal, the real-time running data is transferred to the target screen terminal for display, wherein the running status information includes the real-time running of the target application Data, and/or the identification and progress of the target application.
The method according to claim 13, wherein before sending a paging request to at least one screen terminal according to the screen paging instruction, the method further comprises:

Send a self-check instruction to at least one of the monitoring devices, so that the monitoring device detects whether the target hardware on the monitoring device is faulty, the mapping relationship between the monitoring device and the screen terminal, the target hardware Used to monitor target users;

Receiving at least one self-inspection data fed back by the monitoring device;

According to the self-check data, it is concluded that the target hardware on at least one of the monitoring devices is faulty.
The method according to claim 13 or 14, characterized in that, before the receiving the screen paging instruction sent by the user terminal, the method further comprises:

Receiving a registration request sent by the user terminal, where the registration request is used to request to display real-time running data of the target application on the target screen terminal;

Detecting whether the target application is adapted to at least one of the screen terminals;

If the target application adapts to at least one of the screen terminals, adding the target application to the registration list;

If the target application fails to adapt to at least one of the screen terminals, sending prompt information to the user terminal, so that the user terminal reminds the user who initiated the registration request.
The method according to claim 13 or 14, wherein the real-time running data is transferred to the target screen terminal based on the received running status information of the target application sent by the user terminal. Before showing, it also includes:

Feeding back the selected information of the target screen terminal to the user terminal, so that the user terminal sends the running status information of the target application;

Receiving the running state information of the target application sent by the user terminal.
The method according to claim 13 or 14, wherein the screen paging instruction carries running status information of the target application.
A control method of a smart home system, characterized in that it is applied to a user terminal, and the method includes:

Recognizing that the trigger condition for screen paging is met, the triggering condition includes detecting that the target user is not within the visible range of the user terminal for a first preset duration, and detecting that the target application is activated for a second duration At least one of the preset durations;

Send a screen paging instruction to the control center, so that the control center sends a paging request to at least one monitoring device or screen terminal, and the mapping relationship between the monitoring device and the screen terminal.
The method according to claim 18, wherein the trigger condition includes detecting that the target user is not within the visible range of the user terminal for a period of time reaching a first preset period of time, and detecting that the target application is in an active state. The duration reaches the second preset duration;

Before the identification meets the trigger condition of the screen paging, it further includes:

Acquiring the type of the target application;

According to the type, a trigger condition associated with the type is selected.
The method according to claim 18, characterized in that, before the recognizing that the trigger condition of the screen paging is satisfied, the method further comprises:

Collect the second characteristic information of the user who uses the user terminal within a preset time, and send the second characteristic information to the control center, so that the control center obtains information from the control center according to the second characteristic information. The target user is selected from the monitoring data sent by the monitoring device.
The method according to claim 18, characterized in that, before the recognizing that the trigger condition of the screen paging is satisfied, the method further comprises:

Collect third characteristic information of users who use the user terminal within a preset time, and send the third characteristic information to the monitoring device, so that the monitoring device determines whether to monitor according to the third characteristic information To the target user.
22. The method according to any one of claims 18-21, wherein before the identification meets the trigger condition of the screen paging, the method further comprises:

It is recognized that the user terminal is in a target environment, and the target environment includes a home environment and an office environment.
The method according to any one of claims 18-21, wherein the method further comprises:

Receiving a registration request sent by the target user, where the registration request is used to request to display real-time running data of the target application on a target screen terminal;

The registration request is sent to the control center, so that the control center detects whether the target application is adapted to at least one screen terminal.
The method according to any one of claims 18-21, wherein the screen paging instruction carries running status information of the target application, so that the control center is based on the running status information, The real-time running data of the target application is transferred to the target screen terminal for display, wherein the running state information includes the real-time running data of the target application, and/or the identification and progress of the target application.
The method according to any one of claims 18-21, wherein after the sending a screen paging instruction to the control center, the method further comprises:

Receive the information of the target screen terminal that is fed back by the control center, and send the running status information of the target application to the control center, so that the control center can update the target application's running status information based on the running status information. The real-time running data is transferred to the target screen terminal for display, wherein the running status information includes the real-time running data of the target application, and/or the identification and progress of the target application.
The method according to any one of claims 18-21, wherein after the sending a screen paging instruction to the control center, the method further comprises:

Receiving information from the control center that the target screen terminal is determined, and sending the running status information of the target application to the target screen terminal, so that the target screen terminal displays real-time running data of the target application, Wherein, the running status information includes real-time running data of the target application, and/or the identification and progress of the target application.
A control method of a smart home system, characterized in that it is applied to a monitoring device, and the mapping relationship between the monitoring device and a screen terminal, the method includes:

Receiving a paging request sent by the control center, where the paging request is sent after the control center receives a screen paging instruction sent by a user terminal;

According to the paging request, the target user is monitored, and the monitoring data is sent to the control center, so that the control center selects a target screen terminal from at least one of the screen terminals according to the monitoring data, and sends the monitoring data to the target screen terminal. The real-time running data of the target application on the user terminal is transferred to the target screen terminal, and the monitoring data is used to indicate whether the target user is monitored.
The method according to claim 27, wherein the monitoring data includes a monitoring result, and the monitoring result is used to indicate whether the monitoring device has monitored the target user;

The monitored target users include:

Collect user characteristic information of users within the monitoring range of the monitoring device, and obtain the monitoring result according to the user characteristic information.
The method according to claim 28, wherein said obtaining said monitoring result according to said user characteristic information comprises:

Acquiring pre-stored first characteristic information of the user, where the first characteristic information includes the characteristic information of the target user;

The user characteristic information is compared with the first characteristic information, and if the comparison is successful, the monitoring result is that the target user is detected.
The method according to claim 28, wherein said obtaining said monitoring result according to said user characteristic information comprises:

Receive second characteristic information of the user sent by the control center, where the second characteristic information is sent by the user terminal to the control center, and the second characteristic information is that the user terminal sends a screen page to the control center Collect before the instruction, and the second characteristic information includes the characteristic information of the target user;

The user characteristic information is compared with the second characteristic information, and if the comparison is successful, the monitoring result is that the target user is detected.
The method according to claim 28, wherein said obtaining said monitoring result according to said user characteristic information comprises:

Receive third characteristic information of the user sent by the user terminal, where the third characteristic information is collected before the user terminal sends a screen paging instruction to the control center, and the third characteristic information includes the target user’s Feature information

The user characteristic information is compared with the third characteristic information, and if the comparison is successful, the monitoring result is that the target user is detected.
A control method of a smart home system, characterized in that it is applied to a screen terminal, and the method includes:

Receive a paging request sent by the control center, and display transfer prompt information, the paging request is sent by the control center after receiving the screen paging instruction sent by the user terminal, and the transfer prompt information is used to prompt whether to transfer the Real-time operation data is transferred to the screen terminal;

Receiving the transfer confirmation information issued by the user, and feeding back second confirmation information to the control center, the second confirmation information being used to instruct the transfer of the real-time operating data to the screen terminal;

Receive the real-time running data of the target application on the user terminal transferred from the control center, and display the real-time running data.
A control device for a smart home system, comprising at least one processor configured to execute instructions stored in a memory, so that the control center executes the method according to any one of claims 1-12, or Perform the method according to any one of claims 13-17.
A control device for a smart home system, comprising at least one processor, the processor is used to execute instructions stored in a memory, so that a user terminal executes the method according to any one of claims 18-26.
A control device for a smart home system, which is characterized by comprising at least one processor configured to execute instructions stored in a memory, so that the monitoring device executes the method according to any one of claims 27-31.
A control device for a smart home system, comprising at least one processor, the processor is used to execute instructions stored in the memory, so that the screen terminal executes the method according to claim 32.
A control center, characterized in that it is used to execute the method according to any one of claims 1-12, or execute the method according to any one of claims 13-17.
A user terminal, characterized in that it is used to execute the method according to any one of claims 18-26.
A monitoring device, characterized in that it is used to execute the method according to any one of claims 27-31.
A screen terminal, characterized by being used to execute the method according to claim 32.
A smart home system includes a control center, a user terminal, a monitoring device, and a screen terminal, wherein there is a mapping relationship between the monitoring device and the screen terminal, and the control center is used to execute any one of claims 1-12 According to the method, the user terminal is used to execute the method according to any one of claims 18-26, and the monitoring device is used to execute the method according to any one of claims 27-31.
A smart home system includes a control center, a user terminal, and a screen terminal, wherein the control center is used to execute the method according to any one of claims 13-17, and the user terminal is used to execute any one of claims 18-26. One of the methods described, the screen terminal is used to execute the method according to any one of claim 32.
A computer storage medium in which instructions are stored, and when the instructions are run on a computer, the computer executes the method according to any one of claims 1-32.
A computer program product containing instructions that, when the instructions run on a computer, causes the computer to execute the method according to any one of claims 1-32.