WO2024108905A1 - Serveur, dispositif intelligent et procédé de commande de dispositif intelligent - Google Patents

Serveur, dispositif intelligent et procédé de commande de dispositif intelligent Download PDF

Info

Publication number
WO2024108905A1
WO2024108905A1 PCT/CN2023/091857 CN2023091857W WO2024108905A1 WO 2024108905 A1 WO2024108905 A1 WO 2024108905A1 CN 2023091857 W CN2023091857 W CN 2023091857W WO 2024108905 A1 WO2024108905 A1 WO 2024108905A1
Authority
WO
WIPO (PCT)
Prior art keywords
smart
server
area
master
capability
Prior art date
Application number
PCT/CN2023/091857
Other languages
English (en)
Chinese (zh)
Other versions
WO2024108905A9 (fr
Inventor
李含珍
吴超
王建君
Original Assignee
海信视像科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司 filed Critical 海信视像科技股份有限公司
Publication of WO2024108905A1 publication Critical patent/WO2024108905A1/fr
Publication of WO2024108905A9 publication Critical patent/WO2024108905A9/fr

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • the present disclosure relates to the technical field of display devices, and in particular to a server, an intelligent device, and an intelligent device control method.
  • the smart home system is a multi-device network system based on the Internet of Things technology. It can realize the coordinated operation of multiple devices through network communication to meet the needs of users.
  • the smart home system can be composed of smart terminal devices such as smart TVs, smart refrigerators, smart terminals, and magic cube screens. Users can control the smart TV to play specific media by performing interactive actions on the smart terminal devices, and can also control the refrigeration temperature of the smart refrigerator by performing interactive actions on the magic cube screen, realizing the coordinated operation of multiple devices.
  • Smart home systems can support intelligent voice control interaction, which can control the operation of devices by semantically recognizing the voice commands input by users and then controlling the operation of devices according to the semantic recognition results.
  • voice commands input by users may be vague, for example, the user's voice is "turn on the TV".
  • the smart home system contains multiple TVs, the smart home system does not know which TV the user wants to turn on, so it may control all TVs to turn on, thus affecting the user experience.
  • the current solution is to introduce all devices of the same type to the user and then ask the user which device he wants to control.
  • some embodiments of the present disclosure provide a server, including a communication module and a control module.
  • the communication module is configured to establish a communication connection with a smart device; the control module is configured to:
  • the regional device information includes device information of a plurality of first regions and smart devices in the first regions;
  • the master control device In response to a command response request sent by the master control device after detecting a user's voice command, obtaining a target master control device and a target first area corresponding to the target master control device; the command response request includes the voice command, and the voice command is used to control the smart device;
  • a smart device to be controlled is determined based on the target first area and the target main control device, and a response instruction is sent to the smart device to be controlled.
  • some embodiments of the present disclosure provide a smart device, including a display, a communicator, and a controller.
  • the communicator is configured to establish a communication connection with a server and other smart devices; the controller is configured to:
  • the smart device In response to a user's power-on instruction, the smart device is started and a startup message is sent to the server;
  • Receiving device information sent by the server calculating a capability score based on the device information and sending the score to the server, so that the server acquires the master device according to the capability score;
  • a command response request is generated and sent to a server, so that the server obtains a target main control device and a target first area corresponding to the target main control device, and obtains a smart device to be controlled and sends a response command to the smart device to be controlled, wherein the command response request includes the voice command.
  • some embodiments of the present disclosure provide a smart device control method, which is applied to a smart home system, wherein the smart home system includes a server and a plurality of smart devices, and the smart devices establish a communication connection with the server; the smart device control method includes:
  • the server obtains regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions;
  • the server acquires a master control device of the first area based on the device information, where each first area corresponds to a master control device;
  • the smart device After detecting the user's voice command, the smart device as the master device sends a command response request to the server, wherein the command response request includes the voice command, and the voice command is used to control the smart device;
  • the server obtains a target master control device and a target first area corresponding to the target master control device;
  • the server determines a smart device to be controlled based on the target first area and the target main control device, and sends a response instruction to the smart device to be controlled;
  • the intelligent device to be controlled executes the response instruction.
  • FIG1 is a usage scenario of a smart home system according to some embodiments.
  • FIG2 is a block diagram of a hardware configuration of a smart device according to some embodiments.
  • FIG3 is a schematic diagram of an Internet of Things system according to some embodiments.
  • FIG4 is a schematic diagram of multiple smart devices responding to voice interaction effects according to some embodiments.
  • FIG5 is a schematic diagram of a user interface according to some embodiments.
  • FIG6 is a schematic diagram of an application panel according to some embodiments.
  • FIG7 is a schematic diagram of a device control interface according to some embodiments.
  • FIG8 is a schematic diagram of a functional interface according to some embodiments.
  • FIG9 is a schematic diagram of a control area according to some embodiments.
  • FIG10 is a schematic diagram of a relay device list according to some embodiments.
  • FIG11 is a schematic diagram showing intelligent control mode confirmation information in a display according to some embodiments.
  • FIG12 is a flowchart of the interaction between various parts of a smart home system according to some embodiments.
  • FIG13 is a schematic diagram of a master device confirmation interface according to some embodiments.
  • FIG14 is a schematic diagram of a master control device switching interface according to some embodiments.
  • FIG. 15 is a schematic diagram of a rotation prompt interface according to some embodiments.
  • the smart home system described in the embodiments of the present disclosure is a network system based on a specific area network and established based on a unified control service.
  • the smart home system may include multiple smart devices 200 that establish communication connections with each other. Multiple smart devices 200 can achieve communication connections between devices by accessing the same local area network. Multiple smart devices 200 can also directly form a point-to-point network through a unified communication protocol to achieve communication connection. For example, multiple smart devices 200 can be connected to the same wireless local area network so that multiple smart devices 200 can communicate with each other. For example, a smart device 200 can also establish communication connections with multiple other smart devices 200 through Bluetooth, infrared, cellular network, power carrier communication, etc.
  • the smart device 200 refers to a device with communication function, capable of receiving, sending, executing control instructions and realizing specific functions.
  • the smart device 200 includes but is not limited to smart display devices, smart terminals, smart home appliances, smart gateways, smart lighting devices, smart audio devices, game devices, etc.
  • the multiple smart devices 200 constituting the smart home system can be devices of the same type or different types.
  • a smart TV for example, as shown in FIG1 , in the same smart home system, there may be included a smart TV, a smart speaker, a smart refrigerator, a plurality of smart lamps, etc.
  • These smart devices 200 may be distributed in different locations to meet the usage requirements at the corresponding locations.
  • the smart home system described in the present disclosure does not limit the scope of application of the scheme to be protected by the present disclosure. That is, in practical applications, the server, smart device and multi-device voice wake-up method provided by the present disclosure are not limited to the application in the field of smart home, but are also applicable to other systems supporting intelligent voice control, such as smart office systems, smart service systems, smart management systems, industrial production systems, etc.
  • the user can operate each smart device 200 through the smart terminal 300 or the control device 100.
  • the control device 100 can be a remote controller, and the communication between the remote controller and the smart device 200 includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods. At least one of the above methods controls the smart device 200 wirelessly or wiredly.
  • the user can input user commands through at least one method such as buttons on the remote controller, voice input, and control panel input to control the smart device 200.
  • the smart terminal 300 can realize the same functions as the control device 100, and support further extended functions and multi-device control based on the corresponding control functions of the control device 100.
  • the smart terminal 300 can be a control device such as a mobile phone, a tablet computer, or a computer.
  • the "universal remote control" application can be installed in the smart terminal 300. The user starts the application and selects the model of the smart device 200 or pairs it to simulate the function of the control device 100 that is matched with the smart device 200.
  • the smart device 200 has a specific hardware configuration according to the actual function of the smart device 200. As shown in FIG2 , taking a display device as an example, the smart device 200 with a display function may include at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.
  • the controller 250 includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input/output.
  • the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.
  • the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.
  • the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.
  • the external device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input/output interface formed by the above multiple interfaces.
  • HDMI high-definition multimedia interface
  • component analog or digital high-definition component input interface
  • CVBS composite video input interface
  • USB USB input interface
  • RGB port an RGB port
  • the controller 250 controls the operation of the smart device and responds to user operations through various software control programs stored in the memory.
  • the controller 250 controls the overall operation of the smart device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.
  • the controller 250 includes a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input/output, a communication bus (Bus), etc.
  • CPU central processing unit
  • video processor video processor
  • audio processor audio processor
  • graphics processing unit GPU
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • ROM Read-Only Memory
  • the user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user may input a user command through a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.
  • the smart device 200 also communicates data with the server 400.
  • the smart device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.
  • the server 400 may provide various content and interactions to the smart device 200.
  • the server 400 may be a cluster or multiple clusters, and may include one or more types of server groups.
  • the smart devices 200 and the server 400 constituting the smart home system can communicate with each other to realize comprehensive control of the smart home system. That is, the smart device 200 can report data to the server 400 in the application, so that the server 400 can analyze, judge, and process the data reported by the smart device 200 to generate control instructions for the smart device 200. The server 400 then sends the generated control instructions to the smart device 200 to realize the control of the operating status of the smart device 200.
  • Smart devices 200 in the smart home system can also establish communication connections with each other.
  • Smart devices 200 can be IOT (Internet of Things) devices, and multiple smart devices 200 together constitute an IOT system.
  • FIG3 is a schematic diagram of an IOT system according to some embodiments.
  • the IOT system generates and collects data from different dimensions through the IOT to achieve collaborative control of multiple smart devices 200.
  • Multiple smart devices 200 can be connected to the same wireless local area network to enable multiple A communication connection is established between the smart devices 200, and any device can send shared data to the IoT system, and any device can obtain data shared by other devices through the IoT system to achieve collaborative operation in the smart home system.
  • the smart device 200 can also be an AIoT (Artificial Intelligence & Internet of Things) device, and multiple smart devices 200 can form an AIoT system.
  • AIoT Artificial Intelligence & Internet of Things
  • the smart device 200 may have an intelligent voice control function so that the user can control the smart device 200 by voice.
  • the smart device 200 may have a built-in intelligent voice system to support the user's intelligent voice control.
  • the intelligent voice control refers to the interactive process in which the user operates the smart device 200 by inputting voice audio data.
  • the smart device 200 may include an audio input device and an audio output device.
  • the audio input device is used to collect the voice audio data input by the user, and may be a built-in or external microphone device of the smart device 200.
  • the audio output device is used to make a sound to play a voice response. For example, as shown in Figure 4, when the user inputs a wake-up word such as "Hi! Xiao ⁇ " through the audio input device, the smart device 200 can play a voice response of "I'm here" through the audio output device to guide the user to complete the subsequent voice input.
  • the intelligent voice system built into the smart device 200 also supports a one-shot mode, that is, a "one-shot” mode.
  • a one-shot mode the user can directly implement the control function through fewer voice inputs.
  • the user wants to control the smart device 200 to play movie resources, he needs to first input the voice "Hi, Xiao ⁇ ", and after the smart device 200 responds "I'm here", he needs to input "I want to watch a movie", and then the smart device 200 will respond "The following movies were found for you”.
  • the “one-shot” mode the user can directly input "Hi! Xiao ⁇ , I want to watch a movie", and the smart device 200 will directly respond "The following movies were found for you” after receiving the voice command, reducing the number of voice interactions and improving the efficiency of voice interactions.
  • the smart device 200 may also support some other functions.
  • the smart device 200 may have the ability to control other smart devices, thereby controlling other smart devices to implement some functions.
  • users can control the linkage of multiple devices through intelligent voice. For example, a user can input a voice command "turn on the bedroom light” through a smart speaker, and the smart speaker can respond to the voice command, generate a control command for turning on the light, and then send the control command to the lamp named "bedroom” in the smart home system to control turning on the bedroom light.
  • the smart speaker also responds to the user's voice input and plays feedback voice content such as "The bedroom light has been turned on for you" to reply to the user.
  • the control command can be directly transmitted to the controlled device through the smart device 200 that receives the user's voice audio data, or it can be transmitted to a specific relay device such as a router through the smart device 200, and then transmitted to the controlled device by the relay device.
  • the control command can also be transmitted to the controlled device through the server 400.
  • the smart terminal 300 can first send the control command to the server 400, and the server 400 then transmits the control command to the smart device 200 for control.
  • the user can control other smart devices using the smart device 200.
  • the control process is introduced by taking a display device as an example.
  • FIG. 5 is a schematic diagram of a user interface according to some embodiments.
  • the user interface includes a first navigation bar 500, a second navigation bar 510, a function bar 520, and a content display area 530
  • the function bar 520 includes multiple function controls such as "watch history", "my applications", and "my favorites".
  • the content displayed in the content display area 530 will change with the changes in the selected controls in the first navigation bar 500 and the second navigation bar 510.
  • the user can click on the "my application" control in the function bar 520 to input a display instruction for the application panel page to trigger entry into the corresponding application panel.
  • the user can also input a selection operation for the function control in other ways to trigger entry into the application panel. For example, control entry into the application panel page using a voice control function or a search function.
  • the user can view the applications installed on the display device through the application panel, that is, the functions supported by the display device.
  • the user can select the "My Applications” control to trigger the display device to display the application panel.
  • the user can further select to open one of the applications.
  • the application installed on the display device can be a system application or a third-party application.
  • Figure 6 is a schematic diagram of an application panel according to some embodiments. As shown in Figure 6, the application panel includes several controls such as "Internet Media”, “Cable TV”, “Device Control”, and “Video Chat”. Among them, the user can control the display device to watch online media by clicking the "Internet Media” control.
  • the user can click the "Cable TV” control to use the display device to watch some media channels, including various media programs provided by cable TV providers.
  • the user can click the "Video Chat” control to use the display device for video chat.
  • the user can click the "Device Control” control to use the display device to control other smart devices.
  • the display device can obtain relevant information of other smart devices that can currently be controlled, thereby generating a device control interface and providing it to the user.
  • the display device can obtain the device information of other smart devices in the smart home system, including the device names of other smart devices and supported functions.
  • the display device can send a request to the server to obtain the device information of other smart devices.
  • the device information of the smart device can be logged in to the server, so the server can query the device information of each smart device and send it to the display device.
  • the display device can count other smart device information and generate a device control interface.
  • Figure 7 is a schematic diagram of a device control interface according to some embodiments.
  • the device control interface includes controls for other smart devices, each control represents a smart device, and the device name can be displayed in the control, including the TV in the living room on the second floor, the refrigerator in the kitchen on the first floor, the speaker in the second bedroom, and the TV in the living room on the first floor.
  • the user can customize the device name of the smart device to facilitate subsequent distinction. The user can select the smart device to be controlled according to the device name in the device control interface.
  • FIG8 is a schematic diagram of a function interface according to some embodiments.
  • the display device when the user chooses to control the TV in the living room on the second floor, the display device can display the function interface of the TV in the living room on the second floor, including the functions supported by the TV in the living room on the second floor, including playing cable TV, playing network media, playing music, turning on/off the TV, etc.
  • the display device can send an instruction to the TV in the living room on the second floor. After receiving the control command, the TV in the living room on the second floor can execute the control command to realize the corresponding function.
  • the smart device 200 also has a relay capability, which refers to the ability to continue to display the playback screen of other smart devices. For example, when smart device A is playing a certain media resource, smart device B has a relay capability, then the smart device can take over the smart device A to continue playing the media resource, and the playback progress of the media resource can be kept consistent at the time of relay.
  • a relay capability refers to the ability to continue to display the playback screen of other smart devices. For example, when smart device A is playing a certain media resource, smart device B has a relay capability, then the smart device can take over the smart device A to continue playing the media resource, and the playback progress of the media resource can be kept consistent at the time of relay.
  • FIG. 9 is a schematic diagram of a control area according to some embodiments.
  • the control area includes but is not limited to at least one of a "relay” control, a "playlist” control, a "clarity” control, and a "recommended function” control.
  • the user can trigger the smart device 200 to perform a corresponding operation by operating a certain control. It should be noted that the user can realize the relay function by operating the "relay” control, or can control the smart device 200 to realize the relay function by other means such as using the voice control function.
  • the relay device list can be displayed on the display, and the relay device list includes all relayable smart devices.
  • Figure 10 is a schematic diagram of a relay device list according to some embodiments. As shown in Figure 10, the relay device list includes several smart devices in an embodiment of the present disclosure. The user can select a smart device as a relay device. For example, when the user selects "Second Floor Living Room TV", the second floor living room TV can realize the relay function, thereby continuing to display media resources from the current device.
  • the user can trigger the relay function introduction page by operating the "?" control, and the relay function introduction page is used to at least display the introduction information of the relay function, such as the relay operation method.
  • the relay device list also includes a switch button of "continue playing on the original device” to cancel the relay operation.
  • the instructions issued may be vague, and it is not possible to confirm which smart device the user needs to control.
  • the user's voice is "turn on the TV".
  • the user's voice is "turn on the TV".
  • the user can specify the TV that needs to be turned on, so as to control it accurately.
  • the smart home system has a smart control function.
  • each smart device can have a smart control function.
  • the smart home system can automatically confirm the specific smart device to be controlled instead of repeatedly asking the user, so as to improve the user's experience.
  • the smart device is set with a smart control mode, and all smart devices in the smart control mode can realize the smart control function together.
  • the user can send a smart control instruction to the smart device to make the smart device enter the smart control mode.
  • the user can send a smart control instruction to the smart device 200 by operating a designated button of the remote controller.
  • the user can also send a smart control instruction to the smart device 200 by voice input to control the smart device 200 to perform the smart control mode process.
  • the user can also send a smart control instruction to the smart device 200 through a preset gesture.
  • the smart device 200 can detect the user's behavior through an image collector, such as a camera. When the user makes a preset gesture or action, it can be considered that the user has sent a smart control instruction to the smart device 200.
  • a smart control instruction can also be sent to the smart device 200.
  • a control can be set in the mobile phone, and the control can be used to select whether to enter the smart control mode, thereby sending the smart control instruction to the controller 250.
  • a smart control mode option may be set in the UI interface of the smart device 200 , and when the user clicks on the option, the smart device 200 may be controlled to enter the smart control mode.
  • Fig. 11 is a schematic diagram showing smart control mode confirmation information in the display 260 according to some embodiments.
  • the server when the smart device 200 enters the smart control mode, the server can obtain the smart device to be controlled according to the user's voice command, so that the smart device responds to the user's command.
  • FIG12 is a flowchart of the interaction between the various parts of the smart home system according to some embodiments. It includes a user 121, a server 122, a main control device 123 and a smart device to be controlled 124, wherein:
  • Step 1201 The server 122 obtains the user's regional device information
  • the regional device information includes device information of a plurality of first regions and smart devices in the first regions;
  • Step 1202 The server 122 obtains the master control device of the first area based on the device information
  • Each of the first areas corresponds to a master control device
  • Step 1203 User 121 sends a voice command to the main control device 123;
  • Step 1204 the main control device 123 sends an instruction response request to the server 122;
  • the smart device After detecting the user's voice command, the smart device as the master device sends a command response request to the server, wherein the command response request includes the voice command, and the voice command is used to control the smart device;
  • Step 1205 The server 122 obtains a target master control device and a target first area
  • Step 1206 The server 122 determines the smart device to be controlled based on the target first area and the target main control device;
  • Step 1207 the server 122 sends a response instruction to the smart device 124 to be controlled;
  • Step 1208 The smart device 124 to be controlled executes the response instruction.
  • the server may first count the status of all smart devices in the user's home, which may be to obtain device information of the smart devices in the smart home system.
  • each room can be determined as a zone.
  • the zone corresponding to each room is called a first zone. Therefore, a user's house can include several first zones, such as a master bedroom, a living room on the first floor, a living room on the second floor, and a second bedroom, which are all a first zone.
  • the smart devices in all the first areas may be counted to determine the smart devices included in each first area.
  • the server can determine the first area to which each smart device belongs.
  • the smart home system in the user's home can be divided according to the situation of the user's house area.
  • the smart devices contained in each first area can be distinguished, and a smart device can be selected to manage other smart devices in the area.
  • the server may include a communication module and a control module, wherein the communication module may establish a communication connection with the smart device to perform information exchange.
  • the control module may first obtain the user's regional device information.
  • the regional device information refers to the first regional situation of the user's house, including a plurality of first regions into which the user's house is divided, and device information of the smart devices contained in each first region.
  • the regional device information it is possible to determine how many rooms the user's home contains and what smart devices are installed in each room. Whenever a new smart device is installed in the smart home system, the device information of the smart device and the regional situation can be reported to the server, so that the server can accurately understand the user's regional device information.
  • a smart device in order to facilitate the management of smart devices, can be selected in each first area, which is referred to as a master device in the embodiment of the present disclosure.
  • the master device can manage other smart devices in the first area, such as sending various control instructions to control other smart devices.
  • the main control device in each first area responds to the user's voice commands, and other devices do not respond.
  • the main control device then controls other smart devices to execute user commands, thereby improving the user experience.
  • the server may select a master device for each first area.
  • the server may select master devices for all first areas according to the user's regional device information, and specifically select a corresponding master device according to the device information of the smart device in each first area.
  • the server can filter among all powered-on smart devices to obtain the master control device.
  • the user can send a power-on command to the smart device, such as a voice command or pressing a power-on button of the smart device, to start the smart device.
  • a power-on command such as a voice command or pressing a power-on button of the smart device
  • the smart device may include a display, a communicator, and a controller.
  • the communicator may establish a communication connection with a server and other smart devices to achieve information interaction.
  • the controller may start the smart device to put the smart device into a power-on state.
  • the controller may send a startup message indicating that the smart device is already in a power-on state to the server.
  • the server can obtain the smart devices that have been started in each first area, that is, the smart devices that have sent the startup information. For the smart devices that have been started, the server can determine them as the first candidate devices. The server can select a device from the first candidate devices as the master control device.
  • the server may first filter out the device information of each first candidate device from the regional device information, and send the device information to the first candidate devices.
  • the device information may include the device name of the smart device, the capability information of the smart device, and the region information to which the smart device belongs. Considering that the master device may be required to control other smart devices, the stronger the capability of a smart device, the more suitable it is to be used as a master device to control other devices. To this end, the capability of each first candidate device in the first region may be evaluated to obtain the device with the highest capability as the master device.
  • the server may send the capability information of each first candidate device to each first candidate device.
  • the smart device as the first candidate device may receive the device information sent by the server, which may specifically be capability information.
  • the smart device may obtain its own capability score based on the capability information to characterize the strength of its own capability.
  • the capability information includes capability data of several capabilities of preset types, wherein the several capabilities include nearby wake-up capability, other smart device control capability, collaborative control capability, relay capability, device service capability and display capability.
  • the proximity wake-up capability refers to a function in which a smart device determines the distance between itself and the user who makes the voice, compares it with other smart devices, and selects the closest smart device to respond to the user. Whether the smart device has the proximity wake-up capability can be reported to the server during installation. If the smart device has the proximity wake-up capability, the capability data can be 1; if the smart device does not have the proximity wake-up capability, the capability data can be 0.
  • the other smart device control capability refers to the function that the smart device can control other smart devices. In the embodiment of the present disclosure, it can be further expressed as the function that the smart device can control other smart devices in the area. If the smart device has the other smart device control capability, the capability data can be 1; if the smart device does not have the other smart device control capability, the capability data can be 0.
  • Collaborative control capability refers to the function of a smart device that can collaborate with other smart devices to respond to user commands when the smart device cannot respond to user commands. For example, for a smart device TV, the user command is to adjust the room temperature, but the TV does not have the temperature adjustment capability and cannot respond. At this time, the air conditioner can be notified to adjust the room temperature. If the smart device has collaborative control capability, the capability data can be 1; if the smart device does not have collaborative control capability, the capability data can be 0.
  • the capability data may be 1; if the smart device does not have the relay capability, the capability data may be 0.
  • Device service capability refers to the services that smart devices can achieve. Services can include playing media, playing music, adjusting temperature, adjusting brightness, etc., that is, the regular service functions that smart devices can achieve.
  • the total number of services can be set in advance, and the number of services that smart devices can achieve can be determined by detecting the number of services that smart devices can achieve.
  • the percentage of the total business quantity is determined as the capability data of the smart device in terms of device business capability.
  • Display capability refers to the screen size that the display of a smart device can display, which can be the size of the display.
  • the smart device can obtain capability scores based on the capability data corresponding to these capabilities.
  • a weight coefficient can be set for each capability in advance, and the capability score can be obtained based on the weight coefficient and capability data. It can be calculated according to the following formula (1):
  • ⁇ represents the capability score
  • ⁇ represents the nearby wake-up capability
  • ⁇ represents the control capability of other smart devices
  • ⁇ represents the collaborative control capability
  • ⁇ represents the relay capability
  • ⁇ represents the device business capability
  • ⁇ represents the display capability.
  • the smart device may send the capability score to the server.
  • the server may count the capability scores of all first candidate devices in each first area, and select the first candidate device with the largest capability score as the master device.
  • the server may also calculate the capability score of each first candidate device to determine the master device. The calculation method is described above and will not be described again here.
  • the server may send a master device notification message to the master device corresponding to each first area, and the master device notification message is used to notify the master device that it is the master device of the first area where it is located.
  • the server may send a silent instruction to other smart devices other than the master device in the first area, and the silent instruction indicates not to respond to the user's voice command.
  • the smart device receives the master device notification information sent by the server, it can be determined that it is the master device of the first area to which it currently belongs, and it can respond to the user's voice command.
  • the smart device receives the silent command sent by the server, it can be confirmed that it is not the master device and does not need to respond to the user's voice command, so the user's voice command can be ignored.
  • the smart device may ask the user whether it needs to determine itself as the master device.
  • the automatic setting switch of the master control device may be set in the server, and each room (first area) may be provided with a master control device automatic setting switch.
  • the automatic setting switch of the master control device may control whether the master control device in the current first area can be automatically set without user confirmation.
  • the user may set whether to turn on the automatic setting switch of the master control device in each room through the mobile terminal.
  • the master device in the first area automatically sets the switch to the on state, after a certain smart device receives the master device notification information sent by the server, it can directly confirm that it is the master device.
  • the smart device can ask the user whether to set itself as the master device after receiving the master device notification information sent by the server.
  • the controller of the smart device can control the display to display the master device confirmation interface.
  • Figure 13 is a schematic diagram of the master device confirmation interface according to some embodiments. As shown in Figure 13, the master device confirmation interface includes master device confirmation information, confirmation controls, and cancel controls. Among them, the master device confirmation information is "whether to use this device as the master device", the confirmation control is "yes”, and the cancel control is "no".
  • the controller may determine that the smart device is the master device in the first area. In response to the user selecting the cancellation control, the controller may determine that the smart device is not the master device.
  • a threshold may be preset, and when the time exceeds the threshold, the controller may directly determine the smart device as the master device.
  • the controller detects whether the original master device is offline, for example, due to a power outage or the server notifying the original master device to be offline. If the original master device is offline, the controller directly determines this smart device as the master device; if the original master device is still online, the result obtained by this master device is discarded and the status of the original master device is maintained (that is, the previous master device remains unchanged).
  • the server may continue to search for the device with the highest scoring capability among the remaining smart devices and notify the user, or may stop searching for the master device in the first area, which is not limited in the embodiment of the present disclosure.
  • the server after the server has selected the master device, when a newly started smart device is detected, which may be a new smart device sending startup information to the server, the server can reacquire the master device.
  • the server may determine the smart device as a second candidate device.
  • the server may detect whether the second candidate device can be used as a master device.
  • the server can filter out the device information of the second candidate device and send the capability information of the second candidate device to the second candidate device.
  • the smart device as the second candidate device can calculate its capability score according to the capability information and send the capability score to the server.
  • the server may receive the capability score sent by the second candidate device and compare the capability score.
  • the server may determine the second candidate device as a new master device. At this time, the server may send a master device notification message to the second candidate device and send a silent instruction to the original master device.
  • the server can also send a silent command to the second candidate device so that the second candidate device does not respond to the user's voice command.
  • a master device notification message may be sent to the second candidate device.
  • the smart device as the second candidate device may display a master device switching interface on the display.
  • Figure 14 is a schematic diagram of a master device switching interface according to some embodiments.
  • the master device switching interface includes master device switching information, confirmation controls, and cancel controls.
  • the master device switching information may include the original master device information and information asking whether to switch, which may be "whether to switch the original master device (refrigerator) to this device", the confirmation control is "yes”, and the cancel control is "no".
  • the controller In response to the user's selection operation on the confirmation control, it means that the user switches the master device to the current device, and the controller can determine that the smart device is the master device in the first area.
  • the cancel switch command is sent to the server to notify the server user to cancel the switch of the master control device and keep the original master control device unchanged.
  • the user may not see the interface, resulting in a situation where there is no selection.
  • a threshold can be preset. When the time exceeds the threshold, it can be detected whether the original master device is offline. If the original master device is offline, the second candidate device can determine the device as the new master device. If the original master device is still online, the acquisition result of the master device can be discarded, and the state of the original master device is maintained, that is, the previous master device is kept unchanged.
  • the second candidate device may also display the master device switching interface again to prompt the user whether to switch the master device. If the user still does not make a selection after displaying the master device switching interface for a preset number of times, such as 5 times, the second candidate device may directly determine the smart device as the master device.
  • each master device after acquiring the master devices in each first area, if the user issues a voice command, each master device can respond to the user command. Multiple master devices responding at the same time may cause confusion to the user, so the server can screen all the master devices again and obtain one master device to respond to the user command.
  • All master control devices can make nearby wake-up decisions. By selecting the master control device closest to the user to determine the room the user is in, the smart home system can be woken up nearby.
  • Users can use voice to issue various commands to control smart devices, such as turning on lights, playing music, etc.
  • all master control devices can collect the user's voice command audio data, generate a command response request based on the voice command audio data, and then send the command response request to the server.
  • the server can filter all master control devices based on the command response requests sent by all master control devices to obtain the master control device that is ultimately used to respond to the user's command.
  • the master control device may generate different instruction response requests based on its own computing power.
  • the master device may preset a system occupancy rate threshold, such as 75%. When the current system occupancy rate of the master device is higher than the preset system occupancy rate threshold, it is considered that the computing power is insufficient; when the current system occupancy rate of the master device is lower than or equal to the preset system occupancy rate threshold, it is considered that the computing power is sufficient.
  • a system occupancy rate threshold such as 75%.
  • the master control device may generate a command response request based on the collected voice command audio data, so that the command response request includes the voice command audio data.
  • the master control device may also analyze the voice command audio data to obtain the voice command text.
  • the command response request may include the voice command audio data and the voice command text.
  • the main control device can analyze the collected voice command audio data to calculate the voice energy value.
  • the voice energy value is used to characterize the strength of the audio data signal, which can reflect the distance between the user and the main control device. The larger the voice energy value, the closer the user is to the main control device, and the smaller the voice energy value, the farther the user is from the main control device.
  • the main control device can generate a command response request based on the voice energy value, and the command response request can include the voice energy value and the voice command text.
  • the master device may send a command response request to the server.
  • the master control device may first perform acoustic echo cancellation (AEC, Acoustic Echo Cancellation) processing on the voice command audio data to eliminate the impact of the sound emitted by the master control device itself on the speech energy value.
  • AEC Acoustic Echo Cancellation
  • the server can analyze all the command response requests. Among them, for the command response request containing the speech energy value, the server can obtain the corresponding relationship between the speech energy value and the master device and store it.
  • the speech energy value calculated by the master device itself is called the first speech energy value.
  • the server can obtain a voice energy value based on the voice command audio data.
  • the voice energy value obtained by the server is referred to as a second voice energy value.
  • the server can obtain a corresponding relationship between the second voice energy value and the main control device.
  • the server may select the speech energy value with the largest value among all the first speech energy values and all the second speech energy values, and determine the master control device corresponding to the speech energy value as the target master control device.
  • the server can determine the first area corresponding to the target master device, which is referred to as the target first area in the embodiment of the present disclosure. It can be determined that the area where the user is currently located is the target first area. Therefore, the smart device that the user needs to control is also in the target first area, and the server can control the corresponding smart device in the area.
  • the server may obtain the device type based on the voice command.
  • the type of device to be controlled may be determined based on the voice command text sent by the master device, for example, the device type of "turn on the light" is a light.
  • the server can further narrow down the area where the smart devices to be controlled are located.
  • the first area it can be pre-divided into multiple small areas, which are called second areas in the embodiment of the present disclosure.
  • second areas For example, for the first area, the living room on the first floor, it can be divided into 9 second areas, which can be evenly divided in the form of a nine-square grid.
  • the server may obtain second area information of the target first area, wherein the second area information includes a plurality of second areas included in the target first area and smart devices in each second area.
  • the server can obtain the second area where the target master device is located based on the second area information, which is referred to as the target second area in the embodiment of the present disclosure.
  • the server can obtain the second area that contains the smart device of the device type and is closest to the target second area.
  • the center points of the two second areas can be obtained, and the distance between the two center points can be calculated.
  • the distances between all second areas can be pre-calculated and uploaded to the server for pre-storage. If there are multiple second areas with equal distances from the target second area, the nearest second area can be determined according to a preset priority. For example, a priority of front>left>right>back can be set, and the nearest second area can be screened out according to the priority.
  • the server may obtain a smart device corresponding to the device type in the nearest second area, and determine the smart device as the smart device to be controlled.
  • FIG15 is a schematic diagram of the second area of the target first area according to some embodiments.
  • the target main control device is a TV
  • the target first area can be a living room on the first floor, which can include 9 second areas.
  • the second area in the upper left corner includes a refrigerator and a lamp 4, and the second area in the middle includes The second area includes light 1, and the three second areas on the right include light 2, TV and light 3 respectively.
  • the second area where the TV is located is the target second area.
  • the user's voice command is "turn on the light"
  • the device type of the smart device to be controlled is light.
  • the server can first determine the second area with lights, and there are four second areas in total.
  • the server calculates the distance from the four second areas to the target second area, where the distances from the areas where lights 1, lights 2 and lights 3 are located to the target second area are the same. Therefore, the preset priority of front>left>right>back can be used for screening, so the second area in front of the TV (the area of light 1) is the nearest second area. Among them, light 1 is the smart device to be controlled.
  • the server may generate a response instruction according to the user's voice instruction, and send the response instruction to the smart device to be controlled.
  • the smart device to be controlled After receiving the response instruction, the smart device to be controlled can execute the response instruction to achieve the user's needs.
  • the present disclosure also provides a smart device control method, which is applied to a smart home system.
  • the method includes:
  • the server obtains regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions;
  • the server obtains a master control device in the first area based on the device information, where each first area corresponds to a master control device;
  • the smart device After detecting the user's voice command, the smart device as the master device sends a command response request to the server, where the command response request includes the voice command, and the voice command is used to control the smart device;
  • the server obtains a target master control device and a target first area corresponding to the target master control device;
  • the server determines the smart device to be controlled based on the target first area and the target main control device, and sends a response instruction to the smart device to be controlled;
  • the intelligent device to be controlled executes the response instruction.
  • the device information includes capability information of the smart device.
  • the server acquires the master device of the first area based on the device information, including:
  • the process further includes:
  • the command response request includes voice command audio data collected by the master device or a first voice energy value obtained by the master device based on the voice command.
  • the server obtains the target master device, further comprising:
  • a second voice energy value is obtained based on the voice command audio data; and a master control device corresponding to the largest voice energy value between the first voice energy value and the second voice energy value is determined as the target master control device.
  • the server after the server obtains the master device of the first area based on the device information, the server further includes:
  • the master control device notification information is used to notify the master control device that it is the master control device of the first area where the master control device is located; send a silent command to other smart devices other than the master control device in the first area, the silent command is used to instruct other smart devices not to respond to the user's voice command.
  • the server acquires the smart device to be controlled based on the target first area and the target master device, further comprising:
  • the device information includes capability information of the smart device, and the capability information includes capability data of several capabilities of preset types; the several capabilities include nearby wake-up capability, other smart device control capability, collaborative control capability, relay capability, device business capability and display capability.
  • the smart device obtains the capability score based on the device information, and also includes: calculating the capability score based on a preset weight coefficient and capability data.
  • the controller before the controller determines that the smart device is the master device of the first area where the smart device is located, the controller includes:
  • the display is controlled to display a master control device confirmation interface, which includes master control device confirmation information, confirmation controls and cancel controls; in response to a user selecting the confirmation control, the smart device is determined to be the master control device in the first area; in response to a user selecting the cancel control, the smart device is determined not to be the master control device in the first area.
  • a master control device confirmation interface which includes master control device confirmation information, confirmation controls and cancel controls

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Selective Calling Equipment (AREA)
  • Telephonic Communication Services (AREA)

Abstract

La présente divulgation concerne un serveur, un dispositif intelligent et un procédé de commande de dispositif intelligent. Un serveur peut acquérir des informations de dispositif régional, les informations de dispositif régional comprenant plusieurs premières régions et des informations de dispositif pour des dispositifs intelligents dans les premières régions. Le serveur acquiert des dispositifs de commande principaux dans les premières régions selon les informations de dispositif, et signale les dispositifs de commande principaux. Chaque dispositif de commande principal envoie au serveur une demande de réponse à une instruction après la détection d'une instruction vocale d'un utilisateur, et le serveur peut acquérir un dispositif de commande principal cible et une première région cible correspondant au dispositif de commande principal cible. Sur la base de la première région cible et du dispositif de commande principal cible, le serveur détermine un dispositif intelligent à commander et envoie une instruction de réponse au dispositif intelligent à commander. Le dispositif intelligent à commander peut exécuter l'instruction de réponse de façon à obtenir des exigences d'utilisateur. Par conséquent, le procédé évite de demander à un utilisateur le dispositif qui est en marche, ce qui permet d'améliorer la convivialité d'utilisation pour l'utilisateur.
PCT/CN2023/091857 2022-11-23 2023-04-28 Serveur, dispositif intelligent et procédé de commande de dispositif intelligent WO2024108905A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211476607.8A CN115834271A (zh) 2022-11-23 2022-11-23 服务器、智能设备和智能设备控制方法
CN202211476607.8 2022-11-23

Publications (2)

Publication Number Publication Date
WO2024108905A1 true WO2024108905A1 (fr) 2024-05-30
WO2024108905A9 WO2024108905A9 (fr) 2024-07-11

Family

ID=85530777

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/091857 WO2024108905A1 (fr) 2022-11-23 2023-04-28 Serveur, dispositif intelligent et procédé de commande de dispositif intelligent

Country Status (2)

Country Link
CN (1) CN115834271A (fr)
WO (1) WO2024108905A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115834271A (zh) * 2022-11-23 2023-03-21 海信视像科技股份有限公司 服务器、智能设备和智能设备控制方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160378084A1 (en) * 2015-06-29 2016-12-29 Samsung Electronics Co., Ltd. Method and apparatus for controlling device of one region among a plurality of regions
CN106681292A (zh) * 2017-01-16 2017-05-17 上海智臻智能网络科技股份有限公司 智能中控分机设备、实现智能控制的方法和系统
CN111308904A (zh) * 2020-05-14 2020-06-19 星络智能科技有限公司 智能家居控制方法、主控制设备、子控制设备及存储介质
CN114120996A (zh) * 2021-10-21 2022-03-01 青岛海尔科技有限公司 语音交互方法及装置
CN115834271A (zh) * 2022-11-23 2023-03-21 海信视像科技股份有限公司 服务器、智能设备和智能设备控制方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160378084A1 (en) * 2015-06-29 2016-12-29 Samsung Electronics Co., Ltd. Method and apparatus for controlling device of one region among a plurality of regions
CN106681292A (zh) * 2017-01-16 2017-05-17 上海智臻智能网络科技股份有限公司 智能中控分机设备、实现智能控制的方法和系统
CN111308904A (zh) * 2020-05-14 2020-06-19 星络智能科技有限公司 智能家居控制方法、主控制设备、子控制设备及存储介质
CN114120996A (zh) * 2021-10-21 2022-03-01 青岛海尔科技有限公司 语音交互方法及装置
CN115834271A (zh) * 2022-11-23 2023-03-21 海信视像科技股份有限公司 服务器、智能设备和智能设备控制方法

Also Published As

Publication number Publication date
WO2024108905A9 (fr) 2024-07-11
CN115834271A (zh) 2023-03-21

Similar Documents

Publication Publication Date Title
KR102114003B1 (ko) 음성 사용자 인터페이스들의 시각적 어포던스를 위한 led 설계 언어
WO2023040419A1 (fr) Dispositif d'affichage et procédé de continuation de lecture d'un contenu
CN111277884B (zh) 视频播放方法及设备
US20240053944A1 (en) Display apparatus and method for controlling screen projections from multiple devices to same screen
CN111405331B (zh) 一种蓝牙Mesh设备状态更新方法及显示设备
US20200213653A1 (en) Automatic input selection
WO2024108905A1 (fr) Serveur, dispositif intelligent et procédé de commande de dispositif intelligent
WO2023155529A1 (fr) Dispositif d'affichage, système domestique intelligent et procédé de commande multi-écran pour dispositif d'affichage
CN114067798A (zh) 一种服务器、智能设备及智能语音控制方法
WO2022268136A1 (fr) Dispositif terminal et serveur pour commande vocale
CN112929592A (zh) 一种视频通话方法、显示设备及服务器
CN116761026A (zh) 终端设备和媒资接力播放方法
WO2024098974A1 (fr) Dispositif d'affichage et procédé de commande pour dispositif externe
CN113938634A (zh) 一种多路视频通话处理方法及显示设备
WO2023024630A1 (fr) Dispositif d'affichage, dispositif terminal et procédé d'affichage de contenu
CN109547858A (zh) 电视机控制方法及装置
JP2020061011A (ja) 表示方法、表示装置、及び表示プログラム
WO2021218096A1 (fr) Procédé de réglage d'ordre de commandes de canal et dispositif d'affichage
CN115705222A (zh) 终端设备和内容反向接力方法
CN111312248A (zh) 一种交互方法、装置、系统及存储介质
WO2024174648A1 (fr) Dispositif source, dispositif relais et procédé de commande de dispositif
CN118354022A (zh) 服务器、电子设备和视频通话转接方法
CN115223521B (zh) 显示设备及接力设备展示方法
WO2024087824A9 (fr) Dispositif d'affichage et procédé de traitement de données de canal pour dispositif d'affichage
WO2023173757A1 (fr) Dispositifs d'affichage, procédé de recherche bluetooth et procédé de traitement de demande de connexion bluetooth

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23893006

Country of ref document: EP

Kind code of ref document: A1