WO2019184533A1 - Method and apparatus for remotely configuring control panel of intelligent device - Google Patents

Method and apparatus for remotely configuring control panel of intelligent device Download PDF

Info

Publication number
WO2019184533A1
WO2019184533A1 PCT/CN2018/126030 CN2018126030W WO2019184533A1 WO 2019184533 A1 WO2019184533 A1 WO 2019184533A1 CN 2018126030 W CN2018126030 W CN 2018126030W WO 2019184533 A1 WO2019184533 A1 WO 2019184533A1
Authority
WO
WIPO (PCT)
Prior art keywords
smart device
control panel
interaction interface
protocol
operation information
Prior art date
Application number
PCT/CN2018/126030
Other languages
French (fr)
Chinese (zh)
Inventor
邓鹏�
Original Assignee
深圳市九洲电器有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市九洲电器有限公司 filed Critical 深圳市九洲电器有限公司
Publication of WO2019184533A1 publication Critical patent/WO2019184533A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/02Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • H04L67/025Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2807Exchanging configuration information on appliance services in a home automation network
    • H04L12/2809Exchanging configuration information on appliance services in a home automation network indicating that an appliance service is present in a home automation network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2807Exchanging configuration information on appliance services in a home automation network
    • H04L12/2814Exchanging control software or macros for controlling appliance services in a home automation network
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2816Controlling appliance services of a home automation network by calling their functionalities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0803Configuration setting
    • H04L41/0813Configuration setting characterised by the conditions triggering a change of settings
    • H04L41/0816Configuration setting characterised by the conditions triggering a change of settings the condition being an adaptation, e.g. in response to network events
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/085Retrieval of network configuration; Tracking network configuration history
    • H04L41/0853Retrieval of network configuration; Tracking network configuration history by actively collecting configuration information or by backing up configuration information
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H04M1/72415User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72469User interfaces specially adapted for cordless or mobile telephones for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H04M1/72412User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories using two-way short-range wireless interfaces
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions

Definitions

  • the embodiments of the present invention relate to the field of intelligent terminal devices, and in particular, to a method and an apparatus for remotely setting a control panel of a smart device.
  • the terminal device when the terminal device is used to control other terminals, the terminal device needs to first send the setting adjustment command to the external network server, and the external network server forwards the setting adjustment command to the controlled terminal.
  • the setting adjustment command is transmitted through the external network server, and the reliability of the user data and the reliability of the control are poor.
  • the terminal device can only issue fixed and limited setting adjustment commands for control, the operation is cumbersome and complicated, and the operation mode is relatively rigid.
  • An object of the present invention is to provide a method and system for remotely setting a control panel of a smart device, so that control between the terminal device and the smart device can be implemented not based on an external network server, and the user data security and control reliability are improved. Sex. Moreover, the terminal device can display the control panel interaction interface in the human-computer interaction interface for the user to input the operation information, realize the visual control operation for the smart device, the operation is simple and convenient, and the operation mode is flexible.
  • an embodiment of the present invention provides a method for remotely setting a control panel of a smart device, which is characterized in that, the application to the terminal device includes:
  • Searching for a smart device in a local area network determining a target smart device in the searched smart device, and establishing a connection with the target smart device;
  • the searching for the smart device in the local area network includes:
  • the smart device is searched through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
  • the detecting the operation information input to the control panel interaction interface further includes:
  • a human-computer interaction interface including a display control is created, and the control panel interaction interface is rendered into a display control of the human-computer interaction interface in the form of a video or an image.
  • the projection screen protocol includes at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
  • the detecting operation information input to the control panel interaction interface includes:
  • the operation information including at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
  • an embodiment of the present invention provides an apparatus for remotely setting a control panel of a smart device, wherein the application to the terminal device includes:
  • Searching for a smart device in a local area network determining a target smart device in the searched smart device, and establishing a connection with the target smart device;
  • the searching for the smart device in the local area network includes:
  • the smart device is searched through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
  • the detecting the operation information input to the control panel interaction interface further includes:
  • a human-computer interaction interface including a display control is created, and the control panel interaction interface is rendered into a display control of the human-computer interaction interface in the form of a video or an image.
  • the projection screen protocol includes at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
  • the detecting operation information input to the control panel interaction interface includes:
  • the operation information including at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
  • the method and device for remotely setting a control panel of a smart device so that when a user remotely controls a smart speaker, a smart air conditioner, a smart refrigerator, or the like, which does not display an interactive interface or displays a smart terminal device having a small interactive interface, using a mobile phone, a tablet, or a notebook computer,
  • the adjustable control items can be visually displayed to the user through the control panel interactive interface sent by the smart speaker, the smart air conditioner, and the smart refrigerator through the projection screen protocol, and the user can complete the volume adjustment of the smart speaker through the graphical setting interface.
  • the operation of the temperature adjustment of the intelligent air conditioner improves the convenience of operation.
  • FIG. 1 is a detailed flowchart of a method of remotely setting a control panel of a smart device according to the present invention
  • FIG. 2 is a detailed flowchart of a method of remotely setting a control panel of a smart device according to the present invention
  • FIG. 3 is a detailed flowchart of a method for remotely setting a control panel of a smart device according to the present invention
  • FIG. 4 is a schematic structural diagram of a terminal device according to the present invention.
  • FIG. 5 is a schematic structural diagram of a smart device according to the present invention.
  • FIG. 6 is a schematic structural diagram of a computer device for operating the foregoing method for remotely setting a control panel of a smart device in an embodiment.
  • a method for remotely setting a control panel of the smart device is proposed, and the implementation of the method may depend on Computer program.
  • a terminal device such as a mobile phone, a tablet computer, a palmtop computer, a notebook computer, or a personal PC
  • the user can be in a smart device (such as a smart speaker, a smart air conditioner, a smart refrigerator, etc.) that is on the same local area network as the terminal device.
  • the control panel is adjusted and the operation is more convenient and concise.
  • the method for remotely setting a control panel of the smart device includes:
  • Step 101 Search for a smart device in a local area network, determine a target smart device in the searched smart device, and establish a connection with the target smart device.
  • the terminal device can send a device query request to a network access device (eg, a wireless router).
  • a network access device eg, a wireless router.
  • the network access device acquires information of each smart device that is currently connected to itself, and feeds back the information to the terminal device.
  • the terminal device sets each smart device as a target smart device according to the information of each smart device, and establishes a connection with each smart device.
  • the terminal device may be an electronic device such as a mobile phone, a computer, or a tablet computer, and the smart device may be a video playback device such as a smart speaker or a smart TV.
  • Step 102 Send a control switch instruction to the target smart device by using the connection, and receive a control panel interaction interface of the target smart device that is fed back by the target smart device based on a projection screen protocol.
  • the control switch command is a switch instruction initiated by the terminal device operated by the user terminal to set the target smart device.
  • the terminal device operated by the user terminal sends the control switch command to the target smart device
  • the terminal device operated by the user terminal expects Adjust the settings of the target smart device. For example, volume adjustment, temperature adjustment, power adjustment, and the like.
  • the corresponding control panel interaction interface may be fed back to the terminal device operated by the user terminal.
  • the control panel interactive interface of the volume adjustment can be sent to the mobile phone operated by the user through the projection protocol, and then displayed on the mobile phone, so that the user can know the setting items that can adjust the smart speaker.
  • the projection screen protocol may include at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
  • Step 103 Detect operation information input to the control panel interaction interface.
  • the user can operate on the terminal device according to the content displayed in the interactive interface of the control panel by using an external device such as a mouse or a keyboard, so that the terminal device can detect the operation information input by the user according to the operation input by the external device.
  • an external device such as a mouse or a keyboard
  • the terminal device when the terminal device is provided with the touch display screen, when the terminal device detects the operation information input to the control panel interaction interface, the terminal device performs the touch event monitoring on the control panel interaction interface, according to the monitored touch event. Get the input operation information, so as to obtain the operation information according to the monitored touch event.
  • the operation information includes at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
  • the terminal device does not detect any specific implementation form of the operation information that the user maps to the operation interface in the human-machine interaction interface.
  • Step 104 Send the setting adjustment instruction to the target smart device, so that the target smart device adjusts the configuration of the control panel by executing the setting adjustment instruction.
  • the terminal device prestores a correspondence between the operation information and the setting adjustment command, and the terminal device queries and generates a setting adjustment instruction corresponding to the operation information in the pre-stored correspondence relationship.
  • setting the adjustment command can set the adjustment instruction in real time, and can also set the adjustment instruction for the delay.
  • the terminal device when the terminal device sends the generated setting adjustment command to the smart device, the setting adjustment command and the information of the smart device may be sent to the network access device together, and the network access device forwards the smart device to the smart device;
  • the setting adjustment command is sent to the smart device via the Internet directly according to the information of the smart device.
  • the terminal device sends the setting adjustment command to the smart device by using a TCP network protocol, and the reliability and continuity of the event are high.
  • the control between the terminal device and the smart device can be implemented not based on the external network server, and the user data security and the reliability of the control are improved.
  • the terminal device can display the control panel interaction interface in the human-computer interaction interface for the user to input the operation information, realize real-time control or delay control for the visualization of the smart device, the operation is simple and convenient, and the operation mode is flexible.
  • a second embodiment of the present invention relates to a method for remotely setting a control panel of a smart device.
  • the specific process is shown in FIG. 2 .
  • the second embodiment is improved on the basis of the first embodiment, and the main improvement is that in the second embodiment of the present invention, a specific implementation of establishing a connection between the terminal device and the target smart device in the same local area network is provided.
  • Form which increases the flexibility of embodiments of the invention.
  • Steps 202 to 204 in this embodiment are substantially the same as steps 102 to 104 in the first embodiment. To reduce repetition, details are not described herein. Only the different parts are described below:
  • Step 201 Establish a connection with a target smart device in the same local area network.
  • step 201 includes sub-steps 2011 to sub-steps 2014, which are specifically described below:
  • Sub-step 2011 searching for the smart device through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
  • the terminal device and the smart device interact through a local area network, and the terminal device sends a broadcast message through a user datagram protocol UDP of the local area network.
  • the broadcast message is used to control the smart device to send a response message.
  • the response message includes device information of the smart device. In this way, the terminal device can automatically detect all the smart devices located in the current local area network through the local area network.
  • Sub-step 2012 the searched smart device is used as a target smart device in the same local area network.
  • the terminal device receives the response message fed back by the smart device by listening to a TCP (Transmission Control Protocol, "TCP") port and a UDP port. Then, the terminal device sets the target smart device according to the device information carried in the response message.
  • TCP Transmission Control Protocol
  • the terminal device when acquiring the information of the smart device, adds the information of the smart device to the local list, updates the local list, and maintains the local list. Specifically, the terminal device detects the information of the smart device in the local list before adding the information of the searched smart device to the local list, and discards the information code of the smart device if the information of the smart device already exists. If the information of the smart device does not exist, the information of the searched smart device is added to the local list. In this way, the terminal device can not only add all the smart devices in the local area network at one time, but also achieve the purpose of rapid addition and fast control, and can prevent the information of the same smart device from being repeatedly added, and is easy to maintain and update the local list.
  • the terminal device can automatically acquire all the smart devices located in the current local area network, and provides a basis for the user to implement control on multiple smart devices on the terminal device side.
  • a third embodiment of the present invention relates to a method of remotely setting a control panel of a smart device.
  • the third embodiment is improved on the basis of the first embodiment.
  • the main improvement is that in the third embodiment of the present invention, a terminal device is provided to display a control panel interaction interface in a message based on a projection screen protocol.
  • the specific implementation form increases the flexibility of the embodiment of the present invention.
  • the terminal device After the terminal device establishes a connection with the target smart device in the same local area network, the terminal device sends a request for recording the control panel interaction interface to the target smart device, so that the target smart device records the control panel interaction interface, and generates a data packet. . Then, the target smart device carries the data packet to the terminal device in the message based on the projection screen protocol.
  • the target smart device may transmit the message based on the screencast protocol to the terminal device by using a TCP network protocol or a UDP network protocol.
  • the terminal device parses the data packet in the message based on the projection screen protocol to obtain a control panel interaction interface. Then, the terminal device creates a human-computer interaction interface including a display control, and renders the control panel interaction interface into a display control of the human-computer interaction interface in the form of a video or an image.
  • the data packet carrying the control panel interaction interface may include coding parameters, video frames, and video parameters, and the terminal device parses the data packet, obtains coding parameters, video frames, and video parameters in the data packet, and the video parameters.
  • the video frame is sent to the decoder, and is decoded by using a hard decoder provided by the system to obtain a video image. Then, the terminal device creates a page control, and renders the decoded video image onto the page control, thereby realizing the local display of the control panel interaction interface.
  • the terminal device when the terminal device receives the message based on the screencast protocol sent by the target smart device, the terminal device receives the message based on the screencast protocol sent by the target smart device by listening to the TCP port and the UDP port, thereby ensuring the terminal.
  • the device receives a message based on the projection protocol.
  • control panel interaction interface in the message based on the projection screen protocol is provided, which is simple and easy to implement, and the flexibility of the embodiment of the present invention is increased.
  • a fourth embodiment of the present invention relates to a method for remotely setting a control panel of a smart device, which is applied to a smart device, and the specific process is shown in FIG. 3 .
  • the core of the embodiment is that the control panel interaction interface recorded by the target smart device is displayed in the terminal device, and the control operation for visualizing the smart device is implemented on the terminal device side according to the operation information input by the user on the control panel interaction interface.
  • the implementation details of the method for remotely setting the control panel of the smart device of the present embodiment are specifically described below. The following content is merely for facilitating understanding of the implementation details provided, and is not necessary for implementing the solution.
  • Step 301 establishing a connection with a terminal device in the same local area network.
  • Step 302 Record a control panel interaction interface when detecting a control switch command from the terminal device.
  • the target smart device may or may not have a virtual display screen. More specifically, the target smart device can create a virtual display using the create virtual presentation method of the system display manager class and render the content on the virtual display onto the page control. Second, the target smart device can be hardcoded using the system media encoder class with the page control as its input. Finally, the target smart device takes relevant parameters (such as encoding parameters, video parameters, etc.) and the encoded video frames from the encoder, and encapsulates the relevant parameters and video frames as a recorded control panel interactive interface.
  • relevant parameters such as encoding parameters, video parameters, etc.
  • Step 303 The recorded control panel interaction interface is sent to the terminal device by using a projection screen protocol.
  • the target smart device carries the encapsulated information in a message based on a projection protocol to the terminal device.
  • the target smart device may transmit a message based on a projection screen protocol by using a TCP network protocol or a UDP network protocol.
  • Step 304 Receive a setting adjustment instruction sent by the terminal device.
  • the target smart device listens to the TCP port and receives the setting adjustment command sent by the terminal device, which can ensure the reliability and continuity of the event.
  • Step 305 performing an operation corresponding to setting the adjustment instruction.
  • the smart device can use the instrumentation adjustment instruction according to the setting adjustment instruction to simulate the operation information of the user on the control panel interaction interface in the human-computer interaction interface on the terminal device, and realize the control operation of the terminal device side for the visualization of the smart device.
  • the control between the terminal device and the smart device can be implemented not based on the external network server, and the user data security and the reliability of the control are improved.
  • the terminal device can display the control panel interaction interface in the human-computer interaction interface for the user to input the operation information, realizing the real-time control or delay control for the visualization of the smart device, the operation is simple and convenient, and the operation mode is flexible.
  • a fifth embodiment of the present invention relates to a terminal device 41.
  • the device includes: a smart device connection module 411, a receiving module 412, a display module 413, a detection module 414, an instruction generation module 415, and an instruction sending module 416.
  • a smart device connection module 411 the device includes: a smart device connection module 411, a receiving module 412, a display module 413, a detection module 414, an instruction generation module 415, and an instruction sending module 416.
  • the smart device connection module 411 is configured to establish a connection with a target smart device in the same local area network.
  • the smart device connection module 411 can send a device query request to a network access device (eg, a wireless router).
  • a network access device eg, a wireless router.
  • the network access device acquires the information of each smart device that is currently connected to itself, and feeds back the information to the smart device connection module 411.
  • the smart device connection module 411 sets each smart device as a target smart device according to the information of each smart device, and establishes a connection with each smart device.
  • the receiving module 412 is configured to receive, by using a connection, a projection screen based message sent by the target smart device.
  • the message based on the projection screen protocol includes a control panel interaction interface recorded by the target smart device.
  • the display module 413 is configured to display a control panel interaction interface in the message based on the projection protocol.
  • the terminal device displays the control panel interaction interface in a local human-computer interaction interface for display.
  • the detecting module 414 is configured to detect operation information input to the control panel interaction interface.
  • the user can operate by using an external device such as a mouse or a keyboard according to the content displayed in the interface of the control panel, so that the detecting module 414 detects the operation information input by the user according to the operation input by the external device.
  • an external device such as a mouse or a keyboard according to the content displayed in the interface of the control panel
  • the detecting module 414 can include a monitor sub-module and a processing sub-module.
  • the monitoring sub-module is configured to perform monitoring of the touch event on the control panel interaction interface; and the processing sub-module is configured to obtain the input operation information according to the monitored touch event.
  • the touch event may be an operation event such as clicking, pressing, and sliding.
  • the detection module 414 does not detect any specific implementation form of the user's operation information mapped to the operation interface in the human-computer interaction interface.
  • the instruction generation module 415 is configured to generate a setting adjustment instruction for controlling the target smart device according to the detected operation information.
  • the instruction generation module 415 prestores the correspondence between the operation information and the setting adjustment instruction, and the instruction generation module 415 queries and generates the setting adjustment instruction corresponding to the operation information in the pre-stored correspondence.
  • setting the adjustment command can set the adjustment instruction in real time, and can also set the adjustment instruction for the delay.
  • the instruction sending module 416 is configured to send the setting adjustment instruction to the target smart device, where the target smart device performs an operation corresponding to the setting adjustment instruction.
  • the command sending module 416 may send the set adjustment command and the information of the smart device to the network access device together, and then forward the information to the smart device by the network access device;
  • the command sending module 416 can also directly send the setting adjustment command to the smart device via the Internet according to the information of the smart device.
  • the command sending module 416 sends the setting adjustment command to the smart device by using the TCP network protocol, and the reliability and continuity of the event are high.
  • the present embodiment is an apparatus embodiment corresponding to the first embodiment, and the present embodiment can be implemented in cooperation with the first embodiment.
  • the related technical details mentioned in the first embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
  • the sixth embodiment of the present invention relates to a terminal device.
  • the sixth embodiment is improved on the basis of the fifth embodiment.
  • the main improvement is that in the sixth embodiment of the present invention, the smart device connection module 411 is provided.
  • a specific implementation form increases the flexibility of embodiments of the present invention.
  • the smart device connection module 411 includes: a search submodule, a designation submodule, and a connection submodule.
  • the search sub-module is used to search for smart devices through the broadcast packet and multicast packet protocol of the local area network user datagram UDP. Specifies a sub-module for using the discovered smart device as a target smart device within the same local area network.
  • the search submodule sends a broadcast message through the User Datagram Protocol UDP of the local area network.
  • the broadcast message is used to control the smart device to send a response message.
  • the response message includes device information of the smart device.
  • the search submodule receives the response message fed back by the smart device by listening to the TCP port and the UDP port. Then, the designated sub-module sets the target smart device according to the device information carried in the response message.
  • the present embodiment is an apparatus embodiment corresponding to the second embodiment, and the present embodiment can be implemented in cooperation with the second embodiment.
  • the related technical details mentioned in the second embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the second embodiment.
  • the seventh embodiment of the present invention relates to a terminal device, and the seventh embodiment is improved on the basis of the fifth embodiment.
  • the main improvement is that in the seventh embodiment of the present invention, a display module 413 is provided.
  • the specific implementation form increases the flexibility of the embodiment of the present invention.
  • the receiving module 412 receives the message based on the screencast protocol sent by the target smart device by listening to the TCP port and the UDP port when receiving the message based on the screencast protocol sent by the target smart device. It is possible to ensure that the terminal device receives a message based on the projection protocol.
  • the presentation module 413 includes: a control creation sub-module and a rendering sub-module.
  • the control creates a submodule for creating display controls in the local human interface.
  • a rendering submodule for rendering the control panel interaction interface into the display control in the form of a video image.
  • the present embodiment is an apparatus embodiment corresponding to the third embodiment, and the present embodiment can be implemented in cooperation with the third embodiment.
  • the related technical details mentioned in the third embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the third embodiment.
  • the eighth embodiment of the present invention relates to a smart device 42.
  • the smart device 42 includes a terminal device connection module 421, a response module 422, a sending module 423, an instruction receiving module 424, and a control module 425.
  • the terminal device connection module 421 is configured to establish a connection with the terminal device in the same local area network.
  • the response module 422 is configured to record a control panel interaction interface when detecting a message from the terminal device.
  • the target smart device may or may not have a virtual display screen.
  • response module 422 can create a virtual display using the create virtual presentation method of the system display manager class and render the content on the virtual display onto the page control.
  • the response module 422 can be hard coded using the system media encoder class with the page control as its input.
  • the response module 422 takes relevant parameters (eg, encoding parameters, video parameters, etc.) and the encoded video frames from the encoder, and encapsulates the relevant parameters and video frames as a recorded control panel interactive interface.
  • the sending module 423 is configured to send the recorded control panel interaction interface to the terminal device through the connection.
  • the sending module 423 carries the encapsulated information in a message based on the projection protocol to the terminal device.
  • the sending module 423 can transmit a message based on a projection screen protocol by using a TCP network protocol or a UDP network protocol.
  • the instruction receiving module 424 is configured to receive a setting adjustment instruction sent by the terminal device.
  • the instruction receiving module 424 listens to the TCP port and receives the setting adjustment command sent by the terminal device, which can ensure the reliability and continuity of the event.
  • the control module 425 is configured to perform an operation corresponding to the setting adjustment instruction.
  • control module 425 can use the instrumentation adjustment instruction to simulate the operation information of the user on the control panel interaction interface in the human-computer interaction interface on the terminal device, and realize the control of the visualization of the smart device on the terminal device side. operating.
  • the present embodiment is an apparatus embodiment corresponding to the fourth embodiment, and the present embodiment can be implemented in cooperation with the fourth embodiment.
  • the related technical details mentioned in the fourth embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
  • each module involved in this embodiment is a logic module.
  • a logical unit may be a physical unit, a part of a physical unit, or multiple physical entities. A combination of units is implemented.
  • the present embodiment does not introduce a unit that is not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in the present embodiment.
  • a ninth embodiment of the present invention relates to a remote control system for a smart device, comprising: the terminal device mentioned in any of the fifth to seventh embodiments, and the smart device mentioned in the eighth embodiment .
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .
  • the method and device for remotely setting a control panel of a smart device so that when a user remotely controls a smart speaker, a smart air conditioner, a smart refrigerator, or the like, which does not display an interactive interface or displays a smart terminal device having a small interactive interface, using a mobile phone, a tablet, or a notebook computer,
  • the adjustable control items can be visually displayed to the user through the control panel interactive interface sent by the smart speaker, the smart air conditioner, and the smart refrigerator through the projection screen protocol, and the user can complete the volume adjustment of the smart speaker through the graphical setting interface.
  • the operation of the temperature adjustment of the intelligent air conditioner improves the convenience of operation.
  • FIG. 6 illustrates a server device of a von Neumann system-based computer system 10 that operates the method of remotely setting a control panel of a smart device as described above.
  • the computer system may be a server, a server cluster, or a server cluster device that provides a virtual machine as a server program execution environment to execute the above content delivery method.
  • the network interface 1001, the processor 1002, and the memory 1003 connected through a system bus may be included.
  • the memory 1003 may include an external memory 10032 (eg, a hard disk, an optical disk, or a floppy disk, etc.) and an internal memory 10034.
  • the operation of the method is based on a computer program, the program file of which is stored in the external memory 10032 of the aforementioned von Neumann system-based computer system 10, and is loaded into the internal memory 10034 at runtime, It is then compiled into a machine code and passed to the processor 1002 for execution, so that the logical initialization module 102, the friend lookup module 104, the content propagation evaluation value update module 106, and the content are formed in the von Neumann system-based computer system 10.
  • Data push module 108 During the execution of the content delivery method, the input parameters are received by the network interface 1001, and are transferred to the cache in the memory 1003, and then input to the processor 1002 for processing, and the processed result data is cached in the memory 1003 for subsequent processing. It is processed or passed to the network interface 1001 for output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Security & Cryptography (AREA)
  • Selective Calling Equipment (AREA)
  • Telephonic Communication Services (AREA)

Abstract

A method for remotely configuring a control panel of an intelligent device, which is applied to a terminal device and comprises: searching for intelligent devices in a local area network, determining a target intelligent device among the found intelligent devices, and establishing a connection to the target intelligent device; sending a control switch instruction to the target intelligent device by means of the connection, and receiving a control panel interactive interface of the target intelligent device fed back by the target intelligent device on the basis of a projection screen protocol; detecting operation information inputted for the control panel interactive interface; generating a setting adjustment instruction used for controlling the target intelligent device according to the operation information; and sending the setting adjustment instruction to the target intelligent device so as to enable the target intelligent device to adjust the configuration of the control panel by means of executing the setting adjustment instruction. Also disclosed is an apparatus for remotely configuring the control panel of the intelligent device. The operations of the present method and apparatus for remotely configuring the control panel of the intelligent device are more convenient.

Description

远程设置智能设备的控制面板的方法及装置Method and device for remotely setting control panel of smart device 技术领域Technical field
本发明实施例涉及智能终端设备领域,特别涉及远程设置智能设备的控制面板的方法及装置。The embodiments of the present invention relate to the field of intelligent terminal devices, and in particular, to a method and an apparatus for remotely setting a control panel of a smart device.
背景技术Background technique
随着移动通讯和移动互联网技术的不断发展,以及人们生活水平的不断提高,智能终端设备在人们生活当中越来越普及,已经成为了人们日常生活中不可或缺的一部分。目前,人们越来越关注在不同电子设备之间共享内容、相互控制的应用,如,文字内容、图片、音乐或视频的共享与控制。With the continuous development of mobile communication and mobile Internet technologies, and the continuous improvement of people's living standards, smart terminal devices have become more and more popular in people's lives, and have become an indispensable part of people's daily life. At present, people are paying more and more attention to sharing and controlling applications between different electronic devices, such as text content, pictures, music or video.
但是,本专利申请的发明人发现现有技术中至少存在如下问题:However, the inventors of the present patent application have found that at least the following problems exist in the prior art:
现有技术中人们利用终端设备控制其他终端时,需要先利用终端设备将设置调节指令发送给外网服务器,由外网服务器转发设置调节指令给被控终端。通过外网服务器传输设置调节指令,用户数据安全性和控制的可靠性差。并且,终端设备只能发出固定且有限的设置调节指令进行控制,操作繁琐复杂,且操作模式较为呆板。In the prior art, when the terminal device is used to control other terminals, the terminal device needs to first send the setting adjustment command to the external network server, and the external network server forwards the setting adjustment command to the controlled terminal. The setting adjustment command is transmitted through the external network server, and the reliability of the user data and the reliability of the control are poor. Moreover, the terminal device can only issue fixed and limited setting adjustment commands for control, the operation is cumbersome and complicated, and the operation mode is relatively rigid.
发明内容Summary of the invention
本发明实施方式的目的在于提供一种远程设置智能设备的控制面板的方法及系统,使得终端设备与智能设备之间的控制能够不基于外网服务器实现,提高了用户数据安全性和控制的可靠性。并且,终端设备能够在人机交互界面中显示控制面板交互界面,以供用户输入操作信息,实现了针对智能设备的可视化的控制操作,操作简单便捷,且操作模式较为灵活。An object of the present invention is to provide a method and system for remotely setting a control panel of a smart device, so that control between the terminal device and the smart device can be implemented not based on an external network server, and the user data security and control reliability are improved. Sex. Moreover, the terminal device can display the control panel interaction interface in the human-computer interaction interface for the user to input the operation information, realize the visual control operation for the smart device, the operation is simple and convenient, and the operation mode is flexible.
为解决上述技术问题,本发明的实施方式提供了一种远程设置智能设备的控制面板的方法,其特征在于,应用于终端设备,包括:In order to solve the above technical problem, an embodiment of the present invention provides a method for remotely setting a control panel of a smart device, which is characterized in that, the application to the terminal device includes:
在局域网中搜索智能设备,在搜索到的智能设备中确定目标智能设备,与所述目标智能设备建立连接;Searching for a smart device in a local area network, determining a target smart device in the searched smart device, and establishing a connection with the target smart device;
通过所述连接向所述目标智能设备发送控制开关指令,接收所述目标智能设备基于投屏协议反馈的所述目标智能设备的控制面板交互界面;Sending, by the connection, a control switch instruction to the target smart device, and receiving a control panel interaction interface of the target smart device based on the projection protocol feedback;
检测针对所述控制面板交互界面输入的操作信息;Detecting operation information input to the control panel interaction interface;
根据所述操作信息生成用于控制所述目标智能设备的设置调节指令;Generating a setting adjustment instruction for controlling the target smart device according to the operation information;
将所述设置调节指令发送至所述目标智能设备,以使所述目标智能设备通过执行所述设置调节指令调节控制面板的配置。And transmitting the setting adjustment instruction to the target smart device, so that the target smart device adjusts a configuration of the control panel by executing the setting adjustment instruction.
在其中一个实施例中,所述在局域网中搜索智能设备包括:In one of the embodiments, the searching for the smart device in the local area network includes:
通过局域网用户数据报UDP的广播包和组播包协议搜索智能设备。The smart device is searched through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
在其中一个实施例中,所述检测针对所述控制面板交互界面输入的操作信息之前还包括:In one embodiment, the detecting the operation information input to the control panel interaction interface further includes:
创建包括显示控件的人机交互界面,并将所述控制面板交互界面以视频或图像的形式渲染到所述人机交互界面的显示控件中。A human-computer interaction interface including a display control is created, and the control panel interaction interface is rendered into a display control of the human-computer interaction interface in the form of a video or an image.
在其中一个实施例中,所述投屏协议包括DLNA协议、Miracast协议和Airplay协议中的至少一种。In one of the embodiments, the projection screen protocol includes at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
在其中一个实施例中,所述检测针对所述控制面板交互界面输入的操作信息包括:In one embodiment, the detecting operation information input to the control panel interaction interface includes:
对所述控制面板交互界面进行触摸事件的监听,根据监听到的触摸事件获取输入的操作信息,所述操作信息包括触摸位置、滑动轨迹、触摸次数、触摸时长和按压力度中的至少一种。Performing monitoring of the touch event on the control panel interaction interface, and acquiring the input operation information according to the monitored touch event, the operation information including at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
此外,为解决上述技术问题,本发明的实施方式提供了一种远程设置智能设备的控制面板的装置,其特征在于,应用于终端设备,包括:In addition, in order to solve the above technical problem, an embodiment of the present invention provides an apparatus for remotely setting a control panel of a smart device, wherein the application to the terminal device includes:
在局域网中搜索智能设备,在搜索到的智能设备中确定目标智能设备,与所述目标智能设备建立连接;Searching for a smart device in a local area network, determining a target smart device in the searched smart device, and establishing a connection with the target smart device;
通过所述连接向所述目标智能设备发送控制开关指令,接收所述目标智能设备基于投屏协议反馈的所述目标智能设备的控制面板交互界面;Sending, by the connection, a control switch instruction to the target smart device, and receiving a control panel interaction interface of the target smart device based on the projection protocol feedback;
检测针对所述控制面板交互界面输入的操作信息;Detecting operation information input to the control panel interaction interface;
根据所述操作信息生成用于控制所述目标智能设备的设置调节指令;Generating a setting adjustment instruction for controlling the target smart device according to the operation information;
将所述设置调节指令发送至所述目标智能设备,以使所述目标智能设备通过执行所述设置调节指令调节控制面板的配置。And transmitting the setting adjustment instruction to the target smart device, so that the target smart device adjusts a configuration of the control panel by executing the setting adjustment instruction.
在其中一个实施例中,所述在局域网中搜索智能设备包括:In one of the embodiments, the searching for the smart device in the local area network includes:
通过局域网用户数据报UDP的广播包和组播包协议搜索智能设备。The smart device is searched through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
在其中一个实施例中,所述检测针对所述控制面板交互界面输入的操作信息之前还包括:In one embodiment, the detecting the operation information input to the control panel interaction interface further includes:
创建包括显示控件的人机交互界面,并将所述控制面板交互界面以视频或图像的形式渲染到所述人机交互界面的显示控件中。A human-computer interaction interface including a display control is created, and the control panel interaction interface is rendered into a display control of the human-computer interaction interface in the form of a video or an image.
在其中一个实施例中,所述投屏协议包括DLNA协议、Miracast协议和Airplay协议中的至少一种。In one of the embodiments, the projection screen protocol includes at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
在其中一个实施例中,所述检测针对所述控制面板交互界面输入的操作信息包括:In one embodiment, the detecting operation information input to the control panel interaction interface includes:
对所述控制面板交互界面进行触摸事件的监听,根据监听到的触摸事件获取输入的操作信息,所述操作信息包括触摸位置、滑动轨迹、触摸次数、触摸时长和按压力度中的至少一种。Performing monitoring of the touch event on the control panel interaction interface, and acquiring the input operation information according to the monitored touch event, the operation information including at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
实施本发明实施例,将具有如下有益效果:Implementation of the embodiments of the present invention will have the following beneficial effects:
上述远程设置智能设备的控制面板的方法及装置,使得用户在用手机、平板或笔记本电脑远程控制智能音箱、智能空调、智能冰箱等没有显示交互界面或显示交互界面较小的智能终端设备时,可通过接收智能音箱、智能空调、智能冰箱通过投屏协议发送的控制面板交互界面来可视化地向用户展示可调节的设置项,用户可通过在图形化的设置界面上完成对智能音箱的音量调节、智能空调的温度调节等操作,从而提高了操作的便利性。The method and device for remotely setting a control panel of a smart device, so that when a user remotely controls a smart speaker, a smart air conditioner, a smart refrigerator, or the like, which does not display an interactive interface or displays a smart terminal device having a small interactive interface, using a mobile phone, a tablet, or a notebook computer, The adjustable control items can be visually displayed to the user through the control panel interactive interface sent by the smart speaker, the smart air conditioner, and the smart refrigerator through the projection screen protocol, and the user can complete the volume adjustment of the smart speaker through the graphical setting interface. The operation of the temperature adjustment of the intelligent air conditioner improves the convenience of operation.
附图说明DRAWINGS
为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.
其中:among them:
图1是根据本发明一种远程设置智能设备的控制面板的方法的具体流程图;1 is a detailed flowchart of a method of remotely setting a control panel of a smart device according to the present invention;
图2是根据本发明一种远程设置智能设备的控制面板的方法的具体流程图;2 is a detailed flowchart of a method of remotely setting a control panel of a smart device according to the present invention;
图3是根据本发明一种远程设置智能设备的控制面板的方法的具体流程图;3 is a detailed flowchart of a method for remotely setting a control panel of a smart device according to the present invention;
图4是根据本发明一种终端设备的结构示意图;4 is a schematic structural diagram of a terminal device according to the present invention;
图5是根据本发明一种智能设备的结构示意图;FIG. 5 is a schematic structural diagram of a smart device according to the present invention; FIG.
图6为一个实施例中运行前述远程设置智能设备的控制面板的方法的计算机设备的结构示意图。FIG. 6 is a schematic structural diagram of a computer device for operating the foregoing method for remotely setting a control panel of a smart device in an embodiment.
具体实施方式detailed description
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
为了解决前述传统技术中使用终端设备对智能终端设备进行设置调节的操作过于繁琐的技术问题,在一个实施例中,特提出了远程设置智能设备的控制面板的方法,该方法的实现可依赖于计算机程序。用户通过在手机、平板电脑、掌上电脑、笔记本电脑、个人PC等终端设备上运行该计算机程序,可对与该终端设备处于同一局域网的智能设备(例如智能音箱、智能空调、智能冰箱等)的控制面板进行设置调节,操作更加方便简洁。In order to solve the technical problem that the operation of setting and adjusting the smart terminal device by using the terminal device is too cumbersome in the foregoing conventional technology, in one embodiment, a method for remotely setting a control panel of the smart device is proposed, and the implementation of the method may depend on Computer program. By running the computer program on a terminal device such as a mobile phone, a tablet computer, a palmtop computer, a notebook computer, or a personal PC, the user can be in a smart device (such as a smart speaker, a smart air conditioner, a smart refrigerator, etc.) that is on the same local area network as the terminal device. The control panel is adjusted and the operation is more convenient and concise.
具体的,如图1所示,该远程设置智能设备的控制面板的方法包括:Specifically, as shown in FIG. 1 , the method for remotely setting a control panel of the smart device includes:
步骤101:在局域网中搜索智能设备,在搜索到的智能设备中确定目标智能设备,与所述目标智能设备建立连接。Step 101: Search for a smart device in a local area network, determine a target smart device in the searched smart device, and establish a connection with the target smart device.
具体地说,终端设备可以向网络接入设备(如,无线路由器)发送设备查询请求。网络接入设备在接收到来自终端设备的设备查询请求时,获取当前与自身保持连接的各智能设备的信息,并反馈给终端设备。终端设备根据各智能设备的信息,将各智能设备设为目标智能设备,并与各智能设备建立连接。更具体地说,终端设备可以为手机、电脑、平板电脑等电子设备,智能设备可以为智能音箱、智能电视等视频播放设备。Specifically, the terminal device can send a device query request to a network access device (eg, a wireless router). When receiving the device query request from the terminal device, the network access device acquires information of each smart device that is currently connected to itself, and feeds back the information to the terminal device. The terminal device sets each smart device as a target smart device according to the information of each smart device, and establishes a connection with each smart device. More specifically, the terminal device may be an electronic device such as a mobile phone, a computer, or a tablet computer, and the smart device may be a video playback device such as a smart speaker or a smart TV.
步骤102,通过所述连接向所述目标智能设备发送控制开关指令,接收所述目标智能设备基于投屏协议反馈的所述目标智能设备的控制面板交互界面。Step 102: Send a control switch instruction to the target smart device by using the connection, and receive a control panel interaction interface of the target smart device that is fed back by the target smart device based on a projection screen protocol.
控制开关指令即为用户端操作的终端设备发起的期望对目标智能设备进行设置的开关指令,当用户端操作的终端设备向目标智能设备发送了控制开关指 令后,即用户端操作的终端设备期望对目标智能设备的设置进行调节。例如,音量调节、温度调节,功率调节等。The control switch command is a switch instruction initiated by the terminal device operated by the user terminal to set the target smart device. When the terminal device operated by the user terminal sends the control switch command to the target smart device, the terminal device operated by the user terminal expects Adjust the settings of the target smart device. For example, volume adjustment, temperature adjustment, power adjustment, and the like.
目标智能设备若同意用户端操作的终端设备对其进行设置,则可反馈相应设置的控制面板交互界面给用户端操作的终端设备。例如,对于智能音箱,可将音量调节的控制面板交互界面通过投屏协议发送给用户端操作的手机,然后在手机上展示,从而可让用户获知可对智能音箱进行调节的设置项。If the target smart device agrees to set the terminal device operated by the user terminal, the corresponding control panel interaction interface may be fed back to the terminal device operated by the user terminal. For example, for the smart speaker, the control panel interactive interface of the volume adjustment can be sent to the mobile phone operated by the user through the projection protocol, and then displayed on the mobile phone, so that the user can know the setting items that can adjust the smart speaker.
具体的,投屏协议可包括DLNA协议、Miracast协议和Airplay协议中的至少一种。Specifically, the projection screen protocol may include at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
步骤103,检测针对所述控制面板交互界面输入的操作信息。Step 103: Detect operation information input to the control panel interaction interface.
用户可以在终端设备上根据控制面板交互界面中所显示的内容,利用鼠标、键盘等外接设备进行操作,以便于终端设备根据外接设备输入的操作,检测用户输入的操作信息。The user can operate on the terminal device according to the content displayed in the interactive interface of the control panel by using an external device such as a mouse or a keyboard, so that the terminal device can detect the operation information input by the user according to the operation input by the external device.
更具体地说,当终端设备设有触控显示屏时,终端设备在检测针对控制面板交互界面输入的操作信息时,对所述控制面板交互界面进行触摸事件的监听,根据监听到的触摸事件获取输入的操作信息,以便于根据监听到的触摸事件获取操作信息。More specifically, when the terminal device is provided with the touch display screen, when the terminal device detects the operation information input to the control panel interaction interface, the terminal device performs the touch event monitoring on the control panel interaction interface, according to the monitored touch event. Get the input operation information, so as to obtain the operation information according to the monitored touch event.
具体的,所述操作信息包括触摸位置、滑动轨迹、触摸次数、触摸时长和按压力度中的至少一种。Specifically, the operation information includes at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
然而,上述举例仅为说明,本实施方式中并不对终端设备检测用户对映射到人机交互界面中的操作界面的操作信息的具体实现形式做任何限定。However, the foregoing examples are only illustrative. In this embodiment, the terminal device does not detect any specific implementation form of the operation information that the user maps to the operation interface in the human-machine interaction interface.
步骤104,将所述设置调节指令发送至所述目标智能设备,以使所述目标智能设备通过执行所述设置调节指令调节控制面板的配置。Step 104: Send the setting adjustment instruction to the target smart device, so that the target smart device adjusts the configuration of the control panel by executing the setting adjustment instruction.
具体地说,终端设备中预存有操作信息与设置调节指令的对应关系,终端设备在预存的对应关系中,查询并生成操作信息对应的设置调节指令。其中,设置调节指令可以为实时设置调节指令,也可以为延时设置调节指令。Specifically, the terminal device prestores a correspondence between the operation information and the setting adjustment command, and the terminal device queries and generates a setting adjustment instruction corresponding to the operation information in the pre-stored correspondence relationship. Among them, setting the adjustment command can set the adjustment instruction in real time, and can also set the adjustment instruction for the delay.
具体地说,终端设备在将生成的设置调节指令发送给智能设备时,可以先将设置调节指令以及智能设备的信息一起发送给网络接入设备,由网络接入设备转发给智能设备;也可以直接根据智能设备的信息,通过互联网将设置调节指令发送给智能设备。本实施方式中,终端设备采用TCP网络协议将设置调节指令发送至智能设备,事件的可靠性和连续性较高。Specifically, when the terminal device sends the generated setting adjustment command to the smart device, the setting adjustment command and the information of the smart device may be sent to the network access device together, and the network access device forwards the smart device to the smart device; The setting adjustment command is sent to the smart device via the Internet directly according to the information of the smart device. In this implementation manner, the terminal device sends the setting adjustment command to the smart device by using a TCP network protocol, and the reliability and continuity of the event are high.
与现有技术相比,本实施方式中,终端设备与智能设备之间的控制能够不基于外网服务器实现,提高了用户数据安全性和控制的可靠性。并且,终端设备能够在人机交互界面中显示控制面板交互界面,以供用户输入操作信息,实现了针对智能设备的可视化的实时控制或延时控制,操作简单便捷,且操作模式较为灵活。Compared with the prior art, in the implementation manner, the control between the terminal device and the smart device can be implemented not based on the external network server, and the user data security and the reliability of the control are improved. Moreover, the terminal device can display the control panel interaction interface in the human-computer interaction interface for the user to input the operation information, realize real-time control or delay control for the visualization of the smart device, the operation is simple and convenient, and the operation mode is flexible.
本发明的第二实施方式涉及一种远程设置智能设备的控制面板的方法,具体流程如图2所示。第二实施方式在第一实施方式的基础上加以改进,主要改进之处在于:在本发明第二实施方式中,提供了终端设备与处于同一局域网内的目标智能设备建立连接的一种具体实现形式,增加了本发明实施方式的灵活性。A second embodiment of the present invention relates to a method for remotely setting a control panel of a smart device. The specific process is shown in FIG. 2 . The second embodiment is improved on the basis of the first embodiment, and the main improvement is that in the second embodiment of the present invention, a specific implementation of establishing a connection between the terminal device and the target smart device in the same local area network is provided. Form, which increases the flexibility of embodiments of the invention.
本实施方式中步骤202至步骤204与第一实施方式中步骤102至步骤104大致相同,为减少重复,在此不再赘述,以下仅对不同部分进行说明: Steps 202 to 204 in this embodiment are substantially the same as steps 102 to 104 in the first embodiment. To reduce repetition, details are not described herein. Only the different parts are described below:
步骤201,与处于同一局域网内的目标智能设备建立连接。Step 201: Establish a connection with a target smart device in the same local area network.
本实施方式中,步骤201包括子步骤2011至子步骤2014,以下进行具体说明:In this embodiment, step 201 includes sub-steps 2011 to sub-steps 2014, which are specifically described below:
子步骤2011,通过局域网用户数据报UDP的广播包和组播包协议,搜索智能设备。Sub-step 2011, searching for the smart device through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
具体地说,终端设备和智能设备通过本地局域网交互,终端设备通过本地局域网的用户数据报协议UDP发送广播消息。其中,广播消息用于控制智能设备发出响应消息。响应消息中包括智能设备的设备信息。这样,终端设备能够通过本地局域网自动地检测到位于当前局域网中的所有的智能设备。Specifically, the terminal device and the smart device interact through a local area network, and the terminal device sends a broadcast message through a user datagram protocol UDP of the local area network. The broadcast message is used to control the smart device to send a response message. The response message includes device information of the smart device. In this way, the terminal device can automatically detect all the smart devices located in the current local area network through the local area network.
子步骤2012,将搜索到的智能设备作为处于同一局域网内的目标智能设备。Sub-step 2012, the searched smart device is used as a target smart device in the same local area network.
具体地说,终端设备通过监听TCP(Transmission Control Protocol,简称“TCP”)端口和UDP端口的方式,接收智能设备反馈的响应消息。而后,终端设备根据响应消息中携带的设备信息,设置目标智能设备。Specifically, the terminal device receives the response message fed back by the smart device by listening to a TCP (Transmission Control Protocol, "TCP") port and a UDP port. Then, the terminal device sets the target smart device according to the device information carried in the response message.
本实施方式中,终端设备在获取到智能设备的信息时,将智能设备的信息添加到本地列表中,更新本地列表,并对本地列表进行维护。具体地说,终端设备在将搜索到的智能设备的信息,添加到本地列表前,还检测本地列表中是否已存在智能设备的信息,若已存在智能设备的信息,则丢弃智能设备的信息码,若未存在智能设备的信息,则再执行将搜索到的智能设备的信息,添加到 本地列表。这样,终端设备不仅能够一次性添加局域网中所有的智能设备,达到快速添加和快速控制的目的,而且能够避免同一智能设备的信息被重复添加,易于本地列表的维护与更新。In this embodiment, when acquiring the information of the smart device, the terminal device adds the information of the smart device to the local list, updates the local list, and maintains the local list. Specifically, the terminal device detects the information of the smart device in the local list before adding the information of the searched smart device to the local list, and discards the information code of the smart device if the information of the smart device already exists. If the information of the smart device does not exist, the information of the searched smart device is added to the local list. In this way, the terminal device can not only add all the smart devices in the local area network at one time, but also achieve the purpose of rapid addition and fast control, and can prevent the information of the same smart device from being repeatedly added, and is easy to maintain and update the local list.
与第一实施方式相比,本实施方式中,终端设备能够自动地获取到位于当前局域网中的所有的智能设备,为用户在终端设备侧实现针对多台智能设备的控制提供了基础。Compared with the first embodiment, in the embodiment, the terminal device can automatically acquire all the smart devices located in the current local area network, and provides a basis for the user to implement control on multiple smart devices on the terminal device side.
本发明第三实施方式涉及一种远程设置智能设备的控制面板的方法。第三实施方式在第一实施方式的基础上加以改进,主要改进之处在于:在本发明第三实施方式中,提供了终端设备展示基于投屏协议的消息中的控制面板交互界面的一种具体实现形式,增加了本发明实施方式的灵活性。A third embodiment of the present invention relates to a method of remotely setting a control panel of a smart device. The third embodiment is improved on the basis of the first embodiment. The main improvement is that in the third embodiment of the present invention, a terminal device is provided to display a control panel interaction interface in a message based on a projection screen protocol. The specific implementation form increases the flexibility of the embodiment of the present invention.
具体地说,终端设备在与处于同一局域网内的目标智能设备建立连接后,终端设备向目标智能设备发送录制控制面板交互界面的请求,以便于目标智能设备录制控制面板交互界面,并生成数据包。而后,目标智能设备将数据包携带在基于投屏协议的消息中反馈给终端设备。其中,目标智能设备将基于投屏协议的消息发送给终端设备时,可以利用TCP网络协议或UDP网络协议进行传输。Specifically, after the terminal device establishes a connection with the target smart device in the same local area network, the terminal device sends a request for recording the control panel interaction interface to the target smart device, so that the target smart device records the control panel interaction interface, and generates a data packet. . Then, the target smart device carries the data packet to the terminal device in the message based on the projection screen protocol. The target smart device may transmit the message based on the screencast protocol to the terminal device by using a TCP network protocol or a UDP network protocol.
更具体地说,终端设备在接收到目标智能设备反馈的基于投屏协议的消息时,对基于投屏协议的消息中的数据包进行解析,得到控制面板交互界面。而后,终端设备创建包括显示控件的人机交互界面,并将所述控制面板交互界面以视频或图像的形式渲染到所述人机交互界面的显示控件中。如,携带控制面板交互界面的数据包中可以包括编码参数、视频帧及视频参数等,终端设备对数据包进行解析,获取数据包中的编码参数、视频帧及视频参数等,并将视频参数和视频帧送入解码器,使用系统提供的硬解码器进行解码,获取视频图像。而后,终端设备创建一个页面控件,将解码的视频图像渲染到页面控件上,从而实现控制面板交互界面在本地的显示。More specifically, when receiving the message based on the projection protocol fed back by the target smart device, the terminal device parses the data packet in the message based on the projection screen protocol to obtain a control panel interaction interface. Then, the terminal device creates a human-computer interaction interface including a display control, and renders the control panel interaction interface into a display control of the human-computer interaction interface in the form of a video or an image. For example, the data packet carrying the control panel interaction interface may include coding parameters, video frames, and video parameters, and the terminal device parses the data packet, obtains coding parameters, video frames, and video parameters in the data packet, and the video parameters. And the video frame is sent to the decoder, and is decoded by using a hard decoder provided by the system to obtain a video image. Then, the terminal device creates a page control, and renders the decoded video image onto the page control, thereby realizing the local display of the control panel interaction interface.
本实施方式中,终端设备在通过连接接收目标智能设备发送的基于投屏协议的消息时,通过监听TCP端口和UDP端口的方式,接收目标智能设备发送的基于投屏协议的消息,能够确保终端设备接收到基于投屏协议的消息。In this embodiment, when the terminal device receives the message based on the screencast protocol sent by the target smart device, the terminal device receives the message based on the screencast protocol sent by the target smart device by listening to the TCP port and the UDP port, thereby ensuring the terminal. The device receives a message based on the projection protocol.
与第一实施方式相比,本实施方式中,提供了操作简便、便于实现的展示基于投屏协议的消息中的控制面板交互界面的一种具体实现形式,增加了本发 明实施方式的灵活性。Compared with the first embodiment, in the present embodiment, a specific implementation form of the control panel interaction interface in the message based on the projection screen protocol is provided, which is simple and easy to implement, and the flexibility of the embodiment of the present invention is increased. .
本发明第四实施方式涉及一种远程设置智能设备的控制面板的方法,应用于智能设备,具体流程如图3所示。本实施方式的核心在于:将目标智能设备录制的控制面板交互界面展示到终端设备中,根据用户对控制面板交互界面输入的操作信息,在终端设备侧实现针对于智能设备的可视化的控制操作。下面对本实施方式的远程设置智能设备的控制面板的方法的实现细节进行具体的说明,以下内容仅为方便理解提供的实现细节,并非实施本方案的必须。A fourth embodiment of the present invention relates to a method for remotely setting a control panel of a smart device, which is applied to a smart device, and the specific process is shown in FIG. 3 . The core of the embodiment is that the control panel interaction interface recorded by the target smart device is displayed in the terminal device, and the control operation for visualizing the smart device is implemented on the terminal device side according to the operation information input by the user on the control panel interaction interface. The implementation details of the method for remotely setting the control panel of the smart device of the present embodiment are specifically described below. The following content is merely for facilitating understanding of the implementation details provided, and is not necessary for implementing the solution.
步骤301,与处于同一局域网内的终端设备建立连接。 Step 301, establishing a connection with a terminal device in the same local area network.
步骤302,在检测到来自终端设备的控制开关指令时,录制控制面板交互界面。Step 302: Record a control panel interaction interface when detecting a control switch command from the terminal device.
具体地说,目标智能设备可以设有实体显示屏,也可以不设有虚拟显示屏。更具体地说,目标智能设备可以使用系统显示管理器类的创建虚拟展示方法创建一个虚拟显示器,并把虚拟显示器上的内容渲染到页面控件上。其次,目标智能设备可以使用系统媒体编码器类进行硬编码,将页面控件作为其输入。最后,目标智能设备从编码器中拿出相关参数(如,编码参数、视频参数等)和编码后的视频帧,将相关参数及视频帧作为录制的控制面板交互界面进行封装。Specifically, the target smart device may or may not have a virtual display screen. More specifically, the target smart device can create a virtual display using the create virtual presentation method of the system display manager class and render the content on the virtual display onto the page control. Second, the target smart device can be hardcoded using the system media encoder class with the page control as its input. Finally, the target smart device takes relevant parameters (such as encoding parameters, video parameters, etc.) and the encoded video frames from the encoder, and encapsulates the relevant parameters and video frames as a recorded control panel interactive interface.
步骤303,将录制的控制面板交互界面通过投屏协议发送至终端设备。Step 303: The recorded control panel interaction interface is sent to the terminal device by using a projection screen protocol.
具体地说,目标智能设备将封装好的信息携带在基于投屏协议的消息中发送至终端设备。其中,目标智能设备可以利用TCP网络协议或UDP网络协议传输基于投屏协议的消息。Specifically, the target smart device carries the encapsulated information in a message based on a projection protocol to the terminal device. The target smart device may transmit a message based on a projection screen protocol by using a TCP network protocol or a UDP network protocol.
步骤304,接收终端设备发送的设置调节指令。Step 304: Receive a setting adjustment instruction sent by the terminal device.
具体地说,目标智能设备监听TCP端口,接收终端设备发送的设置调节指令,能够保证事件的可靠性和连续性。Specifically, the target smart device listens to the TCP port and receives the setting adjustment command sent by the terminal device, which can ensure the reliability and continuity of the event.
步骤305,执行与设置调节指令相对应的操作。 Step 305, performing an operation corresponding to setting the adjustment instruction.
具体地说,智能设备可以根据设置调节指令,使用插桩Instrumentation,以模拟终端设备上用户对人机交互界面中的控制面板交互界面的操作信息,实现终端设备侧针对智能设备的可视化的控制操作。Specifically, the smart device can use the instrumentation adjustment instruction according to the setting adjustment instruction to simulate the operation information of the user on the control panel interaction interface in the human-computer interaction interface on the terminal device, and realize the control operation of the terminal device side for the visualization of the smart device. .
与现有技术相比,本实施方式中,终端设备与智能设备之间的控制能够不基于外网服务器实现,提高了用户数据安全性和控制的可靠性。并且,终端设备能够在人机交互界面中显示控制面板交互界面,以供用户输入操作信息,实 现了针对智能设备的可视化的实时控制或延时控制,操作简单便捷,且操作模式较为灵活。Compared with the prior art, in the implementation manner, the control between the terminal device and the smart device can be implemented not based on the external network server, and the user data security and the reliability of the control are improved. Moreover, the terminal device can display the control panel interaction interface in the human-computer interaction interface for the user to input the operation information, realizing the real-time control or delay control for the visualization of the smart device, the operation is simple and convenient, and the operation mode is flexible.
上面各种方法的步骤划分,只是为了描述清楚,实现时可以合并为一个步骤或者对某些步骤进行拆分,分解为多个步骤,只要包括相同的逻辑关系,都在本专利的保护范围内;对算法中或者流程中添加无关紧要的修改或者引入无关紧要的设计,但不改变其算法和流程的核心设计都在该专利的保护范围内。The steps of the above various methods are divided for the sake of clear description. The implementation may be combined into one step or split into certain steps and decomposed into multiple steps. As long as the same logical relationship is included, it is within the protection scope of this patent. The addition of insignificant modifications to an algorithm or process, or the introduction of an insignificant design, without changing the core design of its algorithms and processes, is covered by this patent.
本发明第五实施方式涉及一种终端设备41,如图4所示,包括:智能设备连接模块411、接收模块412、展示模块413、检测模块414、指令生成模块415以及指令发送模块416,以下进行具体说明:A fifth embodiment of the present invention relates to a terminal device 41. As shown in FIG. 4, the device includes: a smart device connection module 411, a receiving module 412, a display module 413, a detection module 414, an instruction generation module 415, and an instruction sending module 416. Give specific instructions:
智能设备连接模块411,用于与处于同一局域网内的目标智能设备建立连接。The smart device connection module 411 is configured to establish a connection with a target smart device in the same local area network.
具体地说,智能设备连接模块411可以向网络接入设备(如,无线路由器)发送设备查询请求。网络接入设备在接收到来自终端设备的设备查询请求时,获取当前与自身保持连接的各智能设备的信息,并反馈给智能设备连接模块411。智能设备连接模块411根据各智能设备的信息,将各智能设备设为目标智能设备,并与各智能设备建立连接。Specifically, the smart device connection module 411 can send a device query request to a network access device (eg, a wireless router). When receiving the device query request from the terminal device, the network access device acquires the information of each smart device that is currently connected to itself, and feeds back the information to the smart device connection module 411. The smart device connection module 411 sets each smart device as a target smart device according to the information of each smart device, and establishes a connection with each smart device.
接收模块412,用于通过连接接收目标智能设备发送的基于投屏协议的消息。其中,基于投屏协议的消息中包括目标智能设备录制的控制面板交互界面。The receiving module 412 is configured to receive, by using a connection, a projection screen based message sent by the target smart device. The message based on the projection screen protocol includes a control panel interaction interface recorded by the target smart device.
展示模块413,用于展示基于投屏协议的消息中的控制面板交互界面。The display module 413 is configured to display a control panel interaction interface in the message based on the projection protocol.
具体地说,终端设备将控制面板交互界面投放在本地的人机交互界面中进行显示。Specifically, the terminal device displays the control panel interaction interface in a local human-computer interaction interface for display.
检测模块414,用于检测针对控制面板交互界面输入的操作信息。The detecting module 414 is configured to detect operation information input to the control panel interaction interface.
具体地说,用户可以根据控制面板交互界面中所显示的内容,利用鼠标、键盘等外接设备进行操作,以便于检测模块414根据外接设备输入的操作,检测用户输入的操作信息。Specifically, the user can operate by using an external device such as a mouse or a keyboard according to the content displayed in the interface of the control panel, so that the detecting module 414 detects the operation information input by the user according to the operation input by the external device.
更具体地说,当终端设备设有触控显示屏时,检测模块414可以包括监听子模块以及处理子模块。监听子模块,用于对控制面板交互界面进行触摸事件的监听;处理子模块,用于根据监听到的触摸事件获取输入的操作信息。其中,触摸事件可以为点击、按压、滑动等操作事件。More specifically, when the terminal device is provided with a touch display screen, the detecting module 414 can include a monitor sub-module and a processing sub-module. The monitoring sub-module is configured to perform monitoring of the touch event on the control panel interaction interface; and the processing sub-module is configured to obtain the input operation information according to the monitored touch event. The touch event may be an operation event such as clicking, pressing, and sliding.
然而,上述举例仅为说明,本实施方式中并不对检测模块414检测用户对映射到人机交互界面中的操作界面的操作信息的具体实现形式做任何限定。However, the above examples are merely illustrative. In the embodiment, the detection module 414 does not detect any specific implementation form of the user's operation information mapped to the operation interface in the human-computer interaction interface.
指令生成模块415,用于根据检测到的操作信息生成用于控制目标智能设备的设置调节指令。The instruction generation module 415 is configured to generate a setting adjustment instruction for controlling the target smart device according to the detected operation information.
具体地说,指令生成模块415中预存有操作信息与设置调节指令的对应关系,指令生成模块415在预存的对应关系中,查询并生成操作信息对应的设置调节指令。其中,设置调节指令可以为实时设置调节指令,也可以为延时设置调节指令。Specifically, the instruction generation module 415 prestores the correspondence between the operation information and the setting adjustment instruction, and the instruction generation module 415 queries and generates the setting adjustment instruction corresponding to the operation information in the pre-stored correspondence. Among them, setting the adjustment command can set the adjustment instruction in real time, and can also set the adjustment instruction for the delay.
指令发送模块416,用于将设置调节指令发送至目标智能设备,供目标智能设备执行与设置调节指令相对应的操作。The instruction sending module 416 is configured to send the setting adjustment instruction to the target smart device, where the target smart device performs an operation corresponding to the setting adjustment instruction.
具体地说,指令发送模块416在将生成的设置调节指令发送给智能设备时,可以先将设置调节指令以及智能设备的信息一起发送给网络接入设备,由网络接入设备转发给智能设备;指令发送模块416也可以直接根据智能设备的信息,通过互联网将设置调节指令发送给智能设备。本实施方式中,指令发送模块416采用TCP网络协议将设置调节指令发送至智能设备,事件的可靠性和连续性较高。Specifically, the command sending module 416 may send the set adjustment command and the information of the smart device to the network access device together, and then forward the information to the smart device by the network access device; The command sending module 416 can also directly send the setting adjustment command to the smart device via the Internet according to the information of the smart device. In this embodiment, the command sending module 416 sends the setting adjustment command to the smart device by using the TCP network protocol, and the reliability and continuity of the event are high.
不难发现,本实施方式为与第一实施方式相对应的装置实施例,本实施方式可与第一实施方式互相配合实施。第一实施方式中提到的相关技术细节在本实施方式中依然有效,为了减少重复,这里不再赘述。相应地,本实施方式中提到的相关技术细节也可应用在第一实施方式中。It is not difficult to find that the present embodiment is an apparatus embodiment corresponding to the first embodiment, and the present embodiment can be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
本发明第六实施方式涉及一种终端设备,第六实施方式在第五实施方式的基础上加以改进,主要改进之处在于:在本发明第六实施方式中,提供了智能设备连接模块411的一种具体实现形式,增加了本发明实施方式的灵活性。The sixth embodiment of the present invention relates to a terminal device. The sixth embodiment is improved on the basis of the fifth embodiment. The main improvement is that in the sixth embodiment of the present invention, the smart device connection module 411 is provided. A specific implementation form increases the flexibility of embodiments of the present invention.
具体地说,智能设备连接模块411包括:搜索子模块、指定子模块以及连接子模块。搜索子模块,用于通过局域网用户数据报UDP的广播包和组播包协议,搜索智能设备。指定子模块,用于将搜索到的智能设备作为处于同一局域网内的目标智能设备。连接子模块,用于与目标智能设备建立连接。Specifically, the smart device connection module 411 includes: a search submodule, a designation submodule, and a connection submodule. The search sub-module is used to search for smart devices through the broadcast packet and multicast packet protocol of the local area network user datagram UDP. Specifies a sub-module for using the discovered smart device as a target smart device within the same local area network. A connection submodule for establishing a connection with a target smart device.
本实施方式中,搜索子模块通过本地局域网的用户数据报协议UDP发送广播消息。其中,广播消息用于控制智能设备发出响应消息。响应消息中包括智能设备的设备信息。并且,搜索子模块通过监听TCP端口和UDP端口的方式,接收智能设备反馈的响应消息。而后,指定子模块根据响应消息中携带的设备信息,设置目标智能设备。In this embodiment, the search submodule sends a broadcast message through the User Datagram Protocol UDP of the local area network. The broadcast message is used to control the smart device to send a response message. The response message includes device information of the smart device. Moreover, the search submodule receives the response message fed back by the smart device by listening to the TCP port and the UDP port. Then, the designated sub-module sets the target smart device according to the device information carried in the response message.
不难发现,本实施方式为与第二实施方式相对应的装置实施例,本实施方式可与第二实施方式互相配合实施。第二实施方式中提到的相关技术细节在本实施方式中依然有效,为了减少重复,这里不再赘述。相应地,本实施方式中提到的相关技术细节也可应用在第二实施方式中。It is not difficult to find that the present embodiment is an apparatus embodiment corresponding to the second embodiment, and the present embodiment can be implemented in cooperation with the second embodiment. The related technical details mentioned in the second embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the second embodiment.
本发明第七实施方式涉及一种终端设备,第七实施方式在第五实施方式的基础上加以改进,主要改进之处在于:在本发明第七实施方式中,提供了展示模块413的一种具体实现形式,增加了本发明实施方式的灵活性。并且,本实施方式中,接收模块412在通过连接接收目标智能设备发送的基于投屏协议的消息时,通过监听TCP端口和UDP端口的方式,接收目标智能设备发送的基于投屏协议的消息,能够确保终端设备接收到基于投屏协议的消息。The seventh embodiment of the present invention relates to a terminal device, and the seventh embodiment is improved on the basis of the fifth embodiment. The main improvement is that in the seventh embodiment of the present invention, a display module 413 is provided. The specific implementation form increases the flexibility of the embodiment of the present invention. In the embodiment, the receiving module 412 receives the message based on the screencast protocol sent by the target smart device by listening to the TCP port and the UDP port when receiving the message based on the screencast protocol sent by the target smart device. It is possible to ensure that the terminal device receives a message based on the projection protocol.
具体地说,展示模块413包括:控件创建子模块以及渲染子模块。控件创建子模块,用于在本地的人机交互界面中创建显示控件。渲染子模块,用于将所述控制面板交互界面以视频图像形式渲染到所述显示控件中。Specifically, the presentation module 413 includes: a control creation sub-module and a rendering sub-module. The control creates a submodule for creating display controls in the local human interface. And a rendering submodule for rendering the control panel interaction interface into the display control in the form of a video image.
不难发现,本实施方式为与第三实施方式相对应的装置实施例,本实施方式可与第三实施方式互相配合实施。第三实施方式中提到的相关技术细节在本实施方式中依然有效,为了减少重复,这里不再赘述。相应地,本实施方式中提到的相关技术细节也可应用在第三实施方式中。It is not difficult to find that the present embodiment is an apparatus embodiment corresponding to the third embodiment, and the present embodiment can be implemented in cooperation with the third embodiment. The related technical details mentioned in the third embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the third embodiment.
本发明第八实施方式涉及一种智能设备42,如图5所示,包括:终端设备连接模块421、响应模块422、发送模块423、指令接收模块424、控制模块425。以下进行具体说明:The eighth embodiment of the present invention relates to a smart device 42. As shown in FIG. 5, the smart device 42 includes a terminal device connection module 421, a response module 422, a sending module 423, an instruction receiving module 424, and a control module 425. The following is a detailed description:
终端设备连接模块421,用于与处于同一局域网内的终端设备建立连接。The terminal device connection module 421 is configured to establish a connection with the terminal device in the same local area network.
响应模块422,用于在检测到来自终端设备的消息时,录制控制面板交互界面。The response module 422 is configured to record a control panel interaction interface when detecting a message from the terminal device.
具体地说,目标智能设备可以设有实体显示屏,也可以不设有虚拟显示屏。更具体地说,响应模块422可以使用系统显示管理器类的创建虚拟展示方法创建一个虚拟显示器,并把虚拟显示器上的内容渲染到页面控件上。其次,响应模块422可以使用系统媒体编码器类进行硬编码,将页面控件作为其输入。最后,响应模块422从编码器中拿出相关参数(如,编码参数、视频参数等)和编码后的视频帧,将相关参数及视频帧作为录制的控制面板交互界面进行封装。Specifically, the target smart device may or may not have a virtual display screen. More specifically, response module 422 can create a virtual display using the create virtual presentation method of the system display manager class and render the content on the virtual display onto the page control. Second, the response module 422 can be hard coded using the system media encoder class with the page control as its input. Finally, the response module 422 takes relevant parameters (eg, encoding parameters, video parameters, etc.) and the encoded video frames from the encoder, and encapsulates the relevant parameters and video frames as a recorded control panel interactive interface.
发送模块423,用于将录制的所述控制面板交互界面通过所述连接发送至所 述终端设备。The sending module 423 is configured to send the recorded control panel interaction interface to the terminal device through the connection.
具体地说,发送模块423将封装好的信息携带在基于投屏协议的消息中发送至终端设备。其中,发送模块423可以利用TCP网络协议或UDP网络协议传输基于投屏协议的消息。Specifically, the sending module 423 carries the encapsulated information in a message based on the projection protocol to the terminal device. The sending module 423 can transmit a message based on a projection screen protocol by using a TCP network protocol or a UDP network protocol.
指令接收模块424,用于接收所述终端设备发送的设置调节指令。The instruction receiving module 424 is configured to receive a setting adjustment instruction sent by the terminal device.
具体地说,指令接收模块424监听TCP端口,接收终端设备发送的设置调节指令,能够保证事件的可靠性和连续性。Specifically, the instruction receiving module 424 listens to the TCP port and receives the setting adjustment command sent by the terminal device, which can ensure the reliability and continuity of the event.
控制模块425,用于执行与所述设置调节指令相对应的操作。The control module 425 is configured to perform an operation corresponding to the setting adjustment instruction.
具体地说,控制模块425可以根据设置调节指令,使用插桩Instrumentation,以模拟终端设备上用户对人机交互界面中的控制面板交互界面的操作信息,实现终端设备侧针对智能设备的可视化的控制操作。Specifically, the control module 425 can use the instrumentation adjustment instruction to simulate the operation information of the user on the control panel interaction interface in the human-computer interaction interface on the terminal device, and realize the control of the visualization of the smart device on the terminal device side. operating.
不难发现,本实施方式为与第四实施方式相对应的装置实施例,本实施方式可与第四实施方式互相配合实施。第四实施方式中提到的相关技术细节在本实施方式中依然有效,为了减少重复,这里不再赘述。相应地,本实施方式中提到的相关技术细节也可应用在第一实施方式中。It is not difficult to find that the present embodiment is an apparatus embodiment corresponding to the fourth embodiment, and the present embodiment can be implemented in cooperation with the fourth embodiment. The related technical details mentioned in the fourth embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
值得一提的是,本实施方式中所涉及到的各模块均为逻辑模块,在实际应用中,一个逻辑单元可以是一个物理单元,也可以是一个物理单元的一部分,还可以以多个物理单元的组合实现。此外,为了突出本发明的创新部分,本实施方式中并没有将与解决本发明所提出的技术问题关系不太密切的单元引入,但这并不表明本实施方式中不存在其它的单元。It is worth mentioning that each module involved in this embodiment is a logic module. In practical applications, a logical unit may be a physical unit, a part of a physical unit, or multiple physical entities. A combination of units is implemented. In addition, in order to highlight the innovative part of the present invention, the present embodiment does not introduce a unit that is not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in the present embodiment.
本发明第九实施方式涉及一种智能设备的远程控制系统,包括:第五实施方式至第七实施方式中任一中提及到的终端设备,以及第八实施方式中提及到的智能设备。A ninth embodiment of the present invention relates to a remote control system for a smart device, comprising: the terminal device mentioned in any of the fifth to seventh embodiments, and the smart device mentioned in the eighth embodiment .
即,本领域技术人员可以理解,实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。That is, those skilled in the art can understand that all or part of the steps in implementing the foregoing embodiment methods can be completed by a program instructing related hardware, and the program is stored in a storage medium, and includes a plurality of instructions for making a device ( The method may be a microcontroller, a chip, etc. or a processor to perform all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .
本领域的普通技术人员可以理解,上述各实施方式是实现本发明的具体实 施例,而在实际应用中,可以在形式上和细节上对其作各种改变,而不偏离本发明的精神和范围。A person skilled in the art can understand that the above embodiments are specific embodiments for implementing the present invention, and various changes can be made in the form and details without departing from the spirit and scope of the present invention. range.
实施本发明实施例,将具有如下有益效果:Implementation of the embodiments of the present invention will have the following beneficial effects:
上述远程设置智能设备的控制面板的方法及装置,使得用户在用手机、平板或笔记本电脑远程控制智能音箱、智能空调、智能冰箱等没有显示交互界面或显示交互界面较小的智能终端设备时,可通过接收智能音箱、智能空调、智能冰箱通过投屏协议发送的控制面板交互界面来可视化地向用户展示可调节的设置项,用户可通过在图形化的设置界面上完成对智能音箱的音量调节、智能空调的温度调节等操作,从而提高了操作的便利性。The method and device for remotely setting a control panel of a smart device, so that when a user remotely controls a smart speaker, a smart air conditioner, a smart refrigerator, or the like, which does not display an interactive interface or displays a smart terminal device having a small interactive interface, using a mobile phone, a tablet, or a notebook computer, The adjustable control items can be visually displayed to the user through the control panel interactive interface sent by the smart speaker, the smart air conditioner, and the smart refrigerator through the projection screen protocol, and the user can complete the volume adjustment of the smart speaker through the graphical setting interface. The operation of the temperature adjustment of the intelligent air conditioner improves the convenience of operation.
在一个实施例中,如图6所示,图6展示了一种运行上述远程设置智能设备的控制面板的方法的基于冯诺依曼体系的计算机系统10的服务器设备。该计算机系统可以是直接执行上述内容投放方法的服务器、服务器集群或者提供虚拟机作为服务器程序运行环境执行上述内容投放方法的服务器集群设备。具体的,可包括通过系统总线连接的网络接口1001、处理器1002、存储器1003。存储器1003可包括外存储器10032(例如硬盘、光盘或软盘等)和内存储器10034。In one embodiment, as shown in FIG. 6, FIG. 6 illustrates a server device of a von Neumann system-based computer system 10 that operates the method of remotely setting a control panel of a smart device as described above. The computer system may be a server, a server cluster, or a server cluster device that provides a virtual machine as a server program execution environment to execute the above content delivery method. Specifically, the network interface 1001, the processor 1002, and the memory 1003 connected through a system bus may be included. The memory 1003 may include an external memory 10032 (eg, a hard disk, an optical disk, or a floppy disk, etc.) and an internal memory 10034.
在本实施例中,本方法的运行基于计算机程序,该计算机程序的程序文件存储于前述基于冯诺依曼体系的计算机系统10的外存储器10032中,在运行时被加载到内存储器10034中,然后被编译为机器码之后传递至处理器1002中执行,从而使得基于冯诺依曼体系的计算机系统10中形成逻辑上的初始化模块102、好友查找模块104、内容传播评价值更新模块106、内容数据推送模块108。且在上述内容投放方法执行过程中,输入的参数均通过网络接口1001接收,并传递至存储器1003中缓存,然后输入到处理器1002中进行处理,处理的结果数据或缓存于存储器1003中进行后续地处理,或被传递至网络接口1001进行输出。In this embodiment, the operation of the method is based on a computer program, the program file of which is stored in the external memory 10032 of the aforementioned von Neumann system-based computer system 10, and is loaded into the internal memory 10034 at runtime, It is then compiled into a machine code and passed to the processor 1002 for execution, so that the logical initialization module 102, the friend lookup module 104, the content propagation evaluation value update module 106, and the content are formed in the von Neumann system-based computer system 10. Data push module 108. During the execution of the content delivery method, the input parameters are received by the network interface 1001, and are transferred to the cache in the memory 1003, and then input to the processor 1002 for processing, and the processed result data is cached in the memory 1003 for subsequent processing. It is processed or passed to the network interface 1001 for output.
以上所揭露的仅为本发明较佳实施例而已,当然不能以此来限定本发明之权利范围,因此依本发明权利要求所作的等同变化,仍属本发明所涵盖的范围。The above is only the preferred embodiment of the present invention, and the scope of the present invention is not limited thereto, and thus equivalent changes made in the claims of the present invention are still within the scope of the present invention.

Claims (10)

  1. 一种远程设置智能设备的控制面板的方法,其特征在于,应用于终端设备,包括:A method for remotely setting a control panel of a smart device, which is characterized by being applied to a terminal device, including:
    在局域网中搜索智能设备,在搜索到的智能设备中确定目标智能设备,与所述目标智能设备建立连接;Searching for a smart device in a local area network, determining a target smart device in the searched smart device, and establishing a connection with the target smart device;
    通过所述连接向所述目标智能设备发送控制开关指令,接收所述目标智能设备基于投屏协议反馈的所述目标智能设备的控制面板交互界面;Sending, by the connection, a control switch instruction to the target smart device, and receiving a control panel interaction interface of the target smart device based on the projection protocol feedback;
    检测针对所述控制面板交互界面输入的操作信息;Detecting operation information input to the control panel interaction interface;
    根据所述操作信息生成用于控制所述目标智能设备的设置调节指令;Generating a setting adjustment instruction for controlling the target smart device according to the operation information;
    将所述设置调节指令发送至所述目标智能设备,以使所述目标智能设备通过执行所述设置调节指令调节控制面板的配置。And transmitting the setting adjustment instruction to the target smart device, so that the target smart device adjusts a configuration of the control panel by executing the setting adjustment instruction.
  2. 根据权利要求1所述的远程设置智能设备的控制面板的方法,其特征在于,所述在局域网中搜索智能设备包括:The method for remotely setting a control panel of a smart device according to claim 1, wherein the searching for a smart device in a local area network comprises:
    通过局域网用户数据报UDP的广播包和组播包协议搜索智能设备。The smart device is searched through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
  3. 根据权利要求1或2所述的远程设置智能设备的控制面板的方法,其特征在于,所述检测针对所述控制面板交互界面输入的操作信息之前还包括:The method for remotely setting a control panel of a smart device according to claim 1 or 2, wherein before the detecting the operation information input to the control panel interaction interface, the method further comprises:
    创建包括显示控件的人机交互界面,并将所述控制面板交互界面以视频或图像的形式渲染到所述人机交互界面的显示控件中。A human-computer interaction interface including a display control is created, and the control panel interaction interface is rendered into a display control of the human-computer interaction interface in the form of a video or an image.
  4. 根据权利要求3所述的远程设置智能设备的控制面板的方法,其特征在于,所述投屏协议包括DLNA协议、Miracast协议和Airplay协议中的至少一种。The method of remotely setting a control panel of a smart device according to claim 3, wherein the projection screen protocol comprises at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
  5. 根据权利要求1或2所述的远程设置智能设备的控制面板的方法,其特征在于,所述检测针对所述控制面板交互界面输入的操作信息包括:The method for remotely setting a control panel of a smart device according to claim 1 or 2, wherein the detecting operation information input to the control panel interaction interface comprises:
    对所述控制面板交互界面进行触摸事件的监听,根据监听到的触摸事件获取输入的操作信息,所述操作信息包括触摸位置、滑动轨迹、触摸次数、触摸时长和按压力度中的至少一种。Performing monitoring of the touch event on the control panel interaction interface, and acquiring the input operation information according to the monitored touch event, the operation information including at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
  6. 一种远程设置智能设备的控制面板的装置,其特征在于,应用于终端设备,包括:An apparatus for remotely setting a control panel of a smart device, which is characterized by being applied to a terminal device, including:
    在局域网中搜索智能设备,在搜索到的智能设备中确定目标智能设备,与 所述目标智能设备建立连接;Searching for a smart device in a local area network, determining a target smart device in the searched smart device, and establishing a connection with the target smart device;
    通过所述连接向所述目标智能设备发送控制开关指令,接收所述目标智能设备基于投屏协议反馈的所述目标智能设备的控制面板交互界面;Sending, by the connection, a control switch instruction to the target smart device, and receiving a control panel interaction interface of the target smart device based on the projection protocol feedback;
    检测针对所述控制面板交互界面输入的操作信息;Detecting operation information input to the control panel interaction interface;
    根据所述操作信息生成用于控制所述目标智能设备的设置调节指令;Generating a setting adjustment instruction for controlling the target smart device according to the operation information;
    将所述设置调节指令发送至所述目标智能设备,以使所述目标智能设备通过执行所述设置调节指令调节控制面板的配置。And transmitting the setting adjustment instruction to the target smart device, so that the target smart device adjusts a configuration of the control panel by executing the setting adjustment instruction.
  7. 根据权利要求6所述的远程设置智能设备的控制面板的装置,其特征在于,所述在局域网中搜索智能设备包括:The device for remotely setting a control panel of a smart device according to claim 6, wherein the searching for the smart device in the local area network comprises:
    通过局域网用户数据报UDP的广播包和组播包协议搜索智能设备。The smart device is searched through the broadcast packet and multicast packet protocol of the local area network user datagram UDP.
  8. 根据权利要求6或7所述的远程设置智能设备的控制面板的装置,其特征在于,所述检测针对所述控制面板交互界面输入的操作信息之前还包括:The apparatus for remotely setting a control panel of a smart device according to claim 6 or 7, wherein the detecting the operation information input to the control panel interaction interface further comprises:
    创建包括显示控件的人机交互界面,并将所述控制面板交互界面以视频或图像的形式渲染到所述人机交互界面的显示控件中。A human-computer interaction interface including a display control is created, and the control panel interaction interface is rendered into a display control of the human-computer interaction interface in the form of a video or an image.
  9. 根据权利要求8所述的远程设置智能设备的控制面板的装置,其特征在于,所述投屏协议包括DLNA协议、Miracast协议和Airplay协议中的至少一种。The apparatus for remotely setting a control panel of a smart device according to claim 8, wherein the projection screen protocol comprises at least one of a DLNA protocol, a Miracast protocol, and an Airplay protocol.
  10. 根据权利要求6或7所述的远程设置智能设备的控制面板的装置,其特征在于,所述检测针对所述控制面板交互界面输入的操作信息包括:The apparatus for remotely setting a control panel of a smart device according to claim 6 or 7, wherein the detecting operation information input to the control panel interaction interface comprises:
    对所述控制面板交互界面进行触摸事件的监听,根据监听到的触摸事件获取输入的操作信息,所述操作信息包括触摸位置、滑动轨迹、触摸次数、触摸时长和按压力度中的至少一种。Performing monitoring of the touch event on the control panel interaction interface, and acquiring the input operation information according to the monitored touch event, the operation information including at least one of a touch position, a sliding track, a touch number, a touch duration, and a pressing force.
PCT/CN2018/126030 2018-03-29 2018-12-31 Method and apparatus for remotely configuring control panel of intelligent device WO2019184533A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810275033.5 2018-03-29
CN201810275033.5A CN108667900A (en) 2018-03-29 2018-03-29 The method and device of the control panel of remotely located smart machine

Publications (1)

Publication Number Publication Date
WO2019184533A1 true WO2019184533A1 (en) 2019-10-03

Family

ID=63782620

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/126030 WO2019184533A1 (en) 2018-03-29 2018-12-31 Method and apparatus for remotely configuring control panel of intelligent device

Country Status (2)

Country Link
CN (1) CN108667900A (en)
WO (1) WO2019184533A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112000410A (en) * 2020-08-17 2020-11-27 努比亚技术有限公司 Screen projection control method and device and computer readable storage medium
CN114115775A (en) * 2021-11-12 2022-03-01 广州视源电子科技股份有限公司 Information screen projection method and device, terminal device, display device and storage medium

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108667900A (en) * 2018-03-29 2018-10-16 深圳市九洲电器有限公司 The method and device of the control panel of remotely located smart machine
CN110083324A (en) * 2019-04-30 2019-08-02 华为技术有限公司 Method, apparatus, electronic equipment and the computer storage medium of Image Rendering
CN113014614A (en) * 2019-12-20 2021-06-22 青岛海信移动通信技术股份有限公司 Equipment control method, control equipment and controlled equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014030929A1 (en) * 2012-08-22 2014-02-27 에스케이텔레콤 주식회사 Apparatus for providing user interface for sharing media contents in home network and recording medium for recording programs
CN104219551A (en) * 2014-09-29 2014-12-17 武汉诚迈科技有限公司 Method for screen sharing and remote control of intelligent device
CN105099517A (en) * 2015-06-24 2015-11-25 小米科技有限责任公司 Connection establishing method and device
CN105160854A (en) * 2015-09-16 2015-12-16 小米科技有限责任公司 Equipment control method, device and terminal equipment
CN106161151A (en) * 2016-06-12 2016-11-23 青岛海信移动通信技术股份有限公司 The control method of a kind of Smart Home and equipment
CN108667900A (en) * 2018-03-29 2018-10-16 深圳市九洲电器有限公司 The method and device of the control panel of remotely located smart machine

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014030929A1 (en) * 2012-08-22 2014-02-27 에스케이텔레콤 주식회사 Apparatus for providing user interface for sharing media contents in home network and recording medium for recording programs
CN104219551A (en) * 2014-09-29 2014-12-17 武汉诚迈科技有限公司 Method for screen sharing and remote control of intelligent device
CN105099517A (en) * 2015-06-24 2015-11-25 小米科技有限责任公司 Connection establishing method and device
CN105160854A (en) * 2015-09-16 2015-12-16 小米科技有限责任公司 Equipment control method, device and terminal equipment
CN106161151A (en) * 2016-06-12 2016-11-23 青岛海信移动通信技术股份有限公司 The control method of a kind of Smart Home and equipment
CN108667900A (en) * 2018-03-29 2018-10-16 深圳市九洲电器有限公司 The method and device of the control panel of remotely located smart machine

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112000410A (en) * 2020-08-17 2020-11-27 努比亚技术有限公司 Screen projection control method and device and computer readable storage medium
CN112000410B (en) * 2020-08-17 2024-03-19 努比亚技术有限公司 Screen projection control method, device and computer readable storage medium
CN114115775A (en) * 2021-11-12 2022-03-01 广州视源电子科技股份有限公司 Information screen projection method and device, terminal device, display device and storage medium

Also Published As

Publication number Publication date
CN108667900A (en) 2018-10-16

Similar Documents

Publication Publication Date Title
WO2019184533A1 (en) Method and apparatus for remotely configuring control panel of intelligent device
WO2021135160A1 (en) Gaming live stream control method and device, computer storage medium, and electronic apparatus
JP7239544B2 (en) Voice wake-up method and apparatus
US20200252571A1 (en) Systems, methods, and media for presenting media content
US11057500B2 (en) Publication of applications using server-side virtual screen change capture
WO2020168571A1 (en) Device control method, apparatus, system, electronic device and cloud server
EP3046331B1 (en) Media control method and system based on cloud desktop
US20190281327A1 (en) Method For Controlling Delivery Of A Video Stream Of A Live-Stream Room, and Corresponding Server and Mobile Terminal
WO2022188595A1 (en) Method and apparatus for displaying application picture, and terminal, screen projection system and medium
WO2017148294A1 (en) Mobile terminal-based apparatus control method, device, and mobile terminal
WO2019090902A1 (en) Screen sharing method and apparatus, electronic device, and storage medium
WO2020042350A1 (en) Desktop screen projection method and device, apparatus, and storage medium
CN203120046U (en) Visualization control system based on video conference terminal
CN111459433A (en) Screen transmission method, equipment and storage medium
CN113286191B (en) Content collaboration method, device, electronic equipment and storage medium
KR101650804B1 (en) Method for sharing media content, terminal device, and content sharing system
AU2010260683B2 (en) Apparatus and method for transmitting and receiving a user interface in a communication system
CN114007125A (en) Volume control method, mobile terminal, target device and storage medium
WO2024045985A1 (en) Screen control method, screen control apparatus, electronic device, program, and medium
US20230333803A1 (en) Enhanced Screen Sharing Method and System, and Electronic Device
TW201730739A (en) Information interaction method and device
WO2022133495A1 (en) Systems and methods for audience interactions in real-time multimedia applications
US10764535B1 (en) Facial tracking during video calls using remote control input
US20210392394A1 (en) Method and apparatus for processing video, electronic device and storage medium
CN114095791A (en) Live broadcast playback method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18911493

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.01.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18911493

Country of ref document: EP

Kind code of ref document: A1