WO2017148294A1 - 基于移动终端的设备控制方法、装置和移动终端 - Google Patents

基于移动终端的设备控制方法、装置和移动终端 Download PDF

Info

Publication number
WO2017148294A1
WO2017148294A1 PCT/CN2017/074168 CN2017074168W WO2017148294A1 WO 2017148294 A1 WO2017148294 A1 WO 2017148294A1 CN 2017074168 W CN2017074168 W CN 2017074168W WO 2017148294 A1 WO2017148294 A1 WO 2017148294A1
Authority
WO
WIPO (PCT)
Prior art keywords
controlled device
control
mobile terminal
instruction
screen
Prior art date
Application number
PCT/CN2017/074168
Other languages
English (en)
French (fr)
Inventor
姚迪狄
张磊
杜武平
Original Assignee
阿里巴巴集团控股有限公司
姚迪狄
张磊
杜武平
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 阿里巴巴集团控股有限公司, 姚迪狄, 张磊, 杜武平 filed Critical 阿里巴巴集团控股有限公司
Publication of WO2017148294A1 publication Critical patent/WO2017148294A1/zh
Priority to US16/119,730 priority Critical patent/US20180375987A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H04M1/72415—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/42208—Display device provided on the remote control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/4222—Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W76/00—Connection management
    • H04W76/10—Connection setup
    • H04W76/14—Direct-mode setup
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00—Transmission systems of control signals via wireless link

Definitions

  • the present application relates to the field of communications technologies, and in particular, to a device control method, apparatus, and mobile terminal based on a mobile terminal.
  • Smart devices have become more and more popular in people's daily lives. Users can use audio-based set-top boxes, smart TVs, personal computers, projectors and other devices to perform various activities such as audio, video, entertainment, and shopping. Most of these smart devices are controlled by a remote controller or a mouse, and the operation mode is single, and the operation is inconvenient, especially in the scenes of inputting Chinese characters, button operations or playing games, the operation is more inconvenient.
  • the present application aims to solve the above technical problems at least to some extent.
  • the first object of the present application is to provide a device control method based on a mobile terminal, which realizes complementary advantages between devices, and greatly improves user interaction freedom and flexibility.
  • a second object of the present application is to propose a device control device based on a mobile terminal.
  • a third object of the present application is to propose a mobile terminal.
  • a device control method based on a mobile terminal includes the following steps: establishing a communication connection with the controlled device; acquiring the screen information currently displayed by the controlled device, and Generating a projection screen according to the screen information, and performing display; receiving a control operation of the user for the projection screen; converting the control operation into an instruction recognizable by the controlled device, and transmitting the instruction to the controlled device, Controlling the controlled device.
  • the device control method based on the mobile terminal of the embodiment of the present application, by using the controlled device
  • the previously displayed screen information is projected to the mobile terminal, and the control operation of the projection screen for the user is converted into an instruction recognizable by the controlled device and sent to the controlled device to control the controlled device, thereby realizing the mobile terminal and the controlled device.
  • the cooperative operation control makes full use of the advantages of the controlled display experience of the controlled device and the natural and convenient operation of the mobile terminal, and converts the diverse, natural and convenient operation of the mobile terminal into an instruction recognizable by the controlled device for control, thereby operating a single operation.
  • the controlled device can perform various diversified collaborative control through the mobile terminal, which realizes the complementary advantages between the devices, and greatly improves the user interaction freedom and flexibility while giving the user a WYSIWYG visual operation experience. Sex.
  • the second aspect of the present application provides a device control device based on a mobile terminal, including: a communication module, configured to establish a communication connection with the controlled device; and an acquiring module, configured to acquire the screen information currently displayed by the controlled device a generating module, configured to generate a projection screen according to the screen information, and perform display; a first receiving module, configured to receive a control operation of the user for the projection screen; and a conversion module, configured to convert the control operation into a An instruction recognizable by the controlled device; a sending module, configured to send an instruction recognizable by the controlled device to the controlled device to control the controlled device.
  • the device control device based on the mobile terminal of the embodiment of the present application, by projecting the screen information currently displayed in the controlled device to the mobile terminal, and converting the control operation of the user for the projection screen into an instruction recognizable by the controlled device, is sent to the
  • the control device controls the controlled device to realize the coordinated operation control between the mobile terminal and the controlled device, and fully utilizes the advantage of the controlled device display experience and the natural terminal convenience of the mobile terminal, and the mobile terminal is diverse, natural and convenient.
  • the operation is converted into an instruction recognizable by the controlled device for control, so that a single controlled device can perform various diversified coordinated control through the mobile terminal, thereby realizing the complementary advantages between the devices, and bringing the user to see That is to say, the resulting visual operation experience greatly improves the freedom and flexibility of user interaction.
  • a third aspect of the present application provides a mobile terminal, including a mobile terminal-based device control apparatus according to an embodiment of the second aspect of the present application.
  • the mobile terminal of the embodiment of the present application by projecting the screen information currently displayed in the controlled device to the mobile terminal, and converting the control operation of the user for the projection screen into an instruction recognizable by the controlled device, is sent to the controlled device, so as to
  • the controlled device controls the mobile terminal and is controlled
  • the cooperative operation control of the equipment fully utilizes the advantages of the controlled display experience of the controlled device and the natural and convenient operation of the mobile terminal, and converts the diverse, natural and convenient operation of the mobile terminal into an instruction recognizable by the controlled device for control, thereby operating
  • a single controlled device can perform various diversified collaborative control through the mobile terminal, which realizes the complementary advantages between the devices, and greatly improves the user interaction freedom while giving the user a WYSIWYG visual operation experience. flexibility.
  • FIG. 1 is a flowchart of a mobile terminal-based device control method according to an embodiment of the present application
  • FIG. 2 is a flowchart of a device control method based on a mobile terminal according to another embodiment of the present application
  • FIG. 3 is a schematic diagram of instruction conversion according to an embodiment of the present application.
  • FIG. 4 is a flowchart of a mobile terminal-based device control method according to another embodiment of the present application.
  • FIG. 5 is a schematic diagram of a projection screen according to an embodiment of the present application.
  • FIG. 6 is a flowchart of a mobile terminal-based device control method according to another embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to another embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to another embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to another embodiment of the present application.
  • the mobile terminal Since the mobile terminal has natural and diverse user interaction modes, such as touch, voice, and body sense, it is easy to control. Therefore, in order to solve the current smart set-top box, smart TV, personal computer, projector, etc., the device has a large screen.
  • the method of displaying or playing, and being easy to view, but having a single operation mode and inconvenient operation, the embodiment of the present application provides a device control method and device based on a mobile terminal, which can combine the advantages of the above two types of devices. Achieve a combination of large screen display + natural control advantages to enhance the interactive experience when users use.
  • the mobile terminal in the embodiment of the present application may be a terminal device such as a mobile phone, a tablet computer, a smart wearable device, or a personal digital assistant.
  • FIG. 1 is a flowchart of a mobile terminal based device control method according to an embodiment of the present application. As shown in FIG. 1 , a mobile terminal-based device control method according to an embodiment of the present application includes:
  • the controlled device may be a smart TV, a television set top box, a personal computer or a projector, or the like.
  • the mobile terminal and the controlled device can pass, but are not limited to, Bluetooth, Wi-Fi (Wireless Fidelity), 2.4G (a wireless communication technology with a frequency band between 2.400 GHz and 2.4835 GHz), iBeacon (a Bluetooth-based wireless communication released by Apple A communication method such as a communication technology establishes a communication connection or establishes a communication connection by a wired method.
  • Wi-Fi Wireless Fidelity
  • 2.4G a wireless communication technology with a frequency band between 2.400 GHz and 2.4835 GHz
  • iBeacon a Bluetooth-based wireless communication released by Apple
  • a communication method such as a communication technology establishes a communication connection or establishes a communication connection by a wired method.
  • the mobile terminal can obtain the screen information currently displayed by the controlled device in a plurality of different manners.
  • the following two methods are exemplified.
  • the controlled device pushes the displayed picture to the mobile terminal in real time.
  • the controlled device provides the data source address of the displayed screen to the mobile terminal, and the mobile terminal can acquire the screen displayed in the controlled device from the data source address synchronously with the controlled device.
  • the mobile terminal may resize the screen information to adapt to the display size in the mobile terminal.
  • generating the projection screen according to the screen information may include: resizing the screen information according to a size of the preset display area to generate the projection screen.
  • the size of the preset display area may be determined according to the actual display area of the mobile terminal. For example, if it is a full screen display, the size of the preset display area may be the screen size of the mobile terminal, if it is displayed in a certain window. , the size of the preset display area may be the size of the window.
  • the mobile terminal When the user needs to operate on the screen information displayed by the controlled device, the mobile terminal can be directly operated according to the projection screen displayed in the mobile terminal.
  • the mobile terminal has various and abundant sensors and devices, such as a touch screen, a camera, a voice input device, a gravity sensor, a gyroscope, an accelerometer, and the like. Therefore, the mobile terminal can receive different forms of control operations.
  • the foregoing control operation may include at least one of the following:
  • the user can input the control operation by clicking an operation control on the projection screen, or inputting a voice instruction, or by controlling the mobile terminal to shake left and right in the space.
  • the device can be queried according to the identification information of the controlled device (such as the device serial number, etc.) or the identifier of the application (such as the application package name, name, and the like) in the controlled device that displays the screen information. instruction.
  • the identification information of the controlled device such as the device serial number, etc.
  • the identifier of the application such as the application package name, name, and the like
  • the converting the control operation into an instruction recognizable by the controlled device may include steps S201-S203 as shown in FIG. 2.
  • the touch operation can be recognized as an operation instruction to the corresponding operation control. For example, if the control operation is to touch the third control button A in the projected screen, it can be recognized as "click operation on the control button A".
  • control operation is a voice operation
  • the voice input by the user can be voice-recognized to obtain an operation instruction for the corresponding control.
  • control operation is a somatosensory operation or a gesture operation
  • the somatosensory operation or the gesture operation can be recognized as a key operation on the corresponding button.
  • the instruction mapping relationship corresponding to the controlled device may be queried according to the identification information of the controlled device (such as the device serial number, etc.). Or, according to the identifier of the application (such as the application package name, name, and the like) of the application device that displays the screen information in the controlled device, the instruction mapping relationship corresponding thereto may be queried.
  • the identification information of the controlled device such as the device serial number, etc.
  • the identifier of the application such as the application package name, name, and the like
  • the instruction mapping relationship is a mapping relationship between an instruction in the mobile terminal and an instruction recognizable by the controlled device. Therefore, based on the instruction mapping relationship, an instruction recognizable by the controlled device corresponding to the identified instruction can be queried.
  • determining the instruction mapping relationship corresponding to the screen information may include: performing image recognition on the screen information to obtain an identifier of an application that generates the screen information in the controlled device; The preset database is queried according to the identifier of the application to obtain a corresponding instruction mapping relationship.
  • the identifier of the currently displayed application may be obtained from the system of the controlled device through a preset interface function; or may be controlled by The screen information currently displayed by the device performs image recognition to obtain text information or an icon in the currently displayed screen information, and determines a corresponding application from the acquired text information or icon.
  • the instruction mapping relationship corresponding to different applications is stored in the preset database, and may be stored locally in the mobile terminal or in the server.
  • the above operation instruction can be parsed, mapped, and converted according to the instruction mapping relationship.
  • the following are three examples of converting the operational instructions into instructions identifiable by the controlled device in accordance with the instruction mapping relationship.
  • Example 1 (as shown in Figure 3)
  • TV (television) control mode no touch response, that is, can not respond to touch operation;
  • Operation command received by Phone touch the third control button
  • the command is parsed as: move, click;
  • the instruction is mapped to: first position the current position of the control focus (for the first control button), then move the control focus from the current position to the third control button, and then click the third control button;
  • the command is converted to: right button (direction key) 2, OK button 1 down.
  • the current interface is a racing game interface, and the control mode is a button response;
  • the operation instruction received by the mobile terminal is: GSensor (gravity sensor) somatosensory control;
  • the command is parsed as: arrow keys, click or long press;
  • the instruction is mapped to: first determine the direction corresponding to the somatosensory operation, and then click;
  • the current interface is a game interface that can respond to touch operations
  • the operation instruction received by the mobile terminal is: touching the third control button;
  • the instruction is parsed as: click;
  • the instruction maps to: click;
  • the command is converted to: Click on the third control button.
  • the mobile terminal may generate a click prompt at a corresponding response position of the projected picture, or may also perform a vibration prompt or the like to enhance the device. Interactivity to enhance the user experience.
  • the projection screen in the mobile terminal also changes.
  • the device control method based on the mobile terminal in the embodiment of the present application, by projecting the screen information currently displayed in the controlled device to the mobile terminal, and converting the control operation of the user for the projection screen into an instruction recognizable by the controlled device to be sent to the
  • the control device controls the controlled device to realize the coordinated operation control between the mobile terminal and the controlled device, and fully utilizes the advantage of the controlled device display experience and the natural terminal convenience of the mobile terminal, and the mobile terminal is diverse, natural and convenient.
  • the operation is converted into an instruction recognizable by the controlled device for control, so that a single controlled device can perform various diversified coordinated control through the mobile terminal, thereby realizing the complementary advantages between the devices, and bringing the user to see That is to say, the resulting visual operation experience greatly improves the freedom and flexibility of user interaction.
  • an abbreviated screen that can generate screen information currently displayed by the controlled device is further improved.
  • the method of the embodiment of the present application may further include steps S401-S402 as shown in FIG. 4. among them,
  • the thumbnail screen may remove the content of the additional information such as the background and the picture from the currently displayed screen of the controlled device. Only the operation controls in the current display screen and the positional information between the operation controls are retained in the thumbnail screen.
  • the display screen in the television can be projected as a thumbnail projection screen in the form of a block diagram in the mobile phone.
  • the device control method based on the mobile terminal in the embodiment of the present application generates a thumbnail image according to the screen displayed in the controlled device, and transmits the thumbnail image to the mobile terminal to generate a projection image, which greatly reduces the amount of data transmitted during the interaction, and improves the data. Transmission efficiency, which can effectively improve the interaction and response time, giving users a smooth and fast experience.
  • steps S601-S602 may also be included. among them:
  • the application scenario may include a scene in progress of the game, a main interface scene, a video playing scene, and the like.
  • the application scenario corresponding to the screen information may be determined according to a type to which the application that generates the screen information belongs. For example, if the application that generates the screen information is a game-like application, the application scenario corresponding to the screen information is a scene in progress of the game; and the application that generates the screen information is a video playback application, The application scenario corresponding to the screen information is a video playback scenario.
  • the application scenarios corresponding to different screen information may be further refined. Specifically, on the basis of determining an application that generates the screen information, image recognition may be further performed on the screen information to determine the content of the screen information, and the corresponding application scenario is determined according to the content of the screen information. For example, if the content of the screen information is an application or a device main interface or a menu interface, the application scenario corresponding to the screen information is a main interface scenario; if the content of the screen information is an interface during a running of the game program , the picture information The corresponding application scenario is a scenario in which the game is in progress.
  • the display parameters may include fluency and/or resolution.
  • the update frequency is low during the display process, or the interface change occurs only when the user operates, so the fluency can be appropriately reduced, for example, the frame rate can be set to 20fps ((Frames per Second); for scenes in the game, or video playback scenes, the picture is dynamically changed, so the fluency can be adjusted appropriately, for example, can be set to 30fps (( Frames per Second, display frames per second).
  • the resolution it can also be adjusted according to the display requirements of the actual scene.
  • the display parameters of the projection screen can be dynamically adjusted according to the application scene corresponding to the screen information in the controlled device, so that the user can effectively reduce the power consumption of the device while giving a good visual experience to the user.
  • the present application further provides a mobile terminal-based device control apparatus.
  • FIG. 7 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to an embodiment of the present application.
  • the mobile terminal-based device control apparatus includes: a communication module 10, an acquisition module 20, a generation module 30, a first receiving module 40, a conversion module 50, and a sending module 60.
  • the communication module 10 is configured to establish a communication connection with the controlled device.
  • the controlled device may be a smart TV, a television set top box, a personal computer or a projector, or the like.
  • the communication module 10 and the controlled device can pass, but are not limited to, Bluetooth, Wi-Fi (Wireless Fidelity), 2.4G (a wireless communication technology with a frequency band between 2.400 GHz and 2.4835 GHz), iBeacon (A wireless communication method such as a low-power Bluetooth-based wireless communication technology issued by Apple establishes a communication connection or establishes a communication connection by wire.
  • Wi-Fi Wireless Fidelity
  • 2.4G a wireless communication technology with a frequency band between 2.400 GHz and 2.4835 GHz
  • iBeacon A wireless communication method such as a low-power Bluetooth-based wireless communication technology issued by Apple establishes a communication connection or establishes a communication connection by wire.
  • the obtaining module 20 is configured to acquire screen information currently displayed by the controlled device.
  • the obtaining module 20 can obtain the screen information currently displayed by the controlled device in a plurality of different manners. In the present application, the following two methods are exemplified.
  • the controlled device pushes the displayed image to the acquisition module 20 in real time.
  • the controlled device provides the data source address of the displayed screen to the obtaining module 20, and the acquiring module 20 can acquire the screen displayed in the controlled device from the data source address synchronously with the controlled device.
  • the generating module 30 is configured to generate a projection screen according to the screen information, and display the same.
  • the generating module 30 may resize the screen information to adapt to the display size in the mobile terminal.
  • the generating module 30 is configured to: resize the screen information according to a preset size of the display area to generate the projected picture.
  • the size of the preset display area may be determined according to the actual display area of the mobile terminal. For example, if it is a full screen display, the size of the preset display area may be the screen size of the mobile terminal, if it is displayed in a certain window. , the size of the preset display area may be the size of the window.
  • the first receiving module 40 is configured to receive a control operation of the user for the projected picture.
  • the mobile terminal When the user needs to operate on the screen information displayed by the controlled device, the mobile terminal can be directly operated according to the projection screen displayed in the mobile terminal.
  • the mobile terminal has various and abundant sensors and devices, such as a touch screen, a camera, a voice input device, a gravity sensor, a gyroscope, an accelerometer, and the like. Therefore, the first receiving module 40 can receive different forms of control operations.
  • the foregoing control operation may include at least one of the following:
  • the user can input the control operation by clicking an operation control on the projection screen, or inputting a voice instruction, or by controlling the mobile terminal to shake left and right in the space.
  • the conversion module 50 is configured to convert the control operation into an instruction recognizable by the controlled device.
  • the conversion module 50 may query the controlled device according to the identification information of the controlled device (such as the device serial number, etc.) or the identifier of the application (such as the application package name, name, etc.) of the device in the controlled device that displays the screen information. Recognizable instructions.
  • the identification information of the controlled device such as the device serial number, etc.
  • the identifier of the application such as the application package name, name, etc.
  • the conversion module 50 may further include: an identification unit 51, a determination unit 52, and a conversion unit 53.
  • the identification unit 51 is configured to identify the control operation to determine an operation instruction corresponding to the control operation.
  • the recognition unit 51 may recognize the touch operation as an operation instruction to the corresponding operation control. For example, if the control operation is to touch the third control button A in the projected screen, it can be recognized as "click operation on the control button A".
  • control operation is a voice operation
  • the recognition unit 51 can perform voice recognition on the voice input by the user to obtain an operation instruction for the corresponding control.
  • the recognition unit 51 may recognize that the somatosensory operation or the gesture operation is a key operation on the corresponding button.
  • the determining unit 52 is configured to determine an instruction mapping relationship corresponding to the screen information
  • the determining unit 52 may query the instruction mapping relationship corresponding thereto according to the identification information of the controlled device (such as the device serial number, etc.). Or the determining unit 52 may query the instruction mapping relationship corresponding to the identifier of the application (such as the application package name, name, and the like) of the application device in the controlled device.
  • the identification information of the controlled device such as the device serial number, etc.
  • the determining unit 52 may query the instruction mapping relationship corresponding to the identifier of the application (such as the application package name, name, and the like) of the application device in the controlled device.
  • the instruction mapping relationship is a mapping relationship between an instruction in the mobile terminal and an instruction recognizable by the controlled device. Therefore, based on the instruction mapping relationship, an instruction recognizable by the controlled device corresponding to the identified instruction can be queried.
  • the determining unit 52 is configured to: perform image recognition on the screen information to acquire an identifier of an application that generates the screen information in the controlled device. Querying the preset database according to the identifier of the application to obtain a corresponding instruction mapping relationship.
  • the identifier of the currently displayed application may be obtained from the system of the controlled device through a preset interface function; or may be controlled by The screen information currently displayed by the device performs image recognition to obtain text information or an icon in the currently displayed screen information, and determines a corresponding application from the acquired text information or icon.
  • the instruction mapping relationship corresponding to different applications is stored in the preset database, and may be stored locally in the mobile terminal or in the server.
  • the converting unit 53 is configured to convert the operation instruction into an instruction recognizable by the controlled device according to the instruction mapping relationship.
  • the converting unit 53 can parse, map, and convert the above operation instructions according to the instruction mapping relationship.
  • the following are three examples of converting the operational instructions into instructions identifiable by the controlled device in accordance with the instruction mapping relationship.
  • Example 1 (as shown in Figure 3)
  • TV (television) control mode no touch response, that is, can not respond to touch operation;
  • Operation command received by Phone touch the third control button
  • the command is parsed as: move, click;
  • the instruction is mapped to: first position the current position of the control focus (for the first control button), then move the control focus from the current position to the third control button, and then click the third control button;
  • the command is converted to: right button (direction key) 2, OK button 1 down.
  • the current interface is a racing game interface, and the control mode is a button response;
  • the operation instruction received by the mobile terminal is: GSensor (gravity sensor) somatosensory control;
  • the command is parsed as: arrow keys, click or long press;
  • the instruction is mapped to: first determine the direction corresponding to the somatosensory operation, and then click;
  • the current interface is a game interface that can respond to touch operations
  • the operation instruction received by the mobile terminal is: touching the third control button;
  • the instruction is parsed as: click;
  • the instruction maps to: click;
  • the command is converted to: Click on the third control button.
  • the sending module 60 is configured to send an instruction recognizable by the controlled device to the controlled device to control the controlled device.
  • the controlled device when it receives the instruction sent by the sending module 60, it can respond.
  • the mobile terminal may generate a click prompt at a corresponding response position of the projection screen, or may also perform a vibration prompt or the like to enhance the interaction between the devices and enhance the user experience.
  • the projection screen in the mobile terminal also changes.
  • the device control device based on the mobile terminal of the embodiment of the present application, by projecting the screen information currently displayed in the controlled device to the mobile terminal, and converting the control operation of the user for the projection screen into an instruction recognizable by the controlled device, is sent to the
  • the control device controls the controlled device to realize the coordinated operation control between the mobile terminal and the controlled device, and fully utilizes the advantage of the controlled device display experience and the natural terminal convenience of the mobile terminal, and the mobile terminal is diverse, natural and convenient.
  • the operation is converted into an instruction recognizable by the controlled device for control, so that a single controlled device can perform various diversified coordinated control through the mobile terminal, thereby realizing the complementary advantages between the devices, and bringing the user to see That is to say, the resulting visual operation experience greatly improves the freedom and flexibility of user interaction.
  • FIG. 9 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to another embodiment of the present application.
  • the mobile terminal-based device control apparatus includes: a communication module 10, an acquisition module 20, a generation module 30, a first receiving module 40, a conversion module 50, a sending module 60, and a determining module. 70 and control module 80.
  • the communication module 10, the acquisition module 20, the generation module 30, the first receiving module 40, the conversion module 50, and the transmission module 60 are the same as the embodiment shown in FIG.
  • the determining module 70 is configured to determine an application scenario corresponding to the screen information.
  • the application scenario may include a scene in progress of the game, a main interface scene, a video playing scene, and the like.
  • the determining module 70 may determine an application scenario corresponding to the screen information according to a type to which the application that generates the screen information belongs. For example, if the application that generates the screen information is a game-like application, the application scenario corresponding to the screen information is a scene in progress of the game; and the application that generates the screen information is a video playback application, The application scenario corresponding to the screen information is a video playback scenario.
  • the determining module 70 may further divide the application scenarios corresponding to different screen information in a more refined manner. Specifically, on the basis of determining the application that generates the screen information, the determining module 70 may further perform image recognition on the screen information to determine the content of the screen information, and determine a corresponding application scenario according to the content of the screen information. . For example, if the content of the screen information is an application or a device main interface or a menu interface, the application scenario corresponding to the screen information is a main interface scenario; if the content of the screen information is an interface during a running of the game program The application scenario corresponding to the screen information is that the game in progress scene control module 80 is configured to control display parameters of the projection screen according to the application scenario.
  • the display parameters may include fluency and/or resolution.
  • the update frequency is low during the display process, or the interface change occurs only when the user performs the operation, so the control module 80 can appropriately reduce the fluency, for example, the frame can be The rate is set to 20fps ((Frames per Second); for a scene in a game, or a video playback scene, the picture is dynamically changed. Therefore, the control module 80 can appropriately adjust the fluency, for example, Can be set to 30fps ((Frames per Second).
  • control module 80 can also adjust according to the display needs of the actual scene.
  • the display parameters of the projection screen can be dynamically adjusted according to the application scene corresponding to the screen information in the controlled device, so that the user can effectively reduce the power consumption of the device while giving a good visual experience to the user.
  • FIG. 10 is a schematic structural diagram of a device control apparatus based on a mobile terminal according to another embodiment of the present application.
  • the mobile terminal-based device control apparatus includes: a communication module 10, an acquisition module 20, a generation module 30, a first receiving module 40, a conversion module 50, a sending module 60, and a second Receive module 90.
  • the communication module 10, the acquisition module 20, the generation module 30, the first receiving module 40, the conversion module 50, and the transmission module 60 are the same as the embodiment shown in FIG.
  • the second receiving module 90 is configured to receive the thumbnail screen sent by the controlled device, where the thumbnail screen includes an operation control included in the screen information, and the operation control is in a thumbnail screen.
  • the location information is the same as the location information of the operation control in the screen information.
  • the thumbnail screen may remove the content of the additional information such as the background and the picture from the currently displayed screen of the controlled device. Only the operation controls in the current display screen and the positional information between the operation controls are retained in the thumbnail screen.
  • the generating module 30 is further configured to generate a projected image according to the thumbnail image.
  • the display screen in the television can be projected as a thumbnail projection screen in the form of a block diagram in the mobile phone.
  • the device control device based on the mobile terminal in the embodiment of the present invention generates a thumbnail image according to the screen displayed in the controlled device, and transmits the thumbnail image to the mobile terminal to generate a projection image, which greatly reduces the amount of data transmitted during the interaction, and improves the amount of data. Transmission efficiency, which can effectively improve the interaction and response time, giving users a smooth and fast experience.
  • the application also proposes a mobile terminal.
  • a mobile terminal includes: a mobile terminal based on any embodiment of the present application End device control device.
  • the mobile terminal of the embodiment of the present application by projecting the screen information currently displayed in the controlled device to the mobile terminal, and converting the control operation of the user for the projection screen into an instruction recognizable by the controlled device, is sent to the controlled device, so as to
  • the controlled device controls the coordinated operation control between the mobile terminal and the controlled device, fully utilizes the advantage of the controlled device display experience, and the natural terminal convenience of the mobile terminal, and converts the diverse, natural and convenient operation of the mobile terminal into the Control the identifiable instructions of the device for control, so that a single controlled device can perform various diversified coordinated control through the mobile terminal, thereby realizing the complementary advantages between the devices, and providing the user with a WYSIWYG visual operation.
  • the user interaction and flexibility are greatly improved.
  • a "computer-readable medium” can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, apparatus, or device, or in conjunction with the instruction execution system, apparatus, or device.
  • computer readable media include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM).
  • the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as may be for example by paper or other media The quality is optically scanned, followed by editing, interpretation or, if necessary, processing in other suitable manner to electronically obtain the program and then store it in computer memory.
  • portions of the application can be implemented in hardware, software, firmware, or a combination thereof.
  • multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: having logic gates for implementing logic functions on data signals. Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.
  • each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.
  • the above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请提出一种基于移动终端的设备控制方法、装置和移动终端，其中，该基于移动终端的设备控制方法，包括以下步骤：与被控设备建立通信连接；获取所述被控设备当前显示的画面信息，并根据所述画面信息生成投影画面，并进行显示；接收用户针对所述投影画面的控制操作；将所述控制操作转换为所述被控设备可识别的指令，并发送至所述被控设备，以对所述被控制设备进行控制。本申请的基于移动终端的设备控制方法，实现了设备间的优势互补，极大地提高了用户交互自由度和灵活性。

Description

基于移动终端的设备控制方法、装置和移动终端 技术领域
本申请涉及通信技术领域，特别涉及一种基于移动终端的设备控制方法、装置和移动终端。
背景技术
智能设备已经越来越广泛的进入人们的日常生活，用户可通过智能机顶盒、智能电视、个人计算机、投影机等设备进行影音、娱乐、购物等各种各样的活动。这些智能设备大多是采用遥控器或鼠标进行控制，操作方式单一，且操作不方便，特别是在输入汉字、按键操作或者玩游戏等场景下，操作不便更为明显。
发明内容
本申请旨在至少在一定程度上解决上述技术问题。
为此，本申请的第一个目的在于提出一种基于移动终端的设备控制方法，实现了设备间的优势互补，极大地提高了用户交互自由度和灵活性。
本申请的第二个目的在于提出一种基于移动终端的设备控制装置。
本申请的第三个目的在于提出一种移动终端。
为达上述目的，根据本申请第一方面实施例提出了一种基于移动终端的设备控制方法，包括以下步骤：与被控设备建立通信连接；获取所述被控设备当前显示的画面信息，并根据所述画面信息生成投影画面，并进行显示；接收用户针对所述投影画面的控制操作；将所述控制操作转换为所述被控设备可识别的指令，并发送至所述被控设备，以对所述被控制设备进行控制。
本申请实施例的基于移动终端的设备控制方法，通过将被控设备中当 前显示的画面信息投影至移动终端，并将用户针对投影画面的控制操作转换为被控设备可识别的指令发送至被控设备，以对被控设备进行控制，实现了移动终端与被控设备的协同操作控制，充分利用被控设备显示体验好、移动终端操控自然、方便的优势，将移动终端多样、自然、方便的操作转换为被控设备可识别的指令，以进行控制，从而操作单一的被控设备可通过移动终端进行各种多样化的协同控制，实现了设备间的优势互补，在带给用户所见即所得的可视化操作体验的同时，极大地提高了用户交互自由度和灵活性。
本申请第二方面实施例提供了一种基于移动终端的设备控制装置，包括：通信模块，用于与被控设备建立通信连接；获取模块，用于获取所述被控设备当前显示的画面信息；生成模块，用于根据所述画面信息生成投影画面，并进行显示；第一接收模块，用于接收用户针对所述投影画面的控制操作；转换模块，用于将所述控制操作转换为所述被控设备可识别的指令；发送模块，用于将所述被控设备可识别的指令发送至所述被控设备，以对所述被控制设备进行控制。
本申请实施例的基于移动终端的设备控制装置，通过将被控设备中当前显示的画面信息投影至移动终端，并将用户针对投影画面的控制操作转换为被控设备可识别的指令发送至被控设备，以对被控设备进行控制，实现了移动终端与被控设备的协同操作控制，充分利用被控设备显示体验好、移动终端操控自然、方便的优势，将移动终端多样、自然、方便的操作转换为被控设备可识别的指令，以进行控制，从而操作单一的被控设备可通过移动终端进行各种多样化的协同控制，实现了设备间的优势互补，在带给用户所见即所得的可视化操作体验的同时，极大地提高了用户交互自由度和灵活性。
本申请第三方面实施例提供了一种移动终端，包括本申请第二方面实施例的基于移动终端的设备控制装置。
本申请实施例的移动终端，通过将被控设备中当前显示的画面信息投影至移动终端，并将用户针对投影画面的控制操作转换为被控设备可识别的指令发送至被控设备，以对被控设备进行控制，实现了移动终端与被控 设备的协同操作控制，充分利用被控设备显示体验好、移动终端操控自然、方便的优势，将移动终端多样、自然、方便的操作转换为被控设备可识别的指令，以进行控制，从而操作单一的被控设备可通过移动终端进行各种多样化的协同控制，实现了设备间的优势互补，在带给用户所见即所得的可视化操作体验的同时，极大地提高了用户交互自由度和灵活性。
本申请的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1为根据本申请一个实施例的基于移动终端的设备控制方法的流程图；
图2为根据本申请另一个实施例的基于移动终端的设备控制方法的流程图；
图3为根据本申请一个实施例的指令转换的示意图；
图4为根据本申请另一个实施例的基于移动终端的设备控制方法的流程图；
图5为根据本申请一个实施例的投影画面的示意图；
图6为根据本申请另一个实施例的基于移动终端的设备控制方法的流程图；
图7为根据本申请一个实施例的基于移动终端的设备控制装置的结构示意图；
图8为根据本申请另一个实施例的基于移动终端的设备控制装置的结构示意图；
图9为根据本申请另一个实施例的基于移动终端的设备控制装置的结构示意图；
图10为根据本申请另一个实施例的基于移动终端的设备控制装置的结构示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能理解为对本申请的限制。
由于移动终端具有自然、多样的用户交互方式，如触摸、语音、体感等控制方式，便于操控的优点，因此，为了解决当前的智能机顶盒、智能电视、个人计算机、投影机等设备虽然具有大屏显示或播放、便于观看的有点，但操作方式单一、操作不方便的问题，本申请的实施例提出了一种基于移动终端的设备控制方法和装置，可以将上述两类设备的优点相结合，实现大屏显示+自然操控优势结合，以提升用户使用时的交互体验。
下面参考附图描述根据本申请实施例的基于移动终端的设备控制方法和装置。
需要说明的是，本申请实施例中的移动终端可以是手机、平板电脑、智能穿戴式设备、个人数字助理等终端设备。
图1为根据本申请一个实施例的基于移动终端的设备控制方法的流程图。如图1所示，根据本申请实施例的基于移动终端的设备控制方法，包括：
S101，与被控设备建立通信连接。
在本申请的一个实施例中，被控设备可为智能电视、电视机顶盒、个人计算机或投影机等。
其中，移动终端与被控设备可通过但不限于蓝牙、Wi-Fi(Wireless Fidelity，无线保真)、2.4G(一种频段处于2.400GHz～2.4835GHz之间的无线通信技术)、iBeacon(一种由苹果公司发布的基于低功耗蓝牙的无线通 信技术)等无线通信方式建立通信连接，或者通过有线方式建立通信连接。
S102，获取所述被控设备当前显示的画面信息，并根据所述画面信息生成投影画面，并进行显示。
在本申请的一个实施例中，移动终端可通过多种不同方式获取被控设备当前显示的画面信息。本申请中以以下两种方式进行举例说明。
方式一
被控设备对其显示的画面实时推送到移动终端。
方式二
被控设备将其显示的画面的数据源地址提供给移动终端，移动终端可与被控设备从该数据源地址同步获取被控设备中显示的画面。
应当理解，上述两种方式仅是示例性的，本申请中，并不限于通过上述两种方式实现移动终端与被控设备中画面同步，其他可行的同步方式也应包含在本申请的范围内。
在本申请的一个实施例中，由于移动终端与被控设备的显示屏幕的尺寸存在差异。因此，移动终端在获取被控设备当前显示的画面信息之后，可对该画面信息进行尺寸调整，以适应移动终端中显示尺寸。具体地，根据所述画面信息生成投影画面可包括：根据预设的显示区域的尺寸对所述画面信息进行尺寸调整，以生成所述投影画面。其中，预设的显示区域的尺寸可根据移动终端的实际显示区域确定，例如，如果是全屏显示，则预设的显示区域的尺寸可为移动终端的屏幕尺寸，如果是在某一窗口中显示，则预设的显示区域的尺寸可可为该窗口的尺寸。
S103，接收用户针对所述投影画面的控制操作。
当用户需要针对被控设备显示的画面信息进行操作时，可直接根据移动终端中显示的投影画面对移动终端进行操作。由于移动终端具有多样、丰富的传感器及装置，如触摸屏、摄像头、语音输入装置、重力感应器、陀螺仪、加速度计等。因此，移动终端可接收不同形式的控制操作。在本申请的一个实施例中，上述控制操作可包括以下至少之一：
触摸操作、体感操作、手势操作、语音操作。
举例来说，用户可通过点击投影画面上的操作控件，或者输入语音指令，或者通过控制移动终端在空间左右晃动等方式输入控制操作。
S104，将所述控制操作转换为所述被控设备可识别的指令，并发送至所述被控设备，以对所述被控制设备进行控制。
具体地，可根据被控设备的标识信息(如设备序列号等)或者被控设备中显示所述画面信息的应用程序的标识(如应用程序包名、名称等)查询被控设备可识别的指令。
在本申请的一个实施例中，所述将所述控制操作转换为所述被控设备可识别的指令，可包括如图2所示的步骤S201-S203。
S201，对所述控制操作进行识别，以确定所述控制操作对应的操作指令。
如果控制操作为触摸操作，则可将该触摸操作识别为对相应操作控件的操作指令。举例来说，如果控制操作为触摸投影画面中的第3个控件按钮A，则可识别为“对控件按钮A的点击操作”。
如果控制操作为语音操作，则可对用户输入的语音进行语音识别，以得到对相应控件的操作指令。
如果控制操作为体感操作或者手势操作，则可识别体感操作或手势操作为对相应按键的按键操作。
S202，确定所述画面信息对应的指令映射关系。
在本申请的实施例中，可根据被控设备的标识信息(如设备序列号等)查询与之对应的指令映射关系。或者可根据被控设备中显示所述画面信息的应用程序的标识(如应用程序包名、名称等)查询与之对应的指令映射关系。
其中，指令映射关系为移动终端中的指令与被控设备可识别的指令之间的映射关系。因此，基于该指令映射关系可查询到识别出的指令对应的被控设备可识别的指令。
在本申请的一个实施例中，确定所述画面信息对应的指令映射关系可包括:对所述画面信息进行图像识别，以获取所述被控设备中生成所述画面信息的应用程序的标识；根据所述应用程序的标识查询预设数据库，以获取对应的指令映射关系。
在本申请的一个实施例中，可通过预设的接口函数从被控设备的系统中获取当前显示的应用程序(即生成所述画面信息的应用程序)的标识；或者，可通过对被控设备当前显示的画面信息进行图像识别，以获取当前显示的画面信息中的文字信息或者图标，并从获取的文字信息或图标确定对应的应用程序。
其中，预设数据库中存储有不同应用程序对应的指令映射关系，可存储在移动终端本地或者服务器中。
S203，根据所述指令映射关系将所述操作指令转换为所述被控设备可识别的指令。
具体地，可根据指令映射关系对上述操作指令进行解析、映射和转换。举例来说，以下为根据所述指令映射关系将所述操作指令转换为所述被控设备可识别的指令的三个示例。
示例一(如图3所示)
TV(电视)操控方式：无触摸响应，即不可响应触摸操作；
Phone(移动终端)接收到的操作指令：触摸第3个控件按钮；
指令解析为：移动，点击；
指令映射为：首先定位控制焦点当前位置(为第1个控件按钮)，接着将控制焦点从当前位置移动到第3个控件按钮，然后点击第3个控件按钮；
指令转换为：右键(方向键)2下，OK键1下。
示例二
当前界面为赛车游戏界面，操控方式为按键响应；
移动终端接收到的操作指令为：GSensor(重力传感器)体感操控；
指令解析为：方向键，单击或长按等；
指令映射为：首先确定体感操作对应的方向，然后点击；
指令转换：右键(方向键)，单击或长按。
示例三
当前界面为可以响应触摸操作的游戏界面；
移动终端接收到的操作指令为：触摸第3个控件按钮；
指令解析为：点击；
指令映射为：点击；
指令转换为：点击第3个控件按钮一下。
在本申请的一个实施例中，当被控设备响应移动终端发送的可识别的指令后，移动终端可在投影画面的相应响应位置产生点击提示，或者，还可以进行振动提示等，以增强设备间的互动性，提升用户体验。
此外，由于移动终端获取的被控设备当前显示的画面信息，因此，当被控设备显示的画面发生变化时，移动终端中的投影画面也随之改变。
本申请实施例的基于移动终端的设备控制方法，通过将被控设备中当前显示的画面信息投影至移动终端，并将用户针对投影画面的控制操作转换为被控设备可识别的指令发送至被控设备，以对被控设备进行控制，实现了移动终端与被控设备的协同操作控制，充分利用被控设备显示体验好、移动终端操控自然、方便的优势，将移动终端多样、自然、方便的操作转换为被控设备可识别的指令，以进行控制，从而操作单一的被控设备可通过移动终端进行各种多样化的协同控制，实现了设备间的优势互补，在带给用户所见即所得的可视化操作体验的同时，极大地提高了用户交互自由度和灵活性。
进一步地，为了提高数据传输速度，进而提高可生成被控设备当前显示的画面信息的缩略画面。具体地，在本申请的另一个实施例中，本申请实施例的方法还可包括如图4所示的步骤S401-S402。其中，
S401，接收所述被控设备发送的缩略画面，其中，所述缩略画面中包 括所述画面信息中所具有的操作控件，且所述操作控件在缩略画面中的位置信息与所述操作控件在所述画面信息中位置信息相同。
其中，缩略画面可为被控设备当前显示的画面中去掉背景、图片等附加信息的内容。缩略画面中仅保留了当前显示画面中的操作控件以及操作控件之间的位置信息。
S402，根据所述缩略画面生成投影画面。
举例来说，如图5所示，可将电视中的显示画面投影为手机中框图形式的缩略投影画面。
本申请实施例的基于移动终端的设备控制方法，根据被控设备中显示的画面生成缩略画面，并传输至移动终端，生成投影画面，大大减小了交互过程中传输的数据量，提高了传输效率，从而能够有效提高交互和响应时间，给用户以流畅、极速的体验。
在本申请的一个实施例中，如图6所示，还可包括步骤S601-S602。其中：
S601，确定所述画面信息对应的应用场景。
举例来说，应用场景可包括游戏进行中的场景、主界面场景、视频播放场景等。
在本申请的一个实施例中，可根据生成所述画面信息的应用程序所属的类型确定所述画面信息对应的应用场景。举例来说，如果生成所述画面信息的应用程序为游戏类应用，则所述画面信息对应的应用场景为游戏进行中的场景；生成所述画面信息的应用程序为视频播放类应用，则所述画面信息对应的应用场景为视频播放场景。
进一步地，可更精细化地对不同画面信息对应的应用场景进行划分。具体地，在确定生成所述画面信息的应用程序的基础上，可进一步对画面信息进行图像识别，以确定该画面信息的内容，并根据所述画面信息的内容确定对应的应用场景。举例来说，如果所述画面信息的内容为应用程序或设备主界面或菜单界面，则所述画面信息对应的应用场景为主界面场景；如果所述画面信息的内容为游戏程序运行中的界面，则所述画面信息 对应的应用场景为游戏进行中的场景。
S602，根据所述应用场景控制所述投影画面的显示参数。
在本申请的一个实施例中，显示参数可包括流畅度和/或分辨率。
举例来说，对于应用程序或者设备的主界面，其在显示过程中更新频率较低，或者只有在用户进行操作时才会发生界面变化，因此可适当降低流畅度，例如可将帧率设置为20fps((Frames per Second，每秒显示帧数)；而对于游戏中的场景、或者视频播放场景，其画面是动态变换的，因此，可适当调高流畅度，例如，可设置为30fps((Frames per Second，每秒显示帧数)。
对于分辨率来说，也可根据实际场景的显示需要进行调整。
由此，可根据被控设备中画面信息对应的应用场景动态调整投影画面的显示参数，在给用户良好的视觉体验的同时，能够有效降低设备功耗。
与上述实施例提供的基于移动终端的设备控制方法相对应，本申请还提出一种基于移动终端的设备控制装置。
图7为根据本申请一个实施例的基于移动终端的设备控制装置的结构示意图。
如图7所示，根据本申请实施例的基于移动终端的设备控制装置，包括：通信模块10、获取模块20、生成模块30、第一接收模块40、转换模块50和发送模块60。
具体地，通信模块10用于与被控设备建立通信连接。
在本申请的一个实施例中，被控设备可为智能电视、电视机顶盒、个人计算机或投影机等。
其中，通信模块10与被控设备可通过但不限于蓝牙、Wi-Fi(Wireless Fidelity，无线保真)、2.4G(一种频段处于2.400GHz～2.4835GHz之间的无线通信技术)、iBeacon(一种由苹果公司发布的基于低功耗蓝牙的无线通信技术)等无线通信方式建立通信连接，或者通过有线方式建立通信连接。
获取模块20用于获取所述被控设备当前显示的画面信息。
在本申请的一个实施例中，获取模块20可通过多种不同方式获取被控设备当前显示的画面信息。本申请中以以下两种方式进行举例说明。
方式一
被控设备对其显示的画面实时推送到获取模块20。
方式二
被控设备将其显示的画面的数据源地址提供给获取模块20，获取模块20可与被控设备从该数据源地址同步获取被控设备中显示的画面。
应当理解，上述两种方式仅是示例性的，本申请中，并不限于通过上述两种方式实现移动终端与被控设备中画面同步，其他可行的同步方式也应包含在本申请的范围内。
生成模块30用于根据所述画面信息生成投影画面，并进行显示。
在本申请的一个实施例中，由于移动终端与被控设备的显示屏幕的尺寸存在差异。因此，获取模块20在获取被控设备当前显示的画面信息之后，生成模块30可对该画面信息进行尺寸调整，以适应移动终端中显示尺寸。具体地，生成模块30可用于：根据预设的显示区域的尺寸对所述画面信息进行尺寸调整，以生成所述投影画面。其中，预设的显示区域的尺寸可根据移动终端的实际显示区域确定，例如，如果是全屏显示，则预设的显示区域的尺寸可为移动终端的屏幕尺寸，如果是在某一窗口中显示，则预设的显示区域的尺寸可可为该窗口的尺寸。
第一接收模块40用于接收用户针对所述投影画面的控制操作。
当用户需要针对被控设备显示的画面信息进行操作时，可直接根据移动终端中显示的投影画面对移动终端进行操作。由于移动终端具有多样、丰富的传感器及装置，如触摸屏、摄像头、语音输入装置、重力感应器、陀螺仪、加速度计等。因此，第一接收模块40可接收不同形式的控制操作。在本申请的一个实施例中，上述控制操作可包括以下至少之一：
触摸操作、体感操作、手势操作、语音操作。
举例来说，用户可通过点击投影画面上的操作控件，或者输入语音指令，或者通过控制移动终端在空间左右晃动等方式输入控制操作。
转换模块50用于将所述控制操作转换为所述被控设备可识别的指令。
具体地，转换模块50可根据被控设备的标识信息(如设备序列号等)或者被控设备中显示所述画面信息的应用程序的标识(如应用程序包名、名称等)查询被控设备可识别的指令。
在本申请的一个实施例中，如图8所示，转换模块50可进一步包括：识别单元51、确定单元52和转换单元53。
其中，识别单元51用于对所述控制操作进行识别，以确定所述控制操作对应的操作指令。
如果控制操作为触摸操作，则识别单元51可将该触摸操作识别为对相应操作控件的操作指令。举例来说，如果控制操作为触摸投影画面中的第3个控件按钮A，则可识别为“对控件按钮A的点击操作”。
如果控制操作为语音操作，则识别单元51可对用户输入的语音进行语音识别，以得到对相应控件的操作指令。
如果控制操作为体感操作或者手势操作，则识别单元51可识别体感操作或手势操作为对相应按键的按键操作。
确定单元52用于确定所述画面信息对应的指令映射关系；
在本申请的实施例中，确定单元52可根据被控设备的标识信息(如设备序列号等)查询与之对应的指令映射关系。或者确定单元52可根据被控设备中显示所述画面信息的应用程序的标识(如应用程序包名、名称等)查询与之对应的指令映射关系。
其中，指令映射关系为移动终端中的指令与被控设备可识别的指令之间的映射关系。因此，基于该指令映射关系可查询到识别出的指令对应的被控设备可识别的指令。
在本申请的一个实施例中，确定单元52可用于：对所述画面信息进行图像识别，以获取所述被控设备中生成所述画面信息的应用程序的标 识；根据所述应用程序的标识查询预设数据库，以获取对应的指令映射关系。
在本申请的一个实施例中，可通过预设的接口函数从被控设备的系统中获取当前显示的应用程序(即生成所述画面信息的应用程序)的标识；或者，可通过对被控设备当前显示的画面信息进行图像识别，以获取当前显示的画面信息中的文字信息或者图标，并从获取的文字信息或图标确定对应的应用程序。
其中，预设数据库中存储有不同应用程序对应的指令映射关系，可存储在移动终端本地或者服务器中。
转换单元53用于根据所述指令映射关系将所述操作指令转换为所述被控设备可识别的指令。
具体地，转换单元53可根据指令映射关系对上述操作指令进行解析、映射和转换。举例来说，以下为根据所述指令映射关系将所述操作指令转换为所述被控设备可识别的指令的三个示例。
示例一(如图3所示)
TV(电视)操控方式：无触摸响应，即不可响应触摸操作；
Phone(移动终端)接收到的操作指令：触摸第3个控件按钮；
指令解析为：移动，点击；
指令映射为：首先定位控制焦点当前位置(为第1个控件按钮)，接着将控制焦点从当前位置移动到第3个控件按钮，然后点击第3个控件按钮；
指令转换为：右键(方向键)2下，OK键1下。
示例二
当前界面为赛车游戏界面，操控方式为按键响应；
移动终端接收到的操作指令为：GSensor(重力传感器)体感操控；
指令解析为：方向键，单击或长按等；
指令映射为：首先确定体感操作对应的方向，然后点击；
指令转换：右键(方向键)，单击或长按。
示例三
当前界面为可以响应触摸操作的游戏界面；
移动终端接收到的操作指令为：触摸第3个控件按钮；
指令解析为：点击；
指令映射为：点击；
指令转换为：点击第3个控件按钮一下。
发送模块60用于将所述被控设备可识别的指令发送至所述被控设备，以对所述被控制设备进行控制。
在本申请的一个实施例中，当被控设备接收到发送模块60发送的指令后，可进行响应。此时，移动终端可在投影画面的相应响应位置产生点击提示，或者，还可以进行振动提示等，以增强设备间的互动性，提升用户体验。
此外，由于移动终端获取的被控设备当前显示的画面信息，因此，当被控设备显示的画面发生变化时，移动终端中的投影画面也随之改变。
本申请实施例的基于移动终端的设备控制装置，通过将被控设备中当前显示的画面信息投影至移动终端，并将用户针对投影画面的控制操作转换为被控设备可识别的指令发送至被控设备，以对被控设备进行控制，实现了移动终端与被控设备的协同操作控制，充分利用被控设备显示体验好、移动终端操控自然、方便的优势，将移动终端多样、自然、方便的操作转换为被控设备可识别的指令，以进行控制，从而操作单一的被控设备可通过移动终端进行各种多样化的协同控制，实现了设备间的优势互补，在带给用户所见即所得的可视化操作体验的同时，极大地提高了用户交互自由度和灵活性。
图9为根据本申请另一个实施例的基于移动终端的设备控制装置的结构示意图。
如图9所示，根据本申请实施例的基于移动终端的设备控制装置，包括：通信模块10、获取模块20、生成模块30、第一接收模块40、转换模块50、发送模块60、确定模块70和控制模块80。
其中，通信模块10、获取模块20、生成模块30、第一接收模块40、转换模块50和发送模块60与图7所示实施例相同。
确定模块70用于确定所述画面信息对应的应用场景。
举例来说，应用场景可包括游戏进行中的场景、主界面场景、视频播放场景等。
在本申请的一个实施例中，确定模块70可根据生成所述画面信息的应用程序所属的类型确定所述画面信息对应的应用场景。举例来说，如果生成所述画面信息的应用程序为游戏类应用，则所述画面信息对应的应用场景为游戏进行中的场景；生成所述画面信息的应用程序为视频播放类应用，则所述画面信息对应的应用场景为视频播放场景。
进一步地，确定模块70可更精细化地对不同画面信息对应的应用场景进行划分。具体地，在确定生成所述画面信息的应用程序的基础上，确定模块70可进一步对画面信息进行图像识别，以确定该画面信息的内容，并根据所述画面信息的内容确定对应的应用场景。举例来说，如果所述画面信息的内容为应用程序或设备主界面或菜单界面，则所述画面信息对应的应用场景为主界面场景；如果所述画面信息的内容为游戏程序运行中的界面，则所述画面信息对应的应用场景为游戏进行中的场景控制模块80用于根据所述应用场景控制所述投影画面的显示参数。
在本申请的一个实施例中，显示参数可包括流畅度和/或分辨率。
举例来说，对于应用程序或者设备的主界面，其在显示过程中更新频率较低，或者只有在用户进行操作时才会发生界面变化，因此控制模块80可适当降低流畅度，例如可将帧率设置为20fps((Frames per Second，每秒显示帧数)；而对于游戏中的场景、或者视频播放场景，其画面是动态变换的，因此，控制模块80可适当调高流畅度，例如，可设置为30fps((Frames per Second，每秒显示帧数)。
对于分辨率来说，控制模块80也可根据实际场景的显示需要进行调整。
由此，可根据被控设备中画面信息对应的应用场景动态调整投影画面的显示参数，在给用户良好的视觉体验的同时，能够有效降低设备功耗。
图10为根据本申请另一个实施例的基于移动终端的设备控制装置的结构示意图。
如图10所示，根据本申请实施例的基于移动终端的设备控制装置，包括：通信模块10、获取模块20、生成模块30、第一接收模块40、转换模块50、发送模块60和第二接收模块90。
其中，通信模块10、获取模块20、生成模块30、第一接收模块40、转换模块50和发送模块60与图7所示实施例相同。
第二接收模块90用于接收所述被控设备发送的缩略画面，其中，所述缩略画面中包括所述画面信息中所具有的操作控件，且所述操作控件在缩略画面中的位置信息与所述操作控件在所述画面信息中位置信息相同.
其中，缩略画面可为被控设备当前显示的画面中去掉背景、图片等附加信息的内容。缩略画面中仅保留了当前显示画面中的操作控件以及操作控件之间的位置信息。
其中，生成模块30还用于根据所述缩略画面生成投影画面。
举例来说，如图5所示，可将电视中的显示画面投影为手机中框图形式的缩略投影画面。
本申请实施例的基于移动终端的设备控制装置，根据被控设备中显示的画面生成缩略画面，并传输至移动终端，生成投影画面，大大减小了交互过程中传输的数据量，提高了传输效率，从而能够有效提高交互和响应时间，给用户以流畅、极速的体验。
本申请还提出一种移动终端。
本申请实施例的种移动终端，包括：本申请任一实施例的基于移动终 端的设备控制装置。
本申请实施例的移动终端，通过将被控设备中当前显示的画面信息投影至移动终端，并将用户针对投影画面的控制操作转换为被控设备可识别的指令发送至被控设备，以对被控设备进行控制，实现了移动终端与被控设备的协同操作控制，充分利用被控设备显示体验好、移动终端操控自然、方便的优势，将移动终端多样、自然、方便的操作转换为被控设备可识别的指令，以进行控制，从而操作单一的被控设备可通过移动终端进行各种多样化的协同控制，实现了设备间的优势互补，在带给用户所见即所得的可视化操作体验的同时，极大地提高了用户交互自由度和灵活性。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本申请的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本申请的实施例所属技术领域的技术人员所理解。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，"计算机可读介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介 质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本申请的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或他们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
尽管已经示出和描述了本申请的实施例，本领域的普通技术人员可以理解：在不脱离本申请的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本申请的范围由权利要求及其等同限定。

Claims (19)

  1. 一种基于移动终端的设备控制方法，其特征在于，包括以下步骤：
    与被控设备建立通信连接；
    获取所述被控设备当前显示的画面信息，并根据所述画面信息生成投影画面，并进行显示；
    接收用户针对所述投影画面的控制操作；
    将所述控制操作转换为所述被控设备可识别的指令，并发送至所述被控设备，以对所述被控制设备进行控制。
  2. 如权利要求1所述的方法，其特征在于，所述将所述控制操作转换为所述被控设备可识别的指令，包括：
    对所述控制操作进行识别，以确定所述控制操作对应的操作指令；
    确定所述画面信息对应的指令映射关系；
    根据所述指令映射关系将所述操作指令转换为所述被控设备可识别的指令。
  3. 如权利要求2所述的方法，其特征在于，所述确定所述画面信息对应的指令映射关系，包括:
    获取所述被控设备中生成所述画面信息的应用程序的标识；
    根据所述应用程序的标识查询预设数据库，以获取对应的指令映射关系。
  4. 如权利要求1所述的方法，其特征在于，还包括：
    确定所述画面信息对应的应用场景；
    根据所述应用场景控制所述投影画面的显示参数。
  5. 如权利要求4所述的方法，其特征在于，所述显示参数包括流畅度和/或分辨率。
  6. 如权利要求1所述的方法，其特征在于，所述根据所述画面信息生成投影画面，包括：
    根据预设的显示区域的尺寸对所述画面信息进行尺寸调整，以生成所述投影画面。
  7. 如权利要求1所述的方法，其特征在于，还包括：
    接收所述被控设备发送的缩略画面，其中，所述缩略画面中包括所述画面信息中所具有的操作控件，且所述操作控件在缩略画面中的位置信息与所述操作控件在所述画面信息中位置信息相同；
    根据所述缩略画面生成投影画面。
  8. 如权利要求1所述的方法，其特征在于，所述控制操作包括以下至少之一：
    触摸操作、体感操作、手势操作、语音操作。
  9. 如权利要求1-8任一项所述的方法，其特征在于，所述被控设备为智能电视、电视机顶盒、个人计算机或投影机。
  10. 一种基于移动终端的设备控制装置，其特征在于，包括：
    通信模块，用于与被控设备建立通信连接；
    获取模块，用于获取所述被控设备当前显示的画面信息；
    生成模块，用于根据所述画面信息生成投影画面，并进行显示；
    第一接收模块，用于接收用户针对所述投影画面的控制操作；
    转换模块，用于将所述控制操作转换为所述被控设备可识别的指令；
    发送模块，用于将所述被控设备可识别的指令发送至所述被控设备，以对所述被控制设备进行控制。
  11. 如权利要求10所述的装置，其特征在于，所述转换模块，包括：
    识别单元，用于对所述控制操作进行识别，以确定所述控制操作对应的操作指令；
    确定单元，用于确定所述画面信息对应的指令映射关系；
    转换单元，用于根据所述指令映射关系将所述操作指令转换为所述被控设备可识别的指令。
  12. 如权利要求11所述的装置，其特征在于，所述确定单元用于：
    获取所述被控设备中生成所述画面信息的应用程序的标识；
    根据所述应用程序的标识查询预设数据库，以获取对应的指令映射关系。
  13. 如权利要求10所述的装置，其特征在于，还包括：
    确定模块，用于确定所述画面信息对应的应用场景；
    控制模块，用于根据所述应用场景控制所述投影画面的显示参数。
  14. 如权利要求13所述的装置，其特征在于，所述显示参数包括流畅度和/或分辨率。
  15. 如权利要求10所述的装置，其特征在于，所述生成模块用于：
    根据预设的显示区域的尺寸对所述画面信息进行尺寸调整，以生成所述投影画面。
  16. 如权利要求10所述的装置，其特征在于，还包括：
    第二接收模块，用于接收所述被控设备发送的缩略画面，其中，所述缩略画面中包括所述画面信息中所具有的操作控件，且所述操作控件在缩略画面中的位置信息与所述操作控件在所述画面信息中位置信息相同；
    其中，所述生成模块还用于根据所述缩略画面生成投影画面。
  17. 如权利要求10所述的装置，其特征在于，所述控制操作包括以下至少之一：
    触摸操作、体感操作、手势操作、语音操作。
  18. 如权利要求10-17任一项所述的装置，其特征在于，所述被控设备为智能电视、电视机顶盒、个人计算机或投影机。
  19. 一种移动终端，其特征在于，包括如权利要求10-18任一项所述的基于移动终端的设备控制装置。
PCT/CN2017/074168 2016-03-01 2017-02-20 基于移动终端的设备控制方法、装置和移动终端 WO2017148294A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/119,730 US20180375987A1 (en) 2016-03-01 2018-08-31 Method, apparatus and mobile terminal for device control based on a mobile terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610115087.6A CN107147769B (zh) 2016-03-01 2016-03-01 基于移动终端的设备控制方法、装置和移动终端
CN201610115087.6 2016-03-01

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/119,730 Continuation US20180375987A1 (en) 2016-03-01 2018-08-31 Method, apparatus and mobile terminal for device control based on a mobile terminal

Publications (1)

Publication Number Publication Date
WO2017148294A1 true WO2017148294A1 (zh) 2017-09-08

Family

ID=59743509

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/074168 WO2017148294A1 (zh) 2016-03-01 2017-02-20 基于移动终端的设备控制方法、装置和移动终端

Country Status (3)

Country Link
US (1) US20180375987A1 (zh)
CN (1) CN107147769B (zh)
WO (1) WO2017148294A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112394893A (zh) * 2019-08-19 2021-02-23 西安诺瓦星云科技股份有限公司 可视化配屏方法、装置、系统以及计算机可读介质
CN113253957A (zh) * 2020-02-10 2021-08-13 西安诺瓦星云科技股份有限公司 画面显示方法、装置、系统以及计算机可读存储介质
CN113285981A (zh) * 2021-04-16 2021-08-20 北京沃东天骏信息技术有限公司 远程控制方法、被控制方法及装置、设备、存储介质
CN113934390A (zh) * 2021-09-22 2022-01-14 青岛海尔科技有限公司 一种投屏的反向控制方法和装置

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108124146B (zh) * 2017-12-20 2020-10-27 联想(北京)有限公司 用于控制电子设备投影的方法、电子设备和投影系统
CN108595137B (zh) * 2018-04-25 2021-05-04 广州视源电子科技股份有限公司 无线投屏方法、装置和投屏器
CN108635856B (zh) * 2018-05-11 2021-05-04 网易（杭州）网络有限公司 游戏中速度反馈的控制方法、装置和计算机可读存储介质
CN108810502A (zh) * 2018-06-04 2018-11-13 深圳汇通九州科技有限公司 一种投影画面校正方法、投影机及计算机可读存储介质
CN111385404B (zh) * 2018-12-28 2021-04-27 Oppo广东移动通信有限公司 一种分体式移动终端及其控制方法、计算机存储介质
CN111385385B (zh) * 2018-12-28 2021-03-30 Oppo广东移动通信有限公司 一种分体式移动终端及其控制方法、计算机存储介质
CN110193204B (zh) * 2019-03-14 2020-12-22 网易（杭州）网络有限公司 操作单位的编组方法、装置、存储介质及电子装置
CN110928509B (zh) * 2019-11-04 2023-06-27 Oppo广东移动通信有限公司 显示控制方法、显示控制装置、存储介质、通信终端
CN111614984A (zh) * 2020-06-17 2020-09-01 广州小海怪科技有限公司 一种投影仪和机顶盒集成系统
CN113031843A (zh) * 2021-04-25 2021-06-25 歌尔股份有限公司 一种手表控制方法、显示终端及手表
CN113312085B (zh) * 2021-04-29 2023-02-03 华人运通(上海)云计算科技有限公司 一种指令块的处理方法、装置、设备及存储介质
WO2022228531A1 (zh) * 2021-04-29 2022-11-03 华人运通(上海)云计算科技有限公司 一种指令块的处理方法、装置、设备、车辆及存储介质
CN113568334A (zh) * 2021-06-21 2021-10-29 海南雷影信息技术有限公司 一种基于雷达的多设备控制方法及系统
CN113434101B (zh) * 2021-06-22 2023-06-09 青岛海尔科技有限公司 设备的控制方法、装置、存储介质及电子装置
CN113596419B (zh) * 2021-07-16 2022-10-14 广景视睿科技（深圳）有限公司 一种控制投影设备的方法、装置及电子设备
CN113709706B (zh) * 2021-08-17 2024-03-26 中汽创智科技有限公司 一种车辆座舱的显示控制方法、装置、系统及终端
CN115756268A (zh) * 2021-09-03 2023-03-07 华为技术有限公司 跨设备交互的方法、装置、投屏系统及终端
CN114389910A (zh) * 2022-01-12 2022-04-22 湖北星纪时代科技有限公司 基于uwb实现的终端控制方法以及终端控制装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090228919A1 (en) * 2007-11-16 2009-09-10 Zott Joseph A Media playlist management and viewing remote control
CN103037256A (zh) * 2011-09-30 2013-04-10 青岛海信电器股份有限公司 移动终端与电视接收终端屏幕共享的方法及电视接收终端
CN103051970A (zh) * 2012-11-09 2013-04-17 青岛海信电器股份有限公司 显示设备与电视机在显示屏幕间的控制方法
CN103957461A (zh) * 2014-04-17 2014-07-30 海信集团有限公司 媒体播放控制的方法、终端及系统
CN104104992A (zh) * 2014-07-08 2014-10-15 深圳市同洲电子股份有限公司 一种多屏互动方法、装置及系统

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1499124A3 (en) * 2003-07-18 2005-05-04 Matsushita Electric Industrial Co., Ltd. Display processing method and display processing apparatus
US9426530B2 (en) * 2006-02-18 2016-08-23 Ol Security Limited Liability Company Methods and apparatus for creating, combining, distributing and reproducing program content for groups of participating users
CN102866771B (zh) * 2011-07-07 2017-06-27 瑞斯康达科技发展股份有限公司 与视频终端设备交互的方法、外部控制平台
US9043841B2 (en) * 2012-08-21 2015-05-26 Sony Corporation Internet TV module for enabling presentation and navigation of non-native user interface on TV having native user interface using either TV remote control or module remote control
KR102131646B1 (ko) * 2013-01-03 2020-07-08 삼성전자주식회사 디스플레이장치 및 그 제어방법
CN103235708A (zh) * 2013-04-17 2013-08-07 东莞宇龙通信科技有限公司 控制显示设备与终端交互的方法和装置
CN104778022B (zh) * 2014-01-09 2019-07-26 联想(北京)有限公司 一种数据处理方法及电子设备
CN104113607B (zh) * 2014-08-04 2017-12-29 联想(北京)有限公司 数据共享方法、电子设备以及数据共享系统
CN104958898A (zh) * 2014-08-13 2015-10-07 腾讯科技（深圳）有限公司 游戏控制方法、装置及系统
JP6566616B2 (ja) * 2014-08-19 2019-08-28 キヤノン株式会社 通信装置、その制御方法、およびプログラム
CN104331243B (zh) * 2014-10-22 2017-11-21 积成电子股份有限公司 基于缩略图重构的移动终端与大屏显示互动控制方法
CN104811780B (zh) * 2015-04-30 2018-04-10 北京奇艺世纪科技有限公司 电子设备控制方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090228919A1 (en) * 2007-11-16 2009-09-10 Zott Joseph A Media playlist management and viewing remote control
CN103037256A (zh) * 2011-09-30 2013-04-10 青岛海信电器股份有限公司 移动终端与电视接收终端屏幕共享的方法及电视接收终端
CN103051970A (zh) * 2012-11-09 2013-04-17 青岛海信电器股份有限公司 显示设备与电视机在显示屏幕间的控制方法
CN103957461A (zh) * 2014-04-17 2014-07-30 海信集团有限公司 媒体播放控制的方法、终端及系统
CN104104992A (zh) * 2014-07-08 2014-10-15 深圳市同洲电子股份有限公司 一种多屏互动方法、装置及系统

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112394893A (zh) * 2019-08-19 2021-02-23 西安诺瓦星云科技股份有限公司 可视化配屏方法、装置、系统以及计算机可读介质
CN112394893B (zh) * 2019-08-19 2023-12-05 西安诺瓦星云科技股份有限公司 可视化配屏方法、装置、系统以及计算机可读介质
CN113253957A (zh) * 2020-02-10 2021-08-13 西安诺瓦星云科技股份有限公司 画面显示方法、装置、系统以及计算机可读存储介质
CN113285981A (zh) * 2021-04-16 2021-08-20 北京沃东天骏信息技术有限公司 远程控制方法、被控制方法及装置、设备、存储介质
CN113285981B (zh) * 2021-04-16 2023-03-07 北京沃东天骏信息技术有限公司 远程控制方法、被控制方法及装置、设备、存储介质
CN113934390A (zh) * 2021-09-22 2022-01-14 青岛海尔科技有限公司 一种投屏的反向控制方法和装置

Also Published As

Publication number Publication date
US20180375987A1 (en) 2018-12-27
CN107147769B (zh) 2020-09-08
CN107147769A (zh) 2017-09-08

Similar Documents

Publication Publication Date Title
WO2017148294A1 (zh) 基于移动终端的设备控制方法、装置和移动终端
US11856322B2 (en) Display apparatus for image processing and image processing method
KR102183413B1 (ko) 콘텐트 표현 방법 및 시스템
US11720179B1 (en) System and method for redirecting content based on gestures
JP6651226B2 (ja) 情報プレゼンテーションアイテムを表示するための方法および装置、ならびにマルチメディア再生デバイス
WO2015192631A1 (zh) 视频会议系统及方法
CN107145278B (zh) 基于移动终端的设备控制方法、装置和移动终端
US20160050449A1 (en) User terminal apparatus, display apparatus, system and control method thereof
CN111405221B (zh) 显示设备及录制文件列表的显示方法
US11917329B2 (en) Display device and video communication data processing method
CN105721904B (zh) 显示装置和控制显示装置的内容输出的方法
CN114286152A (zh) 显示设备、通信终端及投屏画面动态显示方法
CN112073865A (zh) 蓝牙耳机的音量设置方法及装置、电子设备
CN112788422A (zh) 显示设备
CN112399263A (zh) 一种互动方法、显示设备及移动终端
WO2020248714A1 (zh) 一种数据传输方法及设备
WO2020248829A1 (zh) 音视频处理方法及显示设备
WO2020248699A1 (zh) 一种声音处理法及显示设备
CN113489938A (zh) 虚拟会议控制方法、智能设备及终端设备
CN112069094A (zh) 一种数据包封装方法及显示设备
CN115623275A (zh) 一种字幕显示方法及显示设备
KR20190097974A (ko) 전자 장치 및 그 동작 방법
KR102168340B1 (ko) 영상 표시 기기
CN112073812A (zh) 一种智能电视上的应用管理方法及显示设备
CN112071338A (zh) 一种录音控制方法、装置和显示设备

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17759145

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17759145

Country of ref document: EP

Kind code of ref document: A1