KR20200033824A - Methods and apparatus for remote controlling peripheral device in mobile communication terminal - Google Patents

Abstract

The present invention relates to an apparatus for remotely controlling a peripheral device in a mobile communication terminal and a method thereof. According to the present invention, a method for controlling a peripheral device in a mobile communication terminal comprises the processes of: obtaining information on a control target device; configuring a control application for the control target device based on the obtained information; and controlling the control target device using the configured control application.

Description

An apparatus and method for remotely controlling a peripheral device in a mobile communication terminal {METHODS AND APPARATUS FOR REMOTE CONTROLLING PERIPHERAL DEVICE IN MOBILE COMMUNICATION TERMINAL}

The present invention relates to a mobile communication terminal, and more particularly, to an apparatus and method for remotely controlling a peripheral device in a mobile communication terminal.

In the recent IT (information technology) industry, as convergence has become an issue, it is becoming a hot topic to create new consumption contents by interweaving characteristics between devices organically. In addition, the devices are changing from a firmware-level device for the purpose of a single function to a device having a structure that can use various services for various purposes according to the execution of an application by embedding an OS (Operate System). This change is based on the active penetration of smart phones, and is expanding its scope to devices such as smart TVs and tablet PCs.

As devices with OS and communication network interfaces are expanded, communication between devices is possible through various communication methods, and new services through software convergence rather than physical convergence are available. Among the convergence functions, there are services for controlling peripheral devices with a mobile device such as a smart phone. For example, a smart phone can be used to perform TV remote control, home theater control, car remote control, and PC mouse keyboard functions. This is because each remote controller has a physical input device specialized for a corresponding device, so that a user must have a plurality of physical remote controllers in order to control each device, such as a smart phone that the user always has. By moving its role to a mobile device, it is providing users with an environment that can use the functions of multiple remote controllers in one device. Here, the mobile device may collectively refer to various devices having an OS function and a communication network function, such as a mobile phone and a PDA, a smart phone, a tablet PC, and a laptop.

In order to remotely control a target device to be controlled by using a conventional mobile device, a user must connect to an external server through key manipulation to download a control application and install it on the mobile device. In addition, in order to control different control target devices using a conventional mobile device, applications for each control target device must exist in the mobile device. Accordingly, there is a problem that a phenomenon similar to spending time in search of a remote controller does not know where the remote controller is placed in real life.

In addition, the conventional mobile device can only control the unique functions (for example, the volume and channel change function of the TV) of the control target device itself, and the recently emerging smart service, that is, an OS such as a smart TV or a tablet PC There is a problem that the control and interaction (Control & Interaction) corresponding to a service application executed on a device having a built-in device is not supported. In other words, since the conventional mobile device is structured in the form of sending a control signal in one direction like a general TV remote controller or showing only information about a simple driving state, the control function of the conventional mobile device is executed in a device such as a smart TV. The disadvantage is that it cannot provide an optimized control method for various different applications. For example, various applications such as games, explorers, magazines, messengers, and multimedia applications exist in smart TVs, but the functions of general TV remote controllers cannot easily and conveniently control these various applications.

In addition, in order to control various applications for each control target device using a conventional mobile device, the entire process, from designing a program for each control application, to implementing a user interface (UI) and implementing functions, must be developed. And there is a problem that requires a lot of time.

An object of the present invention is to provide an apparatus and method for remotely controlling a peripheral device in a mobile communication terminal.

Another object of the present invention is an apparatus and method for configuring a single integrated control application based on a profile for various controllable devices in a mobile communication terminal, and remotely controlling a controlled device using the configured control application. In providing.

According to a first aspect of the present invention for achieving the above objects, a method for controlling a peripheral device in a mobile communication terminal includes a process of obtaining a profile for a device to be controlled and the control based on the obtained profile And configuring a control application for the target device and controlling the control target device using the configured control application.

According to a second aspect of the present invention, an apparatus for controlling a peripheral device in a mobile communication terminal includes a profile manager for acquiring a profile for a control target device and a control application for the control target device based on the acquired profile It characterized in that it comprises a dynamic configuration manager for configuring, and a control interaction manager for controlling the device to be controlled using the configured control application.

According to the present invention, by configuring one integrated control application based on a profile for various controllable devices in the vicinity, a user downloads a control application for controlling different control target devices by connecting to an external server each time and installs it on a terminal You can reduce the hassle you have to do. In addition, since a list of multiple controllable devices and applications (or services) is managed on one screen, there is no need to select and execute a control application suitable for a control target device among a plurality of control applications installed in a terminal, and one integrated control The application has the advantage of controlling various devices and applications (or services).

In addition, it is possible to easily provide a control application for a new device (service or application) through profile acquisition and update without additional development of device discovery and control protocols and control functions, which can significantly reduce development resources. . In addition, it is not necessary to develop a control application for each model and provide it to users for products that provide various functions specialized for each model even in one product family, so it is possible to respond quickly and conveniently in terms of control application management and distribution. There is an advantage.

1 is a view showing a mobile communication terminal and a peripheral device to be controlled according to the present invention,
2 is a block diagram showing a device configuration of a mobile communication terminal according to the present invention;
3 is a flowchart illustrating a method for remotely controlling a peripheral device in a mobile communication terminal according to an embodiment of the present invention, and
4 is an exemplary view showing an image and arrangement of objects constituting a screen according to an embodiment of the present invention.

Hereinafter, the operation principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, when it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to a user's or operator's intention or practice. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, the present invention will be described for a method for remotely controlling a peripheral device in a mobile communication terminal. Hereinafter, the present invention will be described by using a terminal as an example, but it is needless to say that it can be applied to all mobile devices capable of controlling applications of peripheral devices.

1 is a view showing a mobile communication terminal and a peripheral device to be controlled according to the present invention.

Referring to FIG. 1, the terminal 100 has an input, display, and communication function, and communicates with peripheral devices 110, 120, and 130 to be controlled to control devices 110, 120, and 130. Acquire a profile, configure one integrated control application using the obtained profile, and remotely control an application (or service) of the corresponding control target device (110, 120, 130) using the configured integrated control application.

In addition, although not shown, a profile server manages the profiles of the devices to be controlled, and according to the request of the terminal 100, the profile of the devices to be controlled (110, 120, 130) is the terminal 100 To give. Here, the profile server may be an external server existing outside, or a local server existing in the home or in a certain area. Alternatively, without a separate profile server, the device to be controlled 110, 120, 130 can manage the profile, in which case a direct connection between the terminal 100 and the device to be controlled 110, 120, 130 Through the peer-to-peer method, the profile of the control target devices 110, 120, and 130 may be provided to the terminal 100.

2 is a block diagram showing a device configuration of a mobile communication terminal according to the present invention.

As illustrated, the mobile communication terminal includes a physical communication interface 200, a communication protocol 202, a device link manager 204, a service enabler 206, and a profile manager ( It comprises a Profile Manager (208), a Control Interaction Manager (Control Interaction Manager) 210, a dynamic configuration manager (Dynamic Configuration Manager) 212, a control application (Control Application) 214.

Referring to FIG. 2, the physical communication interface 200 communicates with peripheral control devices through transmission means such as Wi-Fi, Bluetooth, and ZigBee.

The communication protocol 202 includes Universal Plug and Play (UPnP), Digital Living Network Alliance (DLNA), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Simple Network Management Protocol (SNMP), and Radio Frequency for RF4CE. Consumer Electronics).

The device connection manager 204 serves to search for peripheral control target devices. Particularly, the device connection manager 204 broadcasts a search request message to peripheral control target devices using the communication protocol 202 and the physical communication interface 200, and searches the peripheral control target devices from the device. Receive a response message to the request message. Here, the response message is configured to include information about the control target device. For example, the information on the device to be controlled may include various information such as a device name, a model name, a device type (indicating whether control is possible), a network address, and a name and type for services and applications. In addition, the device connection manager 204 extracts device-related information from the response message received for each device to be controlled and displays a list of devices to be controlled on the screen based on the extracted device-related information for each device to be controlled. .

The service enabler 206 serves to search for a service or application performed by peripheral control target devices. In particular, the service enabler 206 broadcasts a search request message to peripheral control target devices by using the communication protocol 202 and the physical communication interface 200, and from the peripheral control target devices The response message to the search request message is received. Here, the response message is configured to include information about a service or application that is currently being performed or changed from the control target device.

The profile manager 208 requests a profile server to transmit the profile of the corresponding device to be controlled when the profile of the device to be controlled does not exist in the memory (not shown), and receives the profile from the profile server. It is stored and managed in a memory (not shown). Here, the profile is configured to include UI information for screen configuration and control command information to be registered in the object, and in addition, it includes a communication protocol, and the UI information includes image information of an object constituting the screen. It consists of the layout (Layout) information. The object may be, for example, various types of objects that can be provided to a user through a terminal such as a button, a touch input, a gesture input, a sensor input, a file list, a video, and music, as shown in FIG. 4.

The control interaction manager 210 remotely controls the control target device using the configured control application 214. That is, when an image of an object disposed on a screen is input from a user, the control interaction manager 210 checks control command information mapped to the object image, and communicates the checked control command information defined in a corresponding profile. It is processed according to the protocol (for example, standards such as TCP, UDP, UPnP, etc.) and transmitted to the corresponding control device. In addition, when information is received from the corresponding control target device, the control interaction manager 210 interprets the received information and compares the interpreted information with registered control command information to perform an appropriate operation.

The dynamic configuration manager 212 analyzes a profile of a control target device and configures a control application 214 for the control target device in real time based on the analysis result. That is, the dynamic configuration manager 212 extracts UI information for screen configuration and control command information to be registered in the object from the profile of the device to be controlled, and then places the image of each object on the screen using the extracted UI information. Then, the extracted control command information is connected to each object, and an image of each object arranged on the screen and control command information for each object are mapped and registered.

3 is a flowchart illustrating a method for remotely controlling a peripheral device in a mobile communication terminal according to an embodiment of the present invention.

Referring to FIG. 3, in step 301, the terminal checks whether a communication control application execution menu is selected according to a user's key operation.

In step 301, when it is determined that the communication control application execution menu is selected, the terminal sends a search request message to surrounding control target devices in order to determine whether the target devices to be controlled exist in a controllable range in step 303. Broadcast. Here, the terminal has various predetermined physical communication interfaces (for example, Wi-Fi, Bluetooth, ZigBee, etc.) and various communication protocols (for example, UPnP, DLNA, TCP, UDP, SNMP, RF4CE, etc.) that can be possessed by each device. By using, broadcast a search request message to peripheral controlled devices. Here, the peripheral control target device includes a device in which a control function for the corresponding device is present in the terminal, and a device in which a control function for the corresponding device is not present in the terminal by newly purchased.

Thereafter, in step 305, the terminal checks whether a response message to the search request message is received from peripheral control target devices. Here, the response message is configured to include information about the control target device. For example, the information on the device to be controlled may include various information such as a device name, a model name, a device type (indicating whether control is possible), a network address, and a name and type for services and applications.

In step 305, when it is determined that the response message for the search request message is received from the peripheral control target devices, the terminal confirms that the target devices to be controlled exist in the controllable range, and in step 307 the peripheral Device-related information is extracted from the response message received for each device to be controlled, and a list of devices to be controlled is displayed on the screen based on the extracted device-related information for each device.

Thereafter, in step 309, the user checks whether one control target device to be controlled is selected from the list of control target devices displayed on the screen according to the user's key manipulation.

In step 309, when it is determined that one control target device to be controlled is selected from the list of control target devices displayed on the screen, the terminal uses the device-related information of the selected control target device in step 311 to store the memory in the memory. It is checked whether a profile of the selected device to be controlled exists.

In step 311, when it is determined that the profile of the selected device to be controlled exists in the memory, the terminal proceeds directly to step 315. Although not shown, if the profile of the selected device to be controlled exists in the memory, the terminal transmits version information of the profile existing in the memory to the profile server, and if there is a newly updated profile in the profile server , It is possible to update a profile existing in the memory by receiving a newly updated profile from the profile server.

On the other hand, if it is determined in step 311 that the profile of the selected device to be controlled does not exist in the memory, the terminal uses the device-related information of the selected device to be controlled in step 313, for example, the device name. Use to request the profile server to transmit the profile of the selected controlled device, and receive the profile of the selected controlled device from the profile server. At this time, the profile server may search for a profile of the selected control target device using device-related information received from the terminal, and transmit the retrieved profile to the terminal.

Thereafter, in step 315, the terminal analyzes a profile of the control target device existing in the memory or received from the profile server, and configures a control application for the selected control target device in real time based on the analysis result to the screen. Display. Here, the profile is configured to include UI information for screen configuration and control command information to be registered in the object, and in addition, it includes a communication protocol, and the UI information includes image information of an object constituting the screen. It consists of the layout (Layout) information. The object may be, for example, various types of objects that can be provided to a user through a terminal such as a button, a touch input, a gesture input, a sensor input, a file list, a video, and music, as shown in FIG. 4. That is, the terminal extracts UI information for screen configuration and control command information to be registered in the object from the profile of the control target device, and then places the image of each object on the screen using the extracted UI information, and extracts the The connected control command information is connected to each object. Thereafter, the terminal registers by mapping the image of each object arranged on the screen and control command information for each object.

Thereafter, the terminal may remotely control the selected control target device according to a user's key manipulation using the configured control application in step 317. That is, when an image of an object placed on a screen is input from a user, the terminal checks control command information mapped to the object image, and the checked communication command information is defined in a corresponding profile (for example, a communication protocol). TCP, UDP, UPnP, etc.) and transmit to the control target device. In addition, when information is received from the corresponding control target device, the terminal interprets the received information and compares the interpreted information with registered control command information to perform an appropriate operation.

Thereafter, the terminal ends the algorithm according to the present invention.

On the other hand, although not shown, while the terminal is remotely controlling a control target device, when the control target device executes an application (or service), the optimal control application capable of interacting with the corresponding application is provided. It can be configured automatically. The terminal can obtain a profile or profile identifier of a service or application that is currently being performed or changed from the device under control (requested from memory or requested by a profile server) and based on this, the user can provide the optimal control function Control applications can be configured in real time. For example, when a PC is searched and selected as a control target device, the terminal may configure a control application related to a mouse and keyboard function of the PC based on the profile of the PC, and remotely control the PC using the control application. At this time, if the PC is running an application such as a media player (Media Player) or a power point (Power Point), the PC transmits the profile or profile identifier of the application to the terminal, so that the terminal obtains the profile (from memory Search or request to a profile server) and based on this, a control application that can provide an optimal control function to a user can be configured in real time. That is, when the media player is running on the PC, the terminal provides the user with a control function for the media player based on a profile containing information such as play, pause, volume control, etc., and when the PC runs PowerPoint, The terminal may provide control functions such as a presentation mode change, a previous page, and a next page.

On the other hand, in the detailed description of the present invention, although specific embodiments have been described, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of the claims to be described later, but also by the scope and equivalents of the claims.

Claims (20)

In the electronic device,
One or more memories storing instructions;
Communication interface;
touch screen; And
And one or more processors functionally coupled to the one or more memories, the communication interface and the touch screen, wherein the one or more processors are:
Receiving an input for selecting one external electronic device among one or more external electronic devices through the touch screen,
Receiving information about a first application currently being driven by the selected external electronic device from the selected external electronic device,
A user interface of a second application for controlling the selected external electronic device is displayed on the touch screen,
On the user interface, at least one object selectable in relation to at least one function of the running first application is displayed on the selected external electronic device,
In response to receiving an input for selecting one of the at least one object through the touch screen, a signal for executing a function indicated by the selected object in the selected external electronic device is selected from the selected external An electronic device configured to execute the stored instructions for transmission to an electronic device.
The method of claim 1, wherein the one or more processors,
On the other user interface of the second application for controlling the selected external electronic device displayed on the touch screen, display one or more objects for selecting the one or more external electronic devices including the selected external electronic device,
In response to detecting an input to one of the one or more objects, information about the running first application of the selected external electronic device indicated by the one object may be An electronic device configured to execute the stored instructions to request a selected external electronic device.
The method of claim 2, wherein the one or more processors,
Broadcast a message to search for the one or more external electronic devices located around the electronic device,
Receive one or more messages including information about the one or more external electronic devices from the one or more external electronic devices,
In response to receiving the one or more messages, an electronic device configured to execute the stored instructions to display the one or more objects pointing to the one or more external electronic devices on the other user interface of the second application.
According to claim 3, The information about the one or more external electronic devices,
Data for a device name of the one or more external electronic devices, data for a model name of the one or more external electronic devices, data for a network address of the one or more external electronic devices, the one or more external An electronic device comprising at least one of data for a service name of electronic devices, or data for a service type of the one or more external electronic devices.
The method of claim 1, wherein the one or more processors,
An electronic device that is set to execute the stored commands to confirm whether data for controlling the first application being driven is stored in the electronic device.
The method of claim 5, wherein the one or more processors,
Upon identifying that the data is not stored in the electronic device, an electronic device further configured to execute the stored instructions to request the data from a server or the selected external electronic device.
The method of claim 6, wherein the data,
And user interface data for the user interface,
The user interface data,
An electronic device including at least one of image information on the at least one object or layout information on an arrangement of the at least one object.
The method of claim 7, wherein the one or more processors,
And an electronic device configured to execute the stored instructions to display the at least one object disposed based on the data on the user interface.
In the operation method of the electronic device,
Receiving an input for selecting one external electronic device among one or more external electronic devices through a touch screen of the electronic device;
Receiving information about a first application currently being driven by the selected external electronic device from the selected external electronic device;
Displaying a user interface of a second application for controlling the selected external electronic device on the touch screen;
At least one selectable in relation to at least one function of the first application being driven by the selected external electronic device, on the user interface, based on information about the first application being driven by the selected external electronic device The action of displaying the object,
In response to receiving an input for selecting one of the at least one object, transmitting a signal for executing a function indicated by the selected object in the selected external electronic device to the external device How to include.
10. The method of claim 9, The operation of receiving information about the first application,
Displaying one or more objects for selecting the one or more external electronic devices including the selected external electronic device on another user interface of the second application for controlling the selected external electronic device displayed on the touch screen and,
In response to detecting an input to one of the one or more objects, information about the running first application of the selected external electronic device indicated by the one object may be And further comprising a request to the selected external electronic device.
11. The method of claim 10, The operation of displaying the one or more objects,
Broadcasting a message to search for the one or more external electronic devices located around the electronic device;
Receiving one or more messages including information on the one or more external electronic devices from the one or more external electronic devices;
And in response to receiving the one or more messages, displaying the one or more objects indicating the one or more external electronic devices on the other user interface of the second application.
12. The method of claim 11, The information about the one or more external electronic devices,
Data for a device name of the one or more external electronic devices, data for a model name of the one or more external electronic devices, data for a network address of the one or more external electronic devices, the one or more external A method comprising at least one of data for a service name of electronic devices, or data for a service type of the one or more external electronic devices.
10. The method of claim 9, The operation of displaying the at least one object,
And checking whether data for controlling the first application being driven is stored in the electronic device.
The method of claim 13,
And upon requesting that the data is not stored in the electronic device, requesting the data from a server or the selected external electronic device.
The method of claim 14, wherein the data,
And user interface data for the user interface,
The user interface data,
And at least one of image information on the at least one object or layout information on an arrangement of the at least one object.
The method of claim 15, wherein the operation of displaying the at least one object,
And displaying the at least one object disposed based on the data on the user interface.
In a non-transitory computer-readable storage medium,
Receiving information about a first application currently being driven by the external electronic device from an external electronic device;
On the user interface of the second application for controlling the external electronic device, based on the information on the first application running in the external electronic device, the first application running in the external electronic device To control, displaying at least one object selectable in relation to at least one function of the first application being driven by the external electronic device;
In response to receiving an input for selecting one of the at least one object, one or more for performing an operation of transmitting a signal for executing a function indicated by the selected object in the external electronic device A medium for storing programs.
The method of claim 17, wherein the operation of receiving information about the first application,
Displaying on the other user interface of the second application for controlling the external electronic device, one or more objects for selecting one or more external electronic devices including the external electronic device;
In response to detecting an input to one of the one or more objects, information about the running first application of the external electronic device indicated by the one object is external to the object. A medium further comprising a request to the electronic device.
The method of claim 17, wherein the operation of displaying the at least one object,
Checking whether data for controlling the first application being driven is stored in the electronic device;
And upon requesting that the data is not stored in the electronic device, requesting the data from a server or the external electronic device.
The method of claim 19, wherein the data,
And user interface data for the user interface,
The user interface data,
A medium including at least one of image information on the at least one object or layout information on an arrangement of the at least one object.

Images