KR102269276B1 - Electronic device, and method for operating the same - Google Patents

Abstract

A method of operating an electronic device according to an embodiment of the present invention includes the steps of connecting to a first electronic device through a network, performing capability negotiation with the first electronic device, and negotiating the function with the first electronic device Transmitting content data in a format based on , transmitting control image data including one or more control menu buttons to a first electronic device, receiving user input information for control image data from the first electronic device and performing a control function corresponding to user input information. Accordingly, the user's convenience can be improved by allowing the electronic devices to easily and conveniently control other devices with one device while sharing content.

Description

Electronic device, and its operation method {ELECTRONIC DEVICE, AND METHOD FOR OPERATING THE SAME}

The present invention relates to an electronic device such as a portable terminal and an image display device and an operating method thereof, and more particularly, by allowing the electronic devices to easily and conveniently control other devices with one device while sharing contents, the user's convenience is improved It relates to an electronic device that can be used and a method for operating the same.

A portable terminal is a portable device equipped with one or more functions such as a function for performing voice and video calls, a function for inputting and outputting information, and a function for storing data while being portable. As the functions of these mobile terminals are diversified, they are equipped with complex functions such as taking pictures or moving pictures, playing music files or video files, receiving games, broadcasting, wireless Internet, sending and receiving messages, etc., and comprehensive multimedia devices ( multimedia player). New attempts are being variously applied to the portable terminal implemented in the form of such a multimedia device in terms of hardware or software in order to implement complex functions.

In addition, the image display device is a device having a function of displaying an image that a user can watch. The user can watch the broadcast through the image display device. The image display device displays a broadcast selected by a user from among broadcast signals transmitted from a broadcast station on a display. Currently, the trend of broadcasting worldwide is converting from analog broadcasting to digital broadcasting. Digital broadcasting refers to broadcasting that transmits digital image and audio signals. Compared to analog broadcasting, digital broadcasting is strong against external noise, so data loss is small, it is advantageous for error correction, and provides a high resolution and clear screen. Also, unlike analog broadcasting, digital broadcasting can provide interactive services.

Recently, network technology has been developed in which various electronic devices are connected to each other so that people can enjoy convenient, safe, and economical service at any time, both at home and outside.

In addition, electronic devices can be connected to each other using wired or wireless communication, users can copy or move content between the connected electronic devices, output the same content screen to a plurality of electronic devices, or store content. Contents can be output to the screen of other devices that are not.

SUMMARY OF THE INVENTION An object of the present invention is to provide an electronic device capable of improving user convenience and an operating method thereof by easily and conveniently controlling screens of a plurality of connected electronic devices.

In order to achieve the above object, a method of operating an electronic device according to an embodiment of the present invention includes the steps of connecting to a first electronic device through a network, performing capability negotiation with the first electronic device, a first transmitting content data in a format based on the function negotiation to an electronic device; transmitting control image data including one or more control menu buttons to a first electronic device; and receiving user input information for the information and performing a control function corresponding to the user input information.

In order to achieve the above object, a method of operating an electronic device according to an embodiment of the present invention includes the steps of connecting to a second electronic device through a network, performing capability negotiation with the second electronic device, and a second Receiving content data in a format based on the function negotiation from an electronic device, receiving control image data including one or more control menu buttons from a second electronic device, one based on the received control image data The method includes displaying a control image including the above control menu button, receiving a user input for the control image, and transmitting user input information for the control image data to a second electronic device.

According to the present invention, the user's convenience can be improved by easily and conveniently controlling another device with one device while the electronic devices share content.

1 is a diagram illustrating examples of electronic devices connectable through a wired/wireless network according to an embodiment of the present invention.
2 is an internal block diagram of an image display device according to an embodiment of the present invention.
FIG. 3 is an internal block diagram of the control unit of FIG. 2 .
4 is a block diagram of a portable terminal according to an embodiment of the present invention.
5 is a flowchart illustrating an operation method of an electronic device capable of controlling another electronic device according to the related art.
6 is a flowchart illustrating an embodiment of a method of operating an electronic device according to the present invention.
7 is a flowchart illustrating an embodiment of a method of operating an electronic device according to the present invention.
8 to 16 are diagrams referenced to explain various embodiments of a method of operating an electronic device according to the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffixes "module" and "part" for the components used in the following description are given simply in consideration of the ease of writing the present specification, and do not give a particularly important meaning or role by themselves. Accordingly, the terms “module” and “unit” may be used interchangeably.

1 is a diagram illustrating examples of electronic devices connectable through a wired/wireless network according to an embodiment of the present invention.

The electronic devices of FIG. 1 are exemplary and the present invention is not limited thereto. Referring to FIG. 1 , electronic devices 100a, 100b, 300, 400, such as TVs 100a and 100b, a tablet PC 300, a notebook computer 400, and a smart phone 500 according to an embodiment of the present invention; 500a and 500b) may transmit or receive data through a wired/wireless communication network with each other, and may share data.

In addition, electronic devices according to the present invention can share and output content with other electronic devices on a 1:1 basis, and share and output content with a plurality of electronic devices 1:N or N:N. can

Meanwhile, according to an embodiment of the present invention, electronic devices may be directly connected to each other or may be connected via a predetermined server.

Meanwhile, in FIG. 1 , TVs 100a and 100b are exemplified as representative examples of image display devices, and smart phones 500 are exemplified as representative examples of portable terminals, but the present invention is not limited thereto.

In addition, in addition to the TV (100a, 100b) and the monitor, which are representative image display devices, the tablet PC 300, a notebook computer (400), a smart phone (smart phone, 500a, 500b), a navigation device, etc. are electric devices having a display. may correspond to the image display device 100 in the present specification.

In addition, since the tablet PC 300 , the notebook computer 400 , the smart phones 500a and 500b , the mobile phone, and the PDA are portable, they may correspond to the portable terminal 500 in the present specification.

2 is an internal block diagram of the image display device 100 according to an embodiment of the present invention.

Referring to FIG. 2 , the image display device 100 according to an embodiment of the present invention includes a broadcast receiving unit 105 , a network interface unit 130 , an external device interface unit 130 , a storage unit 140 , and a user input. It may include an interface unit 150 , a control unit 170 , a display 180 , and an audio output unit 185 .

The image display device 100 according to an embodiment of the present invention may further include an NFC module 125, a sensor unit (not shown), a camera (not shown), and the like, if necessary.

The broadcast receiver 105 may include a tuner 110 and a demodulator 120 . In addition, if necessary, the broadcast receiving unit 105 may be designed to include the tuner unit 110 and the demodulator 120 while not including the network interface unit 130 , and vice versa. It is also possible to design such that the tuner unit 110 and the demodulator unit 120 are not included.

The tuner unit 110 selects an RF broadcast signal corresponding to a channel selected by a user or all channels pre-stored from among radio frequency (RF) broadcast signals received through an antenna. In addition, the selected RF broadcast signal is converted into an intermediate frequency signal or a baseband video or audio signal.

For example, if the selected RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF), and if it is an analog broadcast signal, it is converted into an analog baseband image or audio signal (CVBS/SIF). That is, the tuner unit 110 may process a digital broadcast signal or an analog broadcast signal. An analog baseband video or audio signal (CVBS/SIF) output from the tuner unit 110 may be directly input to the control unit 170 .

Also, the tuner unit 110 may receive a single-carrier RF broadcast signal according to an Advanced Television System Committee (ATSC) method or a multi-carrier RF broadcast signal according to a Digital Video Broadcasting (DVB) method.

Meanwhile, in the present invention, the tuner unit 110 sequentially selects RF broadcast signals of all broadcast channels stored through the channel storage function among the RF broadcast signals received through the antenna in the present invention, and converts them to an intermediate frequency signal or a baseband image or audio signal. can be converted to

Meanwhile, the tuner unit 110 may include a plurality of tuners in order to receive broadcast signals of a plurality of channels. Alternatively, a single tuner that simultaneously receives broadcast signals of a plurality of channels is also possible.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation.

The demodulator 120 may output a stream signal TS after demodulation and channel decoding are performed. In this case, the stream signal may be a signal obtained by multiplexing an image signal, an audio signal, or a data signal.

The stream signal output from the demodulator 120 may be input to the controller 170 . After performing demultiplexing, image/audio signal processing, and the like, the controller 170 outputs an image to the display 180 , and outputs an audio to the audio output unit 185 .

The external device interface unit 130 may transmit or receive data with the connected external device 190 . To this end, the external device interface unit 130 may include an A/V input/output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 130 may be connected to an external device such as a DVD (Digital Versatile Disk), Blu-ray, game device, camera, camcorder, computer (notebook), set-top box, etc. by wire/wireless, , it is also possible to perform input/output operations with an external device.

The A/V input/output unit may receive video and audio signals from an external device. Meanwhile, the wireless communication unit may perform short-range wireless communication with other electronic devices.

The network interface unit 135 provides an interface for connecting the image display device 100 to a wired/wireless network including an Internet network. For example, the network interface unit 135 may receive content or data provided by the Internet or a content provider or network operator through a network.

Meanwhile, the network interface unit 135 may perform data communication with adjacent electronic devices around the image display device 100 . In this case, the network interface unit 135 may receive device information of other electronic devices, remotely controllable channel information of other electronic devices, frequency information or code information, and the like.

Meanwhile, the image display device 100 may include an NFC module 125 to enable NFC communication with other electronic devices such as the mobile terminal 500 .

Meanwhile, it is also possible to receive device information and network information of other electronic devices through the NFC module 125 . Thereafter, based on such other electronic device information, the corresponding electronic device may be network-connected.

The storage unit 140 may store a program for each signal processing and control in the control unit 170 , or may store a signal-processed image, audio, or data signal.

Also, the storage unit 140 may perform a function for temporarily storing an image, audio, or data signal input to the external device interface unit 130 . Also, the storage unit 140 may store information about a predetermined broadcast channel through a channel storage function such as a channel map.

Although the embodiment in which the storage unit 140 of FIG. 2 is provided separately from the control unit 170 is illustrated, the scope of the present invention is not limited thereto. The storage unit 140 may be included in the control unit 170 .

The user input interface unit 150 transmits a signal input by the user to the control unit 170 or transmits a signal from the control unit 170 to the user.

For example, transmitting/receiving user input signals such as power on/off, channel selection, and screen setting from the remote control device 200, or local keys such as power key, channel key, volume key, and setting value (not shown) transmits a user input signal input to the control unit 170, transmits a user input signal input from a sensor unit (not shown) that senses a user's gesture to the control unit 170, or transmits a signal from the control unit 170 It can be transmitted to a sensor unit (not shown).

According to an embodiment of the present invention, the user input interface 150 may receive personal information from the remote control device 200 . And, in addition to the personal information, it is also possible to further receive information about the web server being accessed by the personal information.

For example, when the portable terminal 500 and the remote control device 200 approach within a predetermined distance to perform near-field communication (NFC), the remote control device 200 provides the personal information stored in the portable terminal 500 . information can be received. In addition, the remote control device 200 may transmit personal information to the image display device 100 according to an IR method or an RF method. In this case, the user input interface unit 150 transmits the received personal information to the control unit 170 .

Meanwhile, the personal information at this time may include personal ID information, password information, personal email information, and the like for the portable terminal 500 . Alternatively, the personal information may include personal ID information, password information, personal email information, and the like for the image display device 100 . Alternatively, the personal information may be personal ID information or password information that can be used in common in various electronic devices including the portable terminal 500 . Alternatively, the personal information may be personal ID information, password information, or the like, from the portable terminal 500 to a predetermined web server stored in advance. Alternatively, the personal information may be personal ID information, password information, or the like, to a predetermined web server that can be used in the image display device 100 including the portable terminal 500 . Alternatively, the personal information may be personal ID information, password information, etc. for a server connected to the image display device 100 .

That is, the personal information may be information required to log in to a predetermined account for an electronic device or service.

On the other hand, the user input interface unit 150 is transmitted from the remote control device 200, from adjacent electronic devices around the image display device 100, device information of other electronic devices, and remote controllable channel information of other electronic devices. , frequency information or code information may be received.

The control unit 170 demultiplexes an input stream or processes the demultiplexed signals through the tuner unit 110 or the demodulator 120 or the external device interface unit 130 for video or audio output. It can generate and output signals.

The image signal processed by the controller 170 may be input to the display 180 and displayed as an image corresponding to the image signal. Also, the image signal processed by the controller 170 may be input to an external output device through the external device interface unit 130 .

The audio signal processed by the control unit 170 may be outputted to the audio output unit 185 . Also, the audio signal processed by the controller 170 may be input to an external output device through the external device interface unit 130 .

Although not shown in FIG. 2 , the controller 170 may include a demultiplexer, an image processor, and the like. This will be described later with reference to FIG. 3 .

In addition, the controller 170 may control overall operations in the image display apparatus 100 . For example, the controller 170 may control the tuner 110 to select a channel selected by the user or an RF broadcast corresponding to a pre-stored channel (tuning).

Also, the control unit 170 may control the image display apparatus 100 according to a user command input through the user input interface unit 150 or an internal program.

Meanwhile, the controller 170 may control the display 180 to display an image. In this case, the image displayed on the display 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

Meanwhile, the controller 170 may generate and display a 3D object for a predetermined object among the images displayed on the display 180 . For example, the object may be at least one of an accessed web screen (newspaper, magazine, etc.), an electronic program guide (EPG), various menus, widgets, icons, still images, moving pictures, and text.

Meanwhile, the controller 170 may recognize the location of the user based on the image captured by the photographing unit (not shown). For example, the distance (z-axis coordinate) between the user and the image display apparatus 100 may be determined. In addition, an x-axis coordinate and a y-axis coordinate in the display 180 corresponding to the user's location may be identified.

Meanwhile, according to an embodiment of the present invention, the controller 170 may control to perform a login based on the personal information received from the remote control device 200 . In this case, the log-in may be a log-in of the image display device 100 itself, a log-in to a server connected to the image display device 100, or a log-in to a predetermined web server subscribed by personal information.

For example, when logging into the video display device 100 itself by personal ID information and password information received from the remote control device 200, the controller 170 displays a personal screen set for each user account. It can be controlled to display at (180). When there is an image being viewed, the controller 170 may control to display a personal setting screen together with the image being viewed. Alternatively, it is possible to control the video being viewed to be displayed as a personal setting screen by changing the scene.

As another example, when logging in to a server connected to the image display device 100 , the controller 170 may control to display a server connection screen received from the corresponding server on the display 180 . Specifically, it may be an app server (app server) screen. When there is an image being viewed, the controller 170 may control to display a server connection screen together with the image being viewed. Alternatively, it is possible to control the video being viewed to be displayed on the server connection screen by changing the scene.

As another example, when the received personal information is personal information about predetermined web servers being accessed by another electronic device, specifically, the mobile terminal 500 , the controller 170 may control the image display device 100 to connect the web servers. It is possible to control to access the web server, and to display a screen for the accessed web server on the display 180 . In this case, the web servers may be web servers that provide social network services. When there is an image being viewed, the controller 170 may control to display a screen of the connected web server together with the image being viewed. Alternatively, it is possible to control the video being viewed to be displayed on the screen of the connected web server by changing the scene.

On the other hand, when the power of the image display apparatus 100 is in an off state in a state in which the personal information is received, the controller 170 controls the power supply unit (not shown) to turn on the power of the image display apparatus 100 . can That is, in the standby mode, when personal information is received from the user input interface unit 150, the control unit 170 switches to a wake up mode, and powers to supply power to various modules or units. You can control the supply.

On the other hand, the control unit 170, received from the network interface unit 135 or the user input interface unit 150, the image display device 100 peripheral device information, setting information, and remote control of other electronic devices. Based on controllable channel information, frequency information or code information, it is determined whether each electronic device is connected according to personal information from the remote control device 200, and an object indicating that another electronic device is logged in is displayed. It can be controlled to display at (180).

Alternatively, the control unit 170, the remote control device 200, based on device information, setting information, remote controllable channel information of other electronic devices, frequency information or code information of adjacent electronic devices in the vicinity of the image display device 100, etc. ), it is possible to control to transmit the corresponding device information, setting information, channel information, frequency information, or code information.

Meanwhile, although not shown in the drawings, a channel browsing processing unit for generating a thumbnail image corresponding to a channel signal or an external input signal may be further provided. The channel browsing processing unit receives the stream signal TS output from the demodulator 120 or the stream signal output from the external device interface unit 130, and extracts an image from the input stream signal to generate a thumbnail image. can The generated thumbnail image may be stream-decoded together with the decoded image and the like and input to the controller 170 . The controller 170 may display a thumbnail list including a plurality of thumbnail images on the display 180 by using the input thumbnail images.

In this case, the thumbnail list may be displayed in a simple view method displayed on a partial area while a predetermined image is displayed on the display 180 , or may be displayed in a full view method displayed on most areas of the display 180 . Thumbnail images in such a thumbnail list may be sequentially updated.

The display 180 converts an image signal, a data signal, an OSD signal, a control signal, or an image signal, a data signal, and a control signal received from the external device interface unit 130 processed by the controller 170 to generate a driving signal. create

The display 180 may be a PDP, LCD, OLED, flexible display, or the like, and may also be a three-dimensional display (3D display).

Meanwhile, the display 180 may be configured as a touch screen and used as an input device in addition to an output device.

The audio output unit 185 receives the audio-processed signal from the control unit 170 and outputs it as audio.

The photographing unit (not shown) photographs the user. The photographing unit (not shown) may be implemented with one camera, but is not limited thereto, and may be implemented with a plurality of cameras. Meanwhile, the photographing unit (not shown) may be embedded in the image display device 100 above the display 180 or disposed separately. Image information captured by the photographing unit (not shown) may be input to the control unit 170 .

The controller 170 may detect the user's gesture based on each or a combination of an image captured by a photographing unit (not shown) or a signal sensed from a sensor unit (not shown).

The remote control device 200 transmits a user input to the user input interface unit 150 . To this end, the remote control device 200, Bluetooth (Bluetooth), RF (Radio Frequency) communication, infrared (IR) communication, UWB (Ultra Wideband), ZigBee (ZigBee) method, near magnetic field communication (NFC) method, etc. Can be used.

Also, the remote control device 200 may receive an image, audio, or data signal output from the user input interface unit 150 . Also, the remote control device 200 may display or output audio from the remote control device 200 based on a received image, audio, or data signal.

Meanwhile, the remote control device 200 according to an embodiment of the present invention may receive personal information through short-range magnetic field communication with a predetermined electronic device. As described above, the personal information may be personal ID information and password information that can be used in the portable terminal 500 , the image display device 200 , or an electronic device. At this time, the personal information may be different for each device, but in the embodiment of the present invention, one integrated personal ID and password will be mainly described.

Meanwhile, the remote control device 200 may transmit the received personal information to the image display device 100 . The communication method at this time may use the IR method or the RF method.

In the embodiment of the present invention, the remote control device 200 will be described with a focus on the spatial remote control capable of displaying a pointer corresponding to the user's movement. That is, the remote control device 200 may transmit personal information and the like to the image display device 100 using the RF method.

Meanwhile, the remote control device 200 may further receive, in addition to personal information, web server information being accessed by personal information. For example, the mobile terminal 500 may receive web server information about a social network service that is being logged in and connected. Such web server information is also transmitted to the image display device 100 .

Meanwhile, it is also possible to receive device information of other electronic devices, remotely controllable channel information of other electronic devices, frequency information or code information, and the like, from adjacent electronic devices around the image display device 100 . Based on such other electronic device information, a channel, a frequency, or a code may be allocated to the corresponding electronic device to perform remote control.

Meanwhile, the above-described image display device 100 may be a digital broadcasting receiver capable of receiving fixed or mobile digital broadcasting.

Meanwhile, the block diagram of the image display device 100 shown in FIG. 2 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the image display device 100 that are actually implemented. That is, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed. In addition, the function performed in each block is for explaining the embodiment of the present invention, and the specific operation or device does not limit the scope of the present invention.

On the other hand, the image display apparatus 100 does not include the tuner unit 110 and the demodulator 120 shown in FIG. 2 , unlike the one shown in FIG. 2 , but a network interface unit 130 or an external device interface unit ( 135), the image content may be received and it may be reproduced.

FIG. 3 is an internal block diagram of the control unit of FIG. 2 .

Referring to the drawings, the controller 170 according to an embodiment of the present invention includes a demultiplexer 310 , an image processor 320 , a processor 330 , an OSD generator 340 , and a mixer 345 . , a frame rate converter 350 , and a formatter 360 . In addition, it may further include a voice processing unit (not shown) and a data processing unit (not shown).

The demultiplexer 310 demultiplexes an input stream. For example, when MPEG-2 TS is input, it can be demultiplexed and separated into video, audio, and data signals, respectively. Here, the stream signal input to the demultiplexer 310 may be a stream signal output from the tuner 110 , the demodulator 120 , or the external device interface unit 130 .

The image processing unit 320 may perform image processing on the demultiplexed image signal. To this end, the image processing unit 320 may include an image decoder 325 and a scaler 335 .

The video decoder 325 decodes the demultiplexed video signal, and the scaler 335 performs scaling to output the resolution of the decoded video signal on the display 180 .

The video decoder 325 may include decoders of various standards.

The processor 330 may control overall operations in the image display apparatus 100 or in the controller 170 . For example, the processor 330 may control the tuner 110 to select a channel selected by the user or an RF broadcast corresponding to a pre-stored channel (Tuning).

Also, the processor 330 may control the image display apparatus 100 according to a user command input through the user input interface unit 150 or an internal program.

Also, the processor 330 may perform data transmission control with the network interface unit 135 or the external device interface unit 130 .

Also, the processor 330 may control operations of the demultiplexer 310 , the image processor 320 , the OSD generator 340 , and the like in the controller 170 .

The OSD generator 340 generates an OSD signal according to a user input or by itself. For example, a signal for displaying various types of information as graphics or text on the screen of the display 180 may be generated based on a user input signal. The generated OSD signal may include various data such as a user interface screen of the image display device 100 , various menu screens, widgets, and icons. Also, the generated OSD signal may include a 2D object or a 3D object.

Also, the OSD generator 340 may generate a pointer that can be displayed on a display based on a pointing signal input from the remote control device 200 . In particular, such a pointer may be generated by a pointing signal processing unit, and the OSD generation unit 240 may include such a pointing signal processing unit (not shown). Of course, the pointing signal processing unit (not shown) may be provided separately instead of being provided in the OSD generating unit 240 .

Meanwhile, in relation to an embodiment of the present invention, the OSD generating unit 340 may create or configure a set personal screen when logging into the image display device 100 itself. Alternatively, the OSD generating unit 340 may generate or configure at least a part of the server access screen to display the server access screen received from the corresponding server on the display 180 when logging into the accessed server. Alternatively, the OSD generating unit 340 may generate or configure at least a part of the web server access screen based on the web server information being accessed by personal information.

The mixer 345 may mix the OSD signal generated by the OSD generator 340 and the decoded image signal image-processed by the image processor 320 . In this case, the OSD signal and the decoded image signal may include at least one of a 2D signal and a 3D signal, respectively. The mixed video signal is provided to the frame rate converter 350 .

A frame rate converter (FRC) 350 may convert a frame rate of an input image. On the other hand, the frame rate converter 350 may output as it is without a separate frame rate conversion.

The formatter 360 receives the signal mixed by the mixer 345 , that is, the OSD signal and the decoded image signal, changes the format of the signal to be suitable for the display 180 , and outputs the changed signal. For example, R, G, and B data signals may be output, and the R, G, and B data signals may be output as low voltage differential signaling (LVDS) or mini-LVDS.

Meanwhile, the formatter 360 may separate a 2D image signal and a 3D image signal to display a 3D image. Also, the format of the 3D image signal may be changed or the 2D image signal may be converted into a 3D image signal.

Meanwhile, a voice processing unit (not shown) in the control unit 170 may perform voice processing of the demultiplexed voice signal. To this end, the voice processing unit (not shown) may include various decoders.

In addition, the voice processing unit (not shown) in the control unit 170 may process a base (Base), a treble (Treble), volume control, and the like.

A data processing unit (not shown) in the control unit 170 may perform data processing of the demultiplexed data signal. For example, when the demultiplexed data signal is an encoded data signal, it may be decoded. The encoded data signal may be EPG (Electronic Program Guide) information including broadcast information such as start time and end time of a broadcast program aired on each channel.

Meanwhile, in FIG. 3 , signals from the OSD generating unit 340 and the image processing unit 320 are mixed in the mixer 345 , and then 3D processing is performed in the formatter 360 , but the present invention is not limited thereto. It is also possible that the is placed after the formatter.

Meanwhile, the block diagram of the controller 170 shown in FIG. 3 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specification of the controller 170 that is actually implemented.

In particular, the frame rate converter 350 and the formatter 360 are not provided in the controller 170, but may be provided separately.

4 is a block diagram of a portable terminal according to an embodiment of the present invention.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a camera, a navigation system, a tablet computer ( tablet computer), an e-book terminal, and the like.

4 is a block diagram of a portable terminal according to an embodiment of the present invention. Referring to FIG. 4 , a mobile terminal according to an embodiment of the present invention will be described in terms of components according to functions as follows.

Referring to FIG. 4 , the mobile terminal 500 includes a wireless communication unit 510 , an audio/video (A/V) input unit 520 , a user input unit 530 , a sensing unit 540 , an output unit 550 , and a memory. 560 , an interface unit 570 , a control unit 580 , and a power supply unit 590 may be included. When these components are implemented in an actual application, two or more components may be combined into one component, or one component may be subdivided into two or more components as needed.

The wireless communication unit 510 may include a broadcast reception module 511 , a mobile communication module 513 , a wireless Internet module 515 , a short-range communication module 517 , and a GPS module 519 .

The broadcast receiving module 511 receives at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. In this case, the broadcast channel may include a satellite channel, a terrestrial channel, and the like. The broadcast management server may refer to a server that generates and transmits at least one of a broadcast signal and broadcast-related information, or a server that receives at least one of a previously generated broadcast signal and broadcast-related information and transmits it to a terminal.

The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal. The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. Broadcast-related information may be provided through a mobile communication network, and in this case, may be received by the mobile communication module 513 . Broadcast-related information may exist in various forms.

The broadcast receiving module 511 receives a broadcast signal using various broadcast systems. In particular, Digital Multimedia Broadcasting-Terrestrial (DMB-T), Digital Multimedia Broadcasting-Satellite (DMB-S), Media Forward Link Only (MediaFLO) ), Digital Video Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like can be used to receive digital broadcast signals. In addition, the broadcast receiving module 511 may be configured to be suitable for all broadcast systems that provide broadcast signals as well as such digital broadcast systems. The broadcast signal and/or broadcast related information received through the broadcast reception module 511 may be stored in the memory 560 .

The mobile communication module 513 transmits/receives a wireless signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice call signal, a video call call signal, or various types of data according to text/multimedia message transmission/reception.

The wireless Internet module 515 refers to a module for wireless Internet access, and the wireless Internet module 515 may be built-in or external to the portable terminal 500 . As wireless Internet technologies, wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), etc. may be used.

The short-range communication module 517 refers to a module for short-range communication. Bluetooth, RFID (Radio Frequency Identification), infrared data association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), etc. may be used as short-range communication technologies.

The Global Position System (GPS) module 519 receives location information from a plurality of GPS satellites.

The A/V (Audio/Video) input unit 520 is for inputting an audio signal or a video signal, and may include a camera 521 , a microphone 523 , and the like. The camera 521 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a photographing mode. Then, the processed image frame may be displayed on the display 551 .

The image frame processed by the camera 521 may be stored in the memory 560 or transmitted to the outside through the wireless communication unit 510 . Two or more cameras 521 may be provided according to the configuration of the terminal.

The microphone 523 receives an external audio signal by a microphone in an audio reception mode, for example, a call mode, a recording mode, or a voice recognition mode, and processes it as electrical voice data. In addition, the processed voice data may be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 513 and output in the call mode. Various noise removal algorithms may be used for the microphone 123 to remove noise generated in the process of receiving an external audio signal.

Meanwhile, the microphones 523 may be disposed as a plurality at different positions. The audio signal received from each microphone may be processed by the controller 580 or the like.

The user input unit 530 generates key input data input by the user to control the operation of the terminal. The user input unit 530 may include a keypad, a dome switch, a touch pad (static pressure/capacitance), and the like, through which a command or information can be input by a user's pressing or touch manipulation. Also, the user input unit 530 may be configured as a jog wheel or a jog method for rotating a key, a method for operating like a joystick, or a finger mouse. In particular, when the touch pad forms a layer structure with the display 551 to be described later, it may be referred to as a touch screen.

The sensing unit 540 is configured to control the operation of the mobile terminal 500 by sensing the current state of the mobile terminal 500 such as the opening/closing state of the mobile terminal 500 , the location of the mobile terminal 500 , and the presence or absence of user contact. Generates a sensing signal. For example, when the portable terminal 500 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In addition, it may be responsible for sensing functions related to whether the power supply unit 590 is supplied with power, whether the interface unit 570 is coupled to an external device, and the like.

The sensing unit 540 may include a proximity sensor 541 , a pressure sensor 543 , and a motion sensor 545 . The proximity sensor 541 detects the presence or absence of an object approaching the portable terminal 500 or an object existing in the vicinity of the portable terminal 500 without mechanical contact. The proximity sensor 541 may detect a proximity object using a change in an alternating current or static magnetic field, or a rate of change in capacitance. Two or more proximity sensors 541 may be provided depending on the configuration aspect.

The pressure sensor 543 may detect whether pressure is applied to the portable terminal 500 and the magnitude of the pressure. The pressure sensor 543 may be installed in a portion where pressure detection is required in the portable terminal 500 according to the use environment. If the pressure sensor 543 is installed on the display 551 , according to a signal output from the pressure sensor 543 , a touch input through the display 551 and a pressure touch applied with a greater pressure than the touch input input can be identified. In addition, according to a signal output from the pressure sensor 543 , the magnitude of the pressure applied to the display 551 when a pressure touch input is input may also be known.

The motion sensor 545 detects a position or movement of the portable terminal 500 using an acceleration sensor, a gyro sensor, or the like. An acceleration sensor that can be used for the motion sensor 545 is a device that converts an acceleration change in one direction into an electrical signal, and is widely used with the development of micro-electromechanical systems (MEMS) technology.

Accelerometer sensors include, from measuring large-value acceleration used to detect a collision built into an automobile's airbag system, to measuring small-value acceleration used as input means for games by recognizing minute movements of the human hand. There are various types up to. Accelerometers are usually constructed by mounting two or three axes in one package, and depending on the environment of use, only one Z axis may be required. Therefore, if it is necessary to use the acceleration sensor in the X-axis or Y-axis direction instead of the Z-axis direction for some reason, a separate engraving board may be used to mount the acceleration sensor upright on the main board.

In addition, the gyro sensor is a sensor for measuring angular velocity, and may detect a direction returned to a reference direction.

The output unit 550 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 550 may include a display 551 , a sound output module 553 , an alarm unit 555 , and a haptic module 557 .

The display 551 displays and outputs information processed by the portable terminal 500 . For example, when the mobile terminal 500 is in a call mode, a user interface (UI) or graphic user interface (GUI) related to a call is displayed. In addition, when the mobile terminal 500 is in a video call mode or a shooting mode, the captured or received images may be displayed individually or simultaneously, and a UI and a GUI may be displayed.

On the other hand, as described above, when the display 551 and the touchpad form a layer structure to form a touch screen, the display 551 may be used as an input device capable of inputting information by a user's touch in addition to an output device. can

If the display 551 is configured as a touch screen, it may include a touch screen panel, a touch screen panel controller, and the like. In this case, the touch screen panel is a transparent panel attached to the outside, and may be connected to the internal bus of the portable terminal 500 . The touch screen panel watches the touch result, and sends corresponding signals to the touch screen panel controller when there is a touch input. The touch screen panel controller processes the signals and then transmits the corresponding data to the controller 580 so that the controller 580 can know whether there has been a touch input and which area of the touch screen has been touched.

The display 551 may be formed of e-Paper. E-Paper is a kind of reflective display and has excellent visual characteristics like conventional paper and ink with high resolution, wide viewing angle, and bright white background. Electronic paper (e-Paper) can be implemented on any substrate such as plastic, metal, paper, etc., and the image is maintained even after the power is turned off and the battery life of the mobile terminal 500 is long because there is no backlight power supply. can be maintained As the electronic paper, a hemispherical twisted ball filled with electrostatic charge may be used, or electrophoresis and microcapsules may be used.

In addition, the display 551 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display ( 3D display) may be included. In addition, two or more displays 551 may exist according to an implementation form of the portable terminal 500 . For example, an external display (not shown) and an internal display (not shown) may be simultaneously provided in the portable terminal 500 .

The sound output module 553 outputs audio data received from the wireless communication unit 510 or stored in the memory 560 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. Also, the sound output module 553 outputs an audio signal related to a function performed in the portable terminal 500 , for example, a call signal reception sound, a message reception sound, and the like. The sound output module 553 may include a speaker, a buzzer, and the like.

The alarm unit 555 outputs a signal for notifying the occurrence of an event in the portable terminal 500 . Examples of events occurring in the mobile terminal 500 include call signal reception, message reception, and key signal input. The alarm unit 555 outputs a signal for notifying the occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration. When a call signal is received or a message is received, the alarm unit 555 may output a signal to notify it. Also, when a key signal is input, the alarm unit 555 may output a signal as feedback to the key signal input. The user may recognize the occurrence of an event through a signal output from the alarm unit 555 . A signal for notifying the occurrence of an event in the portable terminal 500 may also be output through the display 551 or the sound output module 553 .

The haptic module 157 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 557 is a vibration effect. When the haptic module 557 generates vibration as a tactile effect, the intensity and pattern of vibration generated by the haptic module 157 may be converted, and different vibrations may be synthesized and outputted or output sequentially.

In addition to vibration, the haptic module 557 has an effect by a stimulus by a pin arrangement that moves vertically with respect to the contact skin surface, an effect by a stimulus through a jet force or suction force of air through a nozzle or an inlet, and an effect by a stimulus rubbing the skin surface Various tactile effects can be generated, such as effects, effects caused by stimulation through contact with electrodes, effects caused by stimulation using electrostatic force, and effects by reproducing a feeling of cold/warm using an element capable of absorbing or generating heat. The haptic module 557 may not only deliver the tactile effect through direct contact, but may also be implemented so that the tactile effect can be felt through the user's kinesthetic sense of a finger or arm. Two or more haptic modules 557 may be provided according to the configuration of the mobile terminal 500 .

The memory 560 may store a program for processing and control of the controller 580, and has a function for temporary storage of input or output data (eg, phone book, message, still image, video, etc.). can also be done

The memory 560 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. , ROM may include at least one type of storage medium. Also, the mobile terminal 500 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The interface unit 570 functions as an interface with all external devices connected to the portable terminal 500 . Examples of the external device connected to the mobile terminal 500 include a wired/wireless headset, an external charger, a wired/wireless data port, a memory card, a subscriber identification module (SIM) card, and a user identity module (UIM) card. There is a card socket such as an audio I/O (Input/Output) terminal, a video I/O (Input/Output) terminal, an earphone, and the like. The interface unit 570 may receive data or receive power from such an external device and transmit it to each component inside the portable terminal 500 , and may allow the data inside the portable terminal 500 to be transmitted to an external device. .

When the portable terminal 500 is connected to an external cradle, the interface unit 570 is a passage through which power from the connected cradle is supplied to the portable terminal 500, or various command signals input from the cradle by the user. It may be a passage through which the terminal 500 is transmitted.

The controller 580 generally controls the overall operation of the portable terminal 500 by controlling the operation of the respective units. For example, it performs related control and processing for voice calls, data communications, video calls, and the like. Also, the controller 580 may include a multimedia playback module 581 for playing multimedia. The multimedia playback module 581 may be configured as hardware in the control unit 580 or may be configured as software separately from the control unit 580 . Meanwhile, the controller 580 may include an application processor (not shown) for driving an application. Alternatively, the application processor (not shown) may be provided separately from the controller 580 .

In addition, the power supply unit 590 receives external power and internal power under the control of the control unit 580 to supply power necessary for the operation of each component. The mobile terminal 500 having such a configuration may be configured to be operable in a communication system capable of transmitting data through a frame or a packet, including a wired/wireless communication system and a satellite-based communication system. have.

5 is a flowchart illustrating an operation method of an electronic device capable of controlling another electronic device according to the related art.

Referring to FIG. 5 , first, two electronic devices to share content may be connected (Wi-Fi Direct Connection Setup) through a Wi-Fi Direct network proposed by the Wi-Fi Alliance (S510), and in this case, one electronic device is a source electronic device for transmitting content, and the remaining electronic devices may correspond to the role of a sink electronic device for receiving and outputting content.

Meanwhile, the source electronic device and the sink electronic device may exchange supported capability information and perform capability exchange and negotiation (S520).

For example, if content sharing is performed according to the Wi-Fi CERTIFIED Miracast™ standard introduced by the Wi-Fi Alliance, Capability Negotiation when connecting to Miracast for the first time ) to determine a video format and an audio format to be transmitted.

On the other hand, Miracast is a user interface back channel for controlling a source electronic device using a user interface of a sink electronic device during Miracast connection. UIBC) standard is included as an option.

For example, when a smartphone is connected as a source electronic device and a TV is connected as a sink electronic device, as a control means such as a mouse, keyboard, and remote control device connected to the TV, not only the sink electronic device is used. It can also control source electronic devices.

During the function negotiation ( S520 ), a function (capability) related to UIBC may also be determined. That is, the source electronic device checks UIBC categories such as mouse and keyboard supported by the sink electronic device, and then the sink electronic device and the source electronic device The UIBC category commonly supported is determined and serviced.

Thereafter, the sink electronic device may transmit UIBC information according to the user input to the source electronic device ( S530 ), and the source electronic device may perform a corresponding operation.

However, the control information that the sink electronic device can send to the source electronic device has a limitation in that it is limited to predetermined items such as vertical scroll, mouse button click, and key input.

6 is a flowchart illustrating an embodiment of a method of operating an electronic device according to the present invention.

Referring to FIG. 6 , the electronic device according to the embodiment of the present invention may be connected to the first electronic device through a wired/wireless network. (S510) The electronic device according to the embodiment of the present invention transmits content to another electronic device Alternatively, it may be connected to an external electronic device through a wired or wireless network for a content sharing mode operation capable of receiving content from another electronic device.

The electronic device according to an embodiment of the present invention may request or respond to a network connection to another electronic device based on a user's command input, for example, a command input for executing a corresponding menu or application, and a content sharing mode can be entered into

Here, the content sharing mode may refer to a mode in which the electronic device can transmit content to an external electronic device connected by wire or wirelessly or receive content transmitted from the external electronic device.

In addition, in the content sharing mode, the content being played on one device among a plurality of devices connected to the mirroring mode in which the connected electronic devices display the same screen can also be used by other devices, and other tasks are performed in the content sharing state It may include a dual screen dual play (DSDP) mode and the like.

In addition, the electronic device according to the present invention is a multimedia transmission technology introduced by the Wi-Fi Alliance, such as Wi-Fi CERTIFIED Miracast™, Wi-Fi Display (WFD), and Wireless Display (WiDi). can support and share content according to the standard.

Meanwhile, data for content sharing may be directly transmitted to an external electronic device through wired or wireless communication according to various communication standards, or may be transmitted to an external electronic device through a separately connected web server.

Meanwhile, the electronic device according to an embodiment of the present invention may be wirelessly connected to the first electronic device through a Wi-Fi Direct Network. The Wi-Fi Direct network is a network in which two electronic devices can be peer-to-peer connected through Wi-Fi, and the electronic devices connected through the Wi-Fi Direct network can recognize resource and capability information with each other.

Meanwhile, the electronic device according to an embodiment of the present invention may perform capability negotiation with the first electronic device (S620).

In the function negotiation step ( S620 ), the electronic device and the first electronic device according to an embodiment of the present invention may determine a video format and an audio format of content to be shared and transmitted.

When the electronic device according to an embodiment of the present invention determines a video format and an audio format to be transmitted through function negotiation (S620) (S620), the first electronic device transmits content ), the content data including video data and audio data may be transmitted to the first electronic device in a format based on the function negotiation (S630).

The electronic device according to an embodiment of the present invention may transmit control image data including one or more control menu buttons to the first electronic device (S640). That is, the electronic device receives a control command from the other electronic device. In order to select , data may be transmitted to the counterpart electronic device in advance when a control event occurs or in the content sharing mode.

Here, the control image data may include a name of the control menu button and location information of the control menu button in the control image.

The electronic device may transmit a control image together with location information of each menu button. For example, control names 'Resolution' and 'Media' of each menu button and location information (xy width hegith) in the control image may be transmitted as follows.

Wfd_uibc_control_image: Resolution 10 10 30 20, Media 10 40 30 20

Also, according to an embodiment, the control image data may further include size information of the control image. That is, by further using the overall size information of the control image in addition to the relative position coordinates of each menu button in the control image, it is possible to help the user to more quickly recognize the position input by the user.

Also, according to an embodiment, the control image data may include location information of the control image, a name of the control menu button, and location information of the control menu button. The electronic device may transmit the control image together with the location information of the control image, the name of each menu button, and the location information. In this case, the control command selected by the user may be determined by comparing the position input by the user through the sync electronic device with the position information of the control image and the position information of the button in the control image as absolute coordinates.

Meanwhile, the first electronic device receiving the control image data may receive a user input by providing a control UI screen based on the received data. The user may select a command for controlling the electronic device according to an embodiment of the present invention by selecting a control menu button included in the UI screen. Accordingly, it is possible to input various control commands without limitation according to the configuration of the control menu button.

For example, the control image data may include a control menu button corresponding to a control command related to the content sharing mode, that is, the control menu button may be configured with control commands frequently used in the content sharing mode. .

Also, the control image data may include a control menu button corresponding to a control command determined according to the type of the first electronic device or the type of content shared with the first electronic device. That is, the control menu button may be configured with control commands optimized according to the type of content being used in the content sharing mode.

Meanwhile, the type of content may be determined based on the format of the received data received from the counterpart device, extracted from metadata, or determined through various known screen analysis algorithms.

Meanwhile, according to an embodiment, the control image data may be transmitted together with data transmitted during the function negotiation ( S620 ).

Thereafter, the electronic device may receive user input information for the control image data from the first electronic device (S650) and may perform a control function corresponding to the user input information (S660).

Here, the user input information may include coordinates of a location input by a user with respect to the control image data performed by the first electronic device.

When the user selects the control menu button in the corresponding image in the sink electronic device, the selected coordinates are transmitted to the source electronic device, and when the source electronic device receives this data, the selected menu button is determined and the corresponding operation can be performed.

Alternatively, the first electronic device, which is a sink electronic device, may determine the selected menu button according to the coordinates of the user's input location for the control image data directly performed, and deliver the determined menu button information as user input information. have,

In this case, the source electronic device may perform an operation corresponding to the received menu button information.

Meanwhile, the user input information may be received as UIBC information through a user interface back channel.

According to an embodiment of the present invention, even in the state of mirroring the screen of the source electronic device to the sink electronic device by Miracast, the source (Source) using the user interface of the sink electronic device UIBC that controls electronic devices can be extended without restrictions.

A source electronic device according to an embodiment of the present invention transmits a control UI image and related data to a sink electronic device.

The source electronic device receives, from the sink electronic device, a selection position on the control UI image where the user would have selected an item on the control UI image, and controls accordingly.

Therefore, according to the present invention, more scalable and convenient source electronic device control becomes possible by controlling the source electronic device using the control UI image output from the sink electronic device.

7 is a flowchart illustrating an embodiment of a method of operating an electronic device according to the present invention. In more detail, when the electronic device described with reference to FIG. 6 operates as a source electronic device in a content sharing mode, a method of operating a sink electronic device corresponding to the operation is disclosed.

In the method of operating an electronic device according to the present invention, the second electronic device may be connected to a wired/wireless network through a wired/wireless network ( S710 ). The electronic device according to an embodiment of the present invention transmits content to another electronic device or another electronic device. For the content sharing mode operation in which the device can receive content, it may be connected to an external electronic device through a wired or wireless network. In addition, the electronic device according to an embodiment of the present invention may be wirelessly connected to the second electronic device through a Wi-Fi Direct Network.

The electronic device according to an embodiment of the present invention may request or respond to a network connection to another electronic device based on a user's command input, for example, a command input for executing a corresponding menu or application, and a content sharing mode can be entered into

Meanwhile, the electronic device according to an embodiment of the present invention may perform capability negotiation with the second electronic device (S720). Here, the capability negotiation is a video format and an audio format of content to be shared and transmitted. may include decisions of

The electronic device according to an embodiment of the present invention may receive content data including video data and audio data in a format based on the function negotiation from the second electronic device (S730).

The electronic device according to an embodiment of the present invention may receive, from the second electronic device, control image data including one or more control menu buttons (S740), and based on the received control image data, the one A control image including the above control menu buttons may be displayed. (S750)

Here, as described with reference to FIG. 6 , the control image data may include a name of the control menu button and location information of the control menu button in the control image.

Here, the control image data may include a name of the control menu button and location information of the control menu button in the control image. Also, according to an embodiment, the control image data may further include size information of the control image. That is, by further using the overall size information of the control image in addition to the relative position coordinates of each menu button in the control image, it is possible to help the user to more quickly recognize the position input by the user.

Also, according to an embodiment, the control image data may include location information of the control image, a name of the control menu button, and location information of the control menu button. The electronic device may receive the control image together with location information of the control image, a name of each menu button, and location information. In this case, the control command selected by the user may be determined by comparing the position input by the user through the sync electronic device with the position information of the control image and the position information of the button in the control image as absolute coordinates.

Meanwhile, the electronic device that has received the control image data may provide a control UI screen based on the received data (S740) and receive a user input (S750).

The user may select a command for controlling the second electronic device by selecting a control menu button included in the control UI screen. Accordingly, it is possible to input various control commands without limitation according to the configuration of the control menu button.

For example, the control image data may include a control menu button corresponding to a control command related to the content sharing mode, that is, the control menu button may be configured with control commands frequently used in the content sharing mode. .

In addition, the control image data may include a control menu button corresponding to a control command determined according to the type of the second electronic device or the type of content shared with the second electronic device. That is, the control menu button may be configured with control commands optimized according to the type of content being used in the content sharing mode.

Meanwhile, according to an embodiment, the control image data may be received together with data transmitted during the function negotiation ( S720 ).

Thereafter, the electronic device according to the present embodiment may receive a user input for the control image (S760) and transmit user input information for the control image data to the second electronic device (S770).

Meanwhile, the user input information may include coordinates of a user's input position with respect to the control image data performed by the electronic device according to an embodiment of the present invention.

When the user selects the control menu button within the corresponding image in the sink electronic device (S760), the selected coordinates are transmitted to the source electronic device (S770), and when the source electronic device receives this data, the selected menu The button can be identified and the corresponding action can be performed.

Alternatively, the sink electronic device may determine the selected menu button according to the coordinates of the user's input position for the control image data directly performed, and deliver the determined menu button information as user input information.

In this case, the source electronic device may perform an operation corresponding to the received menu button information.

Meanwhile, the user input information may be received as UIBC information through a user interface back channel.

According to the present invention, there is no restriction on adding control menu items, so that the user can control the electronic devices in a more scalable and convenient way.

Meanwhile, electronic devices according to an embodiment of the present invention may be network-connected 1:1, and may be connected together with a plurality of electronic devices to operate in a content sharing mode with the plurality of electronic devices.

Accordingly, the electronic device according to an embodiment of the present invention may be connected to the third electronic device through a wired/wireless network even in a state in which the electronic device is connected to the second electronic device.

Even in this case, like the process performed when connecting with the second electronic device, capability negotiation is performed with the third electronic device, and the format is converted from the third electronic device based on the capability negotiation. It can receive video and audio data.

In addition, control image data including one or more control menu buttons may be received from the third electronic device.

On the other hand, the electronic device receiving the control image data including one or more control menu buttons from the third electronic device, the control including the one or more control menu buttons based on the control image data received from the third electronic device Images can be displayed.

Alternatively, according to an embodiment, when an input for selecting an object representing the third electronic device of the user is received while displaying the object representing the connected second electronic device and the third electronic device, the third electronic device It may be configured to display a control image including the one or more control menu buttons based on control image data received from the device.

8 to 16 are diagrams referenced to explain various embodiments of a method of operating an electronic device according to the present invention. An output screen of electronic devices, a menu screen, and a link operation between devices according to embodiments of the present invention will be described with reference to FIGS. 8 to 16 .

Referring to FIG. 8 , a sink electronic device and a source electronic device may be connected through a wired/wireless network for content sharing. For example, a sink electronic device and a source electronic device may be wirelessly connected through a Wi-Fi Direct network.

The sink electronic device and the source electronic device may determine items for content sharing and determine functions through capability negotiation.

Meanwhile, FIGS. 9 to 11 illustrate that the screen 1005 of the notebook PC 400 is mirrored by the TV 100 . If the user wants to change the resolution setting, not only must directly manipulate the notebook PC 400 as the source electronic device, but also set it through many steps in the order of control panel > display > screen resolution 1010 .

However, according to the embodiment of the present invention, as shown in FIG. 9 , the control image 900 including the resolution menu button 910 may be transmitted to the TV 100 .

Also, the source electronic device may transmit a control image together with location information of each menu button.

For example, the location information (xy width hegith) in the control image of each of the control names 'Resolution' button 910, 'Media' button 920 and control menu button of each menu button is 'Wfd_uibc_control_image: Resolution 10 10 30 20, Media 10 40 30 20' can be transmitted. In addition, width, height information, and size information of the control image may be further transmitted.

Referring to FIG. 9 , the control image 900 may have location information of C2(50,0), C3(0,100), and C4(50,100) based on the upper left C1(0,0) as a coordinate reference. In addition, the 'Resolution' button 910 is also a1(10,10), a2(40,10), a3(10,30), a4(40,30) with the upper left C1(0,0) as the coordinate reference. The 'Media' button 920 may be drawn in the control image 900 with the coordinates of the position of 'Media' in the same manner, with the upper left C1(0,0) as the coordinate reference.

Meanwhile, the TV 100 receives, from the notebook PC 400 , content data including video data, audio data, and the like, and control image data including one or more control menu buttons in a format based on function negotiation. can do.

Meanwhile, the TV 100 may display a control image 1060 including the one or more control menu buttons 1061 and 1062 as well as the mirroring screen 1055 .

The user may select a menu button to control the notebook PC 400 by using the remote control device 200 of the TV 100 .

Referring to FIG. 10 , the user selects the 'Resolution' button 1061 by manipulating the pointer 205 that is moved and displayed in response to the movement of the remote control device 200 with the remote control device 200 . .

When there is a user's selection input, the TV 100 may transmit coordinate information of selecting the 'Resolution' button 1061 to the notebook PC 400 as UIBC information.

Accordingly, the notebook PC 400 may display the settable resolution setting window 1010 by confirming that there is a selection input of the 'Resolution' button 1061 . Also, the resolution setting window 1010 may be mirrored and displayed on the screen of the TV 100 .

Thereafter, the user may change the resolution setting by using the remote control device 200 again. The resolution is changed in the notebook PC 400 according to the user's setting input and will be reflected in the TV 100 .

Meanwhile, in the past, when a specific media file of the notebook PC 400 is to be played, it is inconvenient to manually manipulate the notebook PC 400 to find and change a folder.

However, according to the present invention, as shown in FIG. 11 , the user can select the 'Media' button 1062 without directly manipulating the notebook PC 400 by manipulating the remote control device 200 of the TV 100. .

Then, the TV 100 transmits the coordinate information selected by the 'Media' button 1062 to the notebook PC 400 as UIBC information, and the notebook PC 400 that determines the selection of the 'Media' button 1062 is a media file A list 1020 may be displayed. The user may select and play a desired media file on the TV 100 again.

Meanwhile, the control image data may include a control menu button corresponding to a control command related to a content sharing mode.

For example, in addition to the 'Resolution' button 1061 and the 'Media' button 1062 illustrated in FIGS. 10 and 11 , playback related menus such as play, stop, fast forward, and rewind, general setting menu of the notebook PC, network It may include a setting menu and the like.

Meanwhile, the control image data may include a control menu button corresponding to a control command determined according to the type of the electronic device being shared or the type of shared content.

13 exemplifies a case of mirroring from the smart phone 500 to the TV 100.

In the prior art, the menu buttons at the bottom of the smart phone 500, for example, back (back), home (home), menu (menu) key button, etc. are not on the screen, so the selection is not It was impossible.

However, according to the embodiment of the present invention, the control image 1020 including the back 1221 , home 1222 , and menu 1223 key buttons are transmitted to the TV 100 from the smart phone 500 which is the source electronic device. can

Even in this case, the user may control the smart phone 500 in the same process as directly operating the smart phone 500 by manipulating the pointer 205 with the remote control device 200 of the TV 100 .

For example, when the user selects the home 1222 key button of the control image 1020 with the remote control device 200 , the TV 100 selects the home 1222 key button with the smart phone 500 coordinate information , and the smart phone 500 may return to the home screen 510 .

13 exemplifies mirroring of the screen 1331 of the notebook PC 400 to the TV 100 . In more detail, it is an example of opening a PPT file in the notebook PC 400 and mirroring it to a TV.

When the user selects the mirroring screen 1320 on the TV 100 or operates the remote control device 200 to move the pointer to a specific area within the screen, for example, the lower left corner of the screen, the control image 1330 is displayed on the TV screen. can be displayed.

Meanwhile, the control image 1330 may include a control button suitable for the current content sharing situation. For example, since a PPT file is currently used, the control image 1330 has a “Slideshow” button 1331 that can start a slide show, and “Change Resolution” that can change the resolution if the screen is not visible because the resolution is not correct. It may include a button 1332 , a “previous page” button 1333 for changing a page, and a “next page” button 1334 .

Meanwhile, electronic devices according to an embodiment of the present invention may be network-connected 1:1, and may be connected together with a plurality of electronic devices to operate in a content sharing mode with the plurality of electronic devices.

14 to 16 show that the TV 100 operates in a content sharing mode with four mobile terminals 1401, 1402, 1403, and 1404, mirroring the mobile terminal screens 1411, 1412, 1413, and 1414, An example of displaying the mirroring screens 1451, 1452, 1453, and 1454 is shown.

According to an embodiment, when the user operates the remote control device 200 to select any one of the mirroring screens 1451 , as shown in FIG. 15 , a control image based on data received from the corresponding portable terminal 1401 . (1510) can be displayed.

Meanwhile, the control image 1510 may include menu buttons such as play 1511 , stop 1512 , and mute 1513 , and the user may select a desired menu button as the remote control device 200 . .

Alternatively, when the user operates the remote control device 200 and selects any one of the mirroring screens 1451, as shown in FIG. 16, data received from the currently connected mobile terminals 1401, 1402, 1403, 1404) All of the based control images 1510 , 1520 , 1530 , and 1540 may be displayed.

Alternatively, when the user operates the remote control device 200 to move the pointer to an area where the mirroring screens 1451, 1452, 1453, 1454 are not displayed, the currently connected mobile terminals 1401, 1402, 1403, 1404) Control images 1510 , 1520 , 1530 , and 1540 based on data received from .

As described above, in the prior art, only control determined in advance between the source electronic device and the sink electronic device was possible. However, according to the present invention, there is no limit to the control command, so it is easy to expand the control items according to the input means and electronic devices such as various remote control devices. can get

The configuration and method of the above-described embodiments are not limited to the electronic device, such as a mobile terminal, an image display device, and the like, and an operation method thereof according to the present invention, but the embodiments are each so that various modifications can be made. All or some of the embodiments may be selectively combined and configured.

Meanwhile, the method of operating an electronic device according to the present invention can be implemented as a processor-readable code on a processor-readable recording medium provided in the electronic device. The processor-readable recording medium includes all types of recording devices in which data readable by the processor is stored. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also includes those implemented in the form of carrier waves such as transmission over the Internet. . In addition, the processor-readable recording medium is distributed in a computer system connected through a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims Various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Wireless communication unit: 510
A/V(Audio/Video) input : 520
User input: 530
Sensing unit: 540
Output: 550
Memory: 560
Interface part: 570
Control unit: 580
Power supply: 590

Claims (20)

A method of operating an electronic device, comprising:
connecting to the first electronic device through a network;
performing capability negotiation with the first electronic device;
transmitting content data to the first electronic device in a format based on the function negotiation;
transmitting control image data including one or more control menu buttons to the first electronic device;
receiving user input information on the control image data from the first electronic device; and;
Including; performing a control function corresponding to the user input information;
Control image data including the control menu button,
and control image data including a back key, a home key, and a menu key,
The electronic device is
When receiving the coordinate information of the home key from the first electronic device, the operating method of the electronic device for displaying a home screen. According to claim 1,
The step of connecting to the first electronic device through a network includes wirelessly connecting through a Wi-Fi Direct Network. According to claim 1,
The method of operating an electronic device, wherein the function negotiation includes determining a video format and an audio format of the content. According to claim 1,
The method of operating an electronic device, wherein the control image data includes a name of the control menu button and location information of the control menu button in the control image. 5. The method of claim 4,
The method of operating an electronic device, wherein the control image data further includes size information of the control image. According to claim 1,
The method of operating an electronic device, wherein the control image data includes location information of the control image, a name of the control menu button, and location information of the control menu button. According to claim 1,
The method of operating an electronic device, wherein the control image data includes a control menu button corresponding to a control command related to a content sharing mode. According to claim 1,
The method of operating an electronic device, wherein the control image data includes a control menu button corresponding to a control command determined according to a type of the first electronic device or a type of content shared with the first electronic device. According to claim 1,
The receiving of the user input information comprises receiving the user interface back channel information as user interface back channel information. A method of operating an electronic device, comprising:
connecting to a second electronic device through a network;
performing capability negotiation with the second electronic device;
receiving content data from the second electronic device in a format based on the function negotiation;
receiving control image data including one or more control menu buttons from the second electronic device;
displaying a control image including the one or more control menu buttons based on the received control image data;
receiving a user input for the control image; and;
transmitting user input information on the control image data to the second electronic device;
Control image data including the control menu button,
and control image data including a back key, a home key, and a menu key,
The electronic device is
displaying a control image including the back key, home key, and menu key on a display,
When a control image corresponding to the home key is selected, the method of operating an electronic device for transmitting coordinate information on the home key to the second electronic device. 11. The method of claim 10,
The method of operating an electronic device, characterized in that the step of connecting to the second electronic device through a network is wirelessly connected through a Wi-Fi Direct Network. 11. The method of claim 10,
The method of operating an electronic device, wherein the function negotiation includes determining a video format and an audio format of the content. 11. The method of claim 10,
The method of operating an electronic device, wherein the control image data includes a name of the control menu button and location information of the control menu button in the control image. 14. The method of claim 13,
The method of operating an electronic device, wherein the control image data further includes size information of the control image. 11. The method of claim 10,
The method of operating an electronic device, wherein the control image data includes location information of the control image, a name of the control menu button, and location information of the control menu button. 11. The method of claim 10,
The method of operating an electronic device, wherein the control image data includes a control menu button corresponding to a control command related to a content sharing mode. 11. The method of claim 10,
The method of operating an electronic device, wherein the control image data includes a control menu button corresponding to a control command determined according to a type of the second electronic device or a type of content shared with the second electronic device. 11. The method of claim 10,
The transmitting of the user input information includes transmitting the user interface back channel information as user interface back channel information. 11. The method of claim 10,
connecting to a third electronic device through a network;
performing capability negotiation with the third electronic device;
receiving video and audio data from the third electronic device in a format based on the function negotiation;
receiving control image data including one or more control menu buttons from the third electronic device;
and displaying a control image including the one or more control menu buttons based on the control image data received from the third electronic device. 11. The method of claim 10,
connecting to a third electronic device through a network;
performing capability negotiation with the third electronic device;
receiving video and audio data from the third electronic device in a format based on the function negotiation;
receiving control image data including one or more control menu buttons from the third electronic device;
displaying objects representing the second electronic device and the third electronic device;
When an input for selecting an object representing the third electronic device is received, displaying a control image including the one or more control menu buttons based on the control image data received from the third electronic device; A method of operating an electronic device comprising a.

Images