KR20130101886A - System for interworking and controlling devices and reception device used in the same - Google Patents

System for interworking and controlling devices and reception device used in the same Download PDF

Info

Publication number
KR20130101886A
KR20130101886A KR1020120022986A KR20120022986A KR20130101886A KR 20130101886 A KR20130101886 A KR 20130101886A KR 1020120022986 A KR1020120022986 A KR 1020120022986A KR 20120022986 A KR20120022986 A KR 20120022986A KR 20130101886 A KR20130101886 A KR 20130101886A
Authority
KR
South Korea
Prior art keywords
image
user terminal
terminal
signal
display
Prior art date
Application number
KR1020120022986A
Other languages
Korean (ko)
Inventor
유창식
Original Assignee
한양대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한양대학교 산학협력단 filed Critical 한양대학교 산학협력단
Priority to KR1020120022986A priority Critical patent/KR20130101886A/en
Priority to US13/785,266 priority patent/US20130244730A1/en
Priority to US13/785,600 priority patent/US20130234984A1/en
Priority to US13/785,370 priority patent/US20130234959A1/en
Priority to US13/785,498 priority patent/US8913026B2/en
Priority to US13/785,302 priority patent/US20130241854A1/en
Publication of KR20130101886A publication Critical patent/KR20130101886A/en
Priority to US14/155,204 priority patent/US20140125622A1/en
Priority to US15/052,803 priority patent/US20160170703A1/en
Priority to US16/142,998 priority patent/US10656895B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/42224—Touch pad or touch panel provided on the remote control

Abstract

The present invention discloses a system for interworking and controlling terminals. In the system, the receiving terminal receives an image signal that has delivered image data, and receives a signal signal for receiving a control signal having position information of the touch means close to the user terminal and corresponds to the position information of the touch means included in the received control signal. And a display unit for displaying a second image.

Description

Terminal interlocking and control system and receiving terminal for use {SYSTEM FOR INTERWORKING AND CONTROLLING DEVICES AND RECEPTION DEVICE USED IN THE SAME}

The present invention relates to a system for interworking and controlling terminals.

Recently, people enjoy playing games, checking e-mails, or surfing the web using electronic terminals such as smartphones. The electronic terminal has the advantage of being convenient to carry because it is light and small, but there is also a disadvantage that the screen is not small enough to enjoy games. That is, due to the small size of the electronic terminal, the enjoyment of games, web surfing, etc. is halved and it is inconvenient to check e-mail.

The present invention provides a method and system for interlocking a large sized terminal with a small sized terminal and controlling the small sized terminal while viewing the large sized terminal.

In order to achieve the above object, a receiving terminal used in a terminal interworking system according to an embodiment of the present invention receives an image signal that has delivered image data, and a control signal having position information of a touch means close to a user terminal. Signal unit for receiving; And a display unit configured to display a second image corresponding to the position information of the touch means included in the received control signal.

A receiving terminal used in a terminal interworking system according to another embodiment of the present invention receives an image signal that has delivered image data, and has a position indication data corresponding to a second image indicating a position of a touch means proximate to the user terminal. A control signal unit for receiving a signal; And a display unit configured to display a second image corresponding to the position indication data included in the received control signal.

A receiving terminal used in a terminal interworking system according to another exemplary embodiment of the present invention may include composite image data corresponding to a composite image of a second image indicating a location of a touch means close to a user terminal and a first image to be displayed on the user terminal. Signal unit for receiving; And a display unit which displays a composite image corresponding to the received composite image data.

A receiving terminal used in a terminal interworking system according to another embodiment of the present invention transmits an image signal having image data to a user terminal and a signal unit for receiving a control signal having position information of a touch means in proximity to the user terminal. ; And a display unit configured to display a second image corresponding to the position information of the touch means included in the received control signal.

A receiving terminal used in a terminal interworking system according to another embodiment of the present invention transmits an image signal having image data to a user terminal, and displays a position corresponding to a second image indicating a position of a touch means proximate to the user terminal. A signal unit for receiving a control signal having data; And a display unit configured to display a second image corresponding to the position indication data included in the received control signal.

A receiving terminal used in a terminal interworking system according to another embodiment of the present invention transmits an image signal having image data to a user terminal, and transmits an image signal having a second image indicating a position of a touch means close to the user terminal and the user terminal. A signal unit receiving composite image data corresponding to the composite image of the first image to be displayed; And a display unit which displays a composite image corresponding to the received composite image data.

A receiving terminal according to another embodiment of the present invention comprises a signal unit for receiving image data from the user terminal and the position information of the touch means in close proximity to the user terminal; And a display configured to display a second image of the location information.

According to another embodiment of the present invention, a receiving terminal includes a signal unit for receiving composite image data from a user terminal; And a display unit for displaying the composite image data. Here, the composite image data has first image data and second image data indicating a position of a touch means proximate to the user terminal.

Since the method and system for connecting and controlling a terminal according to the present invention share a first image with terminals, a second image related to a user's control of a small terminal is displayed on the first image of the large terminal. The user can control the operation of the small size terminal while looking at the large size terminal. Therefore, the user can fully enjoy games, surfing the web, and the like through a small size terminal.

1 is a view schematically showing a terminal interworking and control system according to an embodiment of the present invention.
2 is a flowchart illustrating a terminal interworking and control method according to a first embodiment of the present invention.
3 is a diagram illustrating a control method used in the terminal interworking and control method of FIG. 2.
4 is a plan view illustrating an image sharing screen according to an exemplary embodiment of the present invention.
5 is a diagram illustrating a touch means sensing method according to an embodiment of the present invention.
6 is a view showing a touch means detection method according to another embodiment of the present invention.
7 is a diagram illustrating a touch means sensing method according to another embodiment of the present invention.
8 is a flowchart illustrating a terminal interworking and control method according to a second embodiment of the present invention.
9 is a flowchart illustrating a terminal interworking and control method according to a third embodiment of the present invention.
10 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.
11 is a flowchart illustrating a terminal interworking and control method according to a fourth embodiment of the present invention.
12 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.
13 is a flowchart illustrating a terminal interworking and control method according to a fifth embodiment of the present invention.
14 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.
15 is a block diagram showing the structure of a user terminal according to an embodiment of the present invention.
FIG. 16 is a block diagram illustrating a structure of the image unit of FIG. 15 according to an embodiment of the present invention. FIG.
17 is a block diagram showing the structure of a user terminal according to a second embodiment of the present invention.
18 is a block diagram showing the structure of a user terminal according to a third embodiment of the present invention.
19 is a block diagram showing the structure of a user terminal according to a fourth embodiment of the present invention.
20 is a block diagram showing the structure of a user terminal according to a fifth embodiment of the present invention.
21 is a block diagram showing the structure of a receiving terminal according to the first embodiment of the present invention.
22 is a block diagram showing the configuration of a receiving terminal according to a second embodiment of the present invention.
23 is a block diagram showing the configuration of a receiving terminal according to a third embodiment of the present invention.
24 is a block diagram showing a configuration of a receiving terminal according to a fourth embodiment of the present invention.
25 is a block diagram showing a configuration of a receiving terminal according to a fifth embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Terminal interworking and control system of the present invention relates to a system for interlocking and controlling the terminals, in particular a system for controlling the operation of the terminal based on the image sharing and the shared image. In particular, the terminal interworking and control system of the present invention, after interlocking a small size terminal directly controlled by the user, for example, a touch method and a large size terminal, while the user looks at the large size terminal of the small size Various methods for controlling the terminal are proposed. However, for convenience of description, a small sized terminal directly controlled by a user is called a user terminal, and a large sized terminal that receives location information of the touch means from the user terminal is called a receiving terminal. Of course, the user terminal preferably has a smaller size than the receiving terminal, but the size of the terminals is not limited to the above size.

The terminal used in the system of the present invention has a function of displaying an image and a wired / wireless communication function, but the communication function is not an essential function. However, considering the real life application, it is preferable that each terminal has an image display function and a communication function. The terminal is not limited to a specific device as long as it can display an image and transmit or receive a signal with another terminal, for example, a smartphone, a smart TV, a PC, a tablet PC, a laptop, a touch pad, a game console, a cloud PC, or the like. Various devices can be used as the terminal of the present invention. However, it is preferable that a terminal having a small size has a touch function.

Hereinafter, a terminal interworking and control system according to various embodiments of the present invention will be described with reference to the accompanying drawings. However, for convenience of description, an image shared by the terminals will be referred to as a first image, and an image representing position information of the touch means will be referred to as a second image (location display image).

1 is a view schematically showing a terminal interworking and control system according to an embodiment of the present invention.

Referring to FIG. 1, the terminal interworking and control system of the present embodiment includes a user terminal 100 and a receiving terminal 102.

The user terminal 100 may be an image display terminal directly controlled by the user, and may be, for example, a smartphone capable of executing a game such as Angry Birds. Preferably, the user terminal 100 is a terminal having a touch function and having a relatively small size.

The receiving terminal 102 is a terminal which is not directly manipulated by a user but interlocks with the user terminal 100 and which can display an image. The reception terminal 102 may be an element used for a different purpose than the user terminal 100, and may be, for example, a TV that broadcasts a broadcast program such as a drama. Preferably, the receiving terminal 102 is a terminal having a relatively large size, but does not necessarily have a touch function.

The terminals 100 and 102 may be connected directly or indirectly by wire or wirelessly using another device, and are connected through a plurality of channels, for example, two channels. One channel is used to transmit image data (image information) corresponding to the shared first image, and the other channel is used to transmit a control signal having position information of the touch means. The channels may all be wireless channels or wired channels, one may be a wireless channel and the other may be a wired channel. Of course, the terminals 100 and 102 may transmit and receive image data and control signals through one channel. For example, the transmission interval of the channel may include a control interval for transmitting a control signal between data intervals for transmitting image data. In particular, since a blank period exists between sections for transmitting image data, the blank period may be used as a control signal transmission section. For example, the user terminal 100 transmits image data to the receiving terminal 102 through one channel, but receives a control signal during a blank period between data sections for transmitting the image data. Can be sent to. However, when real-time interworking of the terminals 100 and 102 is important, it is preferable that a data channel for transmitting image data and a control channel for transmitting a control signal exist separately.

The user terminal 100 transmits the data corresponding to the first image to be displayed or displayed by the user to the receiving terminal 102 through the corresponding channel under the control of the user, and transmits the data to the receiving terminal 102 through the corresponding channel. The control signal may be separately transmitted to the receiving terminal 102 through the corresponding channel. Currently, terminals such as smartphones and TVs use wired communication standards such as high-definition multimedia interface (HDMI), mobile high definition link (MHL), displayport, and wireless communication standards such as WiFi, and they each transmit data. It is equipped with a separate control channel for transmitting and receiving control signals as well as a data channel. For example, HDMI uses CEC (Consumer Electronic Control), DDC (Display Data Channel), Utility, SCL / SDA as control channels, MHL uses CBUS as control channels, and Displayport controls auxiliary channels. I am using it as a channel. Therefore, it is not necessary to set up separate channels in order to interwork with the terminals 100 and 102, and channels already existing in the terminals 100 and 102 may be used as channels for transmitting the control signal of the present invention.

Although not mentioned above, the receiving terminal 102 may also transmit an image signal or the like to the user terminal 100. In this case, the receiving terminal 102 may use channels other than the above data channel / control channel, but it is preferable to use the same data channel / control channel.

When the user terminal 100 distributes the image data corresponding to the first image displayed by the user terminal to the image signal and transmits the image data to the receiving terminal 102 through the corresponding channel, the receiving terminal 102 transmits the image data through the image signal. Acquire and display a first image corresponding to the image data. As a result, the user terminal 100 and the receiving terminal 102 display the same first image, that is, share the first image. However, the first image displayed on the receiving terminal 102 is substantially the same as the first image displayed on the user terminal 100, but due to the characteristics of the terminal of the receiving terminal 102, the color is different or the display ratio is different. There is a number. Although the receiving terminal 102 may display the transmitted image data as a corresponding image without special processing, the receiving terminal 102 may display the first image by processing the transmitted image data according to its characteristics. That is, as long as the receiving terminal 102 can display an image substantially the same as the first image of the user terminal 100, the method of processing the transmitted image data may be variously modified.

When the same image as the first image of the user terminal 100 is displayed on the receiving terminal 102, the receiving terminal 102 may stop the existing executing program and display the first image or execute the existing executing program. It can also be run through a separate channel. In addition, the receiving terminal 102 may display the existing execution program together on the screen on which the first image is displayed. For example, the receiving terminal 102 may stop the drama, which is an existing execution program, and display the first image of the Angry Bird game that is executed in the user terminal 100, for example, by switching the drama to a separate channel. The first image may be broadcasted and the first image may be displayed on a corresponding channel, or the drama may be continuously broadcasted on the current channel, but the first image may be displayed on another channel. In addition, the receiving terminal 102 may display the first image on the current channel, but display the drama that was being aired in a separate pop-up window. However, since the present invention is intended for a user to view the first image of the user terminal 100 through the receiving terminal 102, the receiving terminal 102 stops a program executed in the current channel or displays the display through another channel. It is preferable to switch the screen of the current channel to the screen on which the first image is displayed.

According to another embodiment of the present invention, the receiving terminal 102 may share the first image by transmitting an image signal to the user terminal 100. Even in this case, the control signal reflecting the user's operation (touch operation) is transmitted from the user terminal 100 to the receiving terminal 102.

Image sharing between the user terminal 100 and the receiving terminal 102 is not performed only once, but the same image is displayed to the receiving terminal 102 whenever the first image of the user terminal 100 is switched. Therefore, when the user operates the user terminal 100, the first image corresponding to the operation is displayed on the receiving terminal 102. In particular, since the receiving terminal 102 has a relatively larger size than the user terminal 100, the user may control the user terminal 100, but the gaze may look at the receiving terminal 102. That is, the user may manipulate the user terminal 100 while looking at the receiving terminal 102.

However, if the user terminal 100 is not viewed, it is difficult to grasp its own operation, and if the receiver terminal 102 is not present, the screen is small, so that the present invention can easily control the user terminal 100 while watching the receiver terminal 102. Various methods and systems are proposed.

According to an embodiment of the present invention, when a touch means such as a user's finger is close to a touch sensing area of the user terminal 100, for example, a screen, the touch means may be displayed on the first image displayed on the receiving terminal 102. A corresponding second image, for example a shadow image, may be displayed at that location, ie the position of the touch means is specified above the first image. In the case of a general PC, not an interlocking system, if the mouse is moved, the position of the mouse is not specified to the exact position, and in the case of a smartphone, the image indicating the position is not displayed even if it is touched. On the other hand, the present invention specifies the position of the touch means on the user terminal 100 to the receiving terminal 102 as a shadow image, while interlocking the user terminal 100 and the receiving terminal 102. Therefore, the user can check the position and the operation touched by the user while not looking at the user terminal 100 but can control the receiver terminal 102 without looking at the receiver terminal 102. Subsequently, when the user touches and moves the screen of the user terminal 100, the position of the second image may change according to the movement. In this case, the second image may be maintained as a shadow image, or the finger image may move on the first image according to the movement after the shadow image as the second image is changed to a finger image. Therefore, the user can check the operation status of the touch means in the user terminal 100 through the receiving terminal 102, and as a result, the user does not look at the user terminal 100, but only while looking at the receiving terminal 102. 100 can be easily controlled. For example, when a user is playing an Angry Birds game with a smartphone, the user may enjoy the Angry Birds game by operating the smartphone while watching only a TV screen having a large size. If the first image displayed on the user terminal 100 is changed to another first image according to a user's operation, the same first image is displayed on the receiving terminal 102, in which case the second image is displayed on the new first image. It is displayed at that location. The second image may be displayed on the screen of the receiving terminal 102 based on the coordinates of the user terminal 100. Of course, the first image may be shared according to image data transmission of the user terminal 100 or may be shared according to image data transmission of the receiving terminal 102.

According to another exemplary embodiment of the present invention, when the user approaches the touch sensing unit from the user terminal 100 having a touch sensing unit, for example, a touch pad, separately from the display unit, such as a laptop, the sensing unit 102 detects this. The second image may be displayed on the first image. That is, as long as the user terminal 100 can detect the touch means, the area for detecting the touch means is not limited to a specific portion of the user terminal 100.

In summary, the user terminal 100 and the receiving terminal 102 share the first image, and the user can easily manipulate the user terminal 100 having the small size while viewing the receiving terminal 102 having the large size. have. Accordingly, the user may move the touch means such as a finger or a touch pen on the image of the user terminal 100 while looking at the receiving terminal 102 or click a menu icon displayed on the first image of the user terminal 100 to specify You can also run the function. Of course, according to the above execution, an image substantially identical to the new first image displayed on the user terminal 100 is displayed on the receiving terminal 102.

Although not described above, once the second image is specified on the receiving terminal 102, the second image may reflect the movement of the touch means, but the second image may be moved by manipulating a physical keyboard, a keyboard displayed on the screen, and the like. It may be.

According to another embodiment of the present invention, the companion system may include a plurality of user terminals 100 and one receiving terminal 102. For example, when a plurality of users are playing the same game, the user terminals 100 are wired or wirelessly connected to the receiving terminal 102. When the user terminals 100 respectively transmit the position information of the corresponding touch means to the receiving terminal 102, the receiving terminal 102 displays second images indicating the positions of the touch means on the same first image. Can be. In this case, the first image shared by the user terminals 100 and the receiving terminal 102 may be transmitted by the specific user terminal 100 to the receiving terminal 102.

According to another embodiment of the present invention, when a plurality of touch means touches one user terminal 100, for example, a user touches the user terminal 100 with one left and right fingers or a plurality of users When the user terminal 100 is touched, the user terminal 100 transmits a control signal including the location information of the touch means to the receiving terminal 102, and the receiving terminal 102 according to the control signal has a first image. Second images corresponding to the location information may be displayed. Of course, shapes, colors or sizes may be different so that the second images are distinguished from each other. Meanwhile, the user terminal 100 may transmit a plurality of control signals having location information of each touch means to the receiving terminal 102.

According to another embodiment of the present invention, the user terminal 100 may sense the touch means by another method by realizing a plurality of sensing methods. For example, the user terminal 100 may sense a finger through a capacitive method and a touch pen using a receiver. Therefore, the user may implement multi-touch by simultaneously using a finger and a touch pen. As a result, the user can realize a separate operation by using a plurality of touch means.

According to another embodiment of the present invention, the companion system may include a user terminal, a receiver terminal, and a companion server. The companion server transmits the first image to both the user terminal and the receiving terminal to display the same first image, and the user terminal transmits the position information of the touch means to the receiving terminal to display the second image on the first image. May be displayed.

Hereinafter, various embodiments of the terminal interworking and control method of the present invention will be described with reference to the accompanying drawings.

2 is a flowchart illustrating a terminal interworking and control method according to a first embodiment of the present invention, FIG. 3 is a diagram illustrating a control method used in the terminal interworking and control method of FIG. 2, and FIG. FIG. 1 is a plan view illustrating an image sharing screen according to an exemplary embodiment. This embodiment proposes a linkage and control method when the user terminal 100 transmits image data (image signal) to the reception terminal 102.

Referring to FIG. 2, the user terminal 100 requests the linkage of an image to the receiving terminal 102 (S200). Specifically, when the user connects the user terminal 100 to the receiving terminal 102 by wire or wirelessly and requests interlocking, or when the user terminal 100 is connected to the receiving terminal 102, the user terminal 100 or the receiving terminal 102. After the terminal 102 identifies the counterpart terminal, the terminal 102 may inquire whether to interwork with the counterpart terminal. Interworking of the terminals 100 and 102 may be performed at the request of a user, or may be automatically performed when the terminals 100 and 102 mutually identify each other. According to an embodiment of the present invention, as shown in FIG. 3A, the user terminal 100 and the receiving terminal 102 may be connected to two channels, that is, a data channel and a control channel, and may request an interworking request or Control signals for permission can be transmitted and received through the control channel.

Subsequently, the user terminal 100 distributes image data corresponding to the first image displayed on the screen to an image signal and transmits the image data to the receiving terminal 102 through the data channel, and the receiving terminal 102 transmits the transmitted image. The first image corresponding to the data is displayed (S202). That is, the user terminal 100 and the receiving terminal 102 share the first image. Since it is useful to display the first image of the user terminal 100 and the first image of the receiving terminal 102 at substantially the same time, the image portion of the user terminal 100 transmits the image data to the display unit when the image data is transmitted. It is desirable to transmit simultaneously to the receiving terminal 102 through the channel.

Subsequently, when a user approaches a touch means such as a finger or a touch pen to the user terminal 100, the user terminal 100 detects the touch means (S204). Specifically, the user terminal 100 detects the touch means when the touch means enters a detectable area. For example, the user terminal 100 may use a capacitive touch panel, and when the user approaches the finger to the user terminal 100, the user terminal 100 may detect the capacitance change caused by the finger to detect the position of the finger. have. However, for this sensing, the user terminal 100 may set the sensing level to a plurality of levels, for example, two levels, as shown in FIG. 3B. When not performing the terminal interworking process, that is, when the user uses only the user terminal 100, the detection level of the user terminal 100 is set to a high level (L2) so that the touch position is detected only when the user touches. . On the other hand, when the user performs the terminal interworking process, that is, also using the receiving terminal 102, the user terminal 100 changes the detection level to a low level (L1). Therefore, the user terminal 100 may detect the touch means even when the touch means is not in touch but in close proximity. That is, the user terminal 100 may set different sensing levels depending on whether or not the terminal interworking process is used. According to another embodiment of the present invention, the user terminal 100 may use a variety of methods, such as electromagnetic induction, resistive film, optical, ultrasonic method, in addition to the capacitive method, the setting conditions may vary depending on the usage method .

Subsequently, the user terminal 100 transmits a control signal including position information of the touch means according to the detection result to the receiving terminal 102 through a control channel, and the receiving terminal 102 transmits the control signal according to the control signal. A second image according to the location information, for example, a shadow image 400 as shown in FIG. 4 is displayed on the first image (S206). In the above, signals are transmitted differently by being divided into a data channel and a control channel, but both image data and a control signal may be transmitted through the data channel. For example, the user terminal 100 sets the control section 112 between the data sections 110 and 114 as shown in FIG. 3C to control the signal C1 during the control section 112. Can transmit

Subsequently, when the user touches and moves the user terminal 100 by using a touch means, the user terminal 100 generates a control signal including the position information according to the movement and transmits it to the receiving terminal 102, and the receiving terminal 102. 102 displays the second image on the first image according to the transmitted control signal (S208). As a result, the second image is displayed on the first image of the receiving terminal 102 to reflect the movement of the touch means. Of course, when the user clicks a menu for performing a specific operation on the first image on the user terminal 100, only image switching is performed, and thus, the receiving terminal 102 maintains the second image at the current position. Only an image may be changed into a new image of the user terminal 100.

In summary, the terminal interworking and control method of the present invention detects the proximity of the touch means in the state where the first image is shared, displays a second image on the first image of the receiving terminal 102, and the user selects the user terminal ( When touching or moving after the touch, the second image reflecting the movement is displayed on the first image of the receiving terminal 102.

The first image and the second image displayed on the receiving terminal 102 may overlap as separate images, or a composite image generated by combining the first image and the second image may be displayed.

Although the second image is referred to as being displayed on the first image, the display method may be variously modified as long as the second image is displayed together with the first image. For example, a portion of the first image corresponding to the position of the touch unit may be darkly expressed, and the darkly expressed portion may be the second image.

Hereinafter, various methods of sensing the touch means will be described with reference to the accompanying drawings.

As a basic method of detecting the touch means, the display unit of the user terminal 100 is configured by a capacitive touch panel, and a touch means such as a finger, or the like, is used. In this case, the sensing unit of the user terminal 100 detects the position of the touch means by changing the capacitance when approaching and touching the touch means.

There are other methods of sensing the touch means, which will be described with reference to FIGS. 5 to 7 below.

5 is a diagram illustrating a touch means sensing method according to an embodiment of the present invention.

Referring to FIG. 5, the touch pen 500 may be used as a touch means for touching the user terminal 100. The user terminal 100 uses a capacitive touch panel.

The touch pen 500 includes a body 510 and a touch unit 512. Body 510 is made of a material that does not conduct electricity, the touch unit 512 is a conductor. Therefore, due to the touch unit 512, a change in capacitance occurs when the touch pen 500 approaches the user terminal 100 or when the user touches the user terminal 100. As a result, the user terminal 100 touches. The pen 500 may be sensed. That is, unlike the general touch pen that could not be used in the capacitive method, the touch pen 500 of the present invention can be used in the capacitive method. Of course, the entire touch pen 500 may be made of a conductor. However, in consideration of cost, it is preferable to manufacture only the touch unit 512 that is actually touched with a conductor.

6 is a view showing a touch means detection method according to another embodiment of the present invention.

Referring to FIG. 6A, a receiver 600 may be mounted on a part of the user terminal 100, and a touch pen 602 may be used.

The receiver 600 may include one infrared sensor and two ultrasonic sensors. The receiver 600 may detect the movement of the touch pen 602 by receiving infrared rays and ultrasonic waves transmitted from the touch unit (tip) of the touch pen 602. The control signal having the position information of the touch pen 602 according to the detection result is transmitted to the receiving terminal 102. The receiving terminal 102 analyzes the control signal to detect the position of the touch pen 602 and displays a second image at the detected position. As a result, the second image is displayed together with the first image. In this case, the control signal may be transmitted by the receiver 600 to the receiving terminal 102 or the user terminal 100 may be transmitted to the receiving terminal 102.

Looking at how the receiver 600 detects the position of the touch pen 602, three sensors respectively measure the transmission time of the infrared ray and the transmission time of the ultrasonic wave, converts the transmission times into distances, and converts the converted distances. The position of the touch pen 602 is detected through a method similar to triangulation. Therefore, even if the touch pen 602 is close without directly touching the user terminal 100, the position of the touch pen 602 may be sensed, and the touch and movement of the touch pen 602 may be accurately sensed. Of course, signals other than infrared and ultrasonic may also be used.

The second image reflecting the position of the touch pen 602 may not be displayed on the user terminal 100 but only on the receiving terminal 102. Of course, the user terminal 100 operates in response to the movement of the touch pen 602.

According to another embodiment of the present invention, the receiver 600 not only detects the position of the touch pen 602 but also transmits an image signal and a control signal including image data to the receiving terminal 102. Can be. In detail, the receiver 600 may include a touch controller 610, an image signal unit 612, a control signal unit 614, and a transceiver unit 616. The touch controller 610 detects the position of the touch pen 602 by using the received infrared rays and ultrasonic waves, and provides information about the detected position to the user terminal 100. The user terminal 100 displays the location of the touch pen 602 or performs a related operation according to the provided information. The image signal unit 612 receives image data from the user terminal 100, and transmits the provided image data to the receiving terminal 102 through the transceiver 616. The control signal unit 614 transmits a control signal having the detected position information of the touch pen 602 to the receiving terminal 102. That is, since the receiver 600 transmits a control signal having the image data transmitted from the user terminal 100 and the position information of the touch pen 602 to the receiving terminal 102, the user terminal 100 does not have a communication function. Not enough. Accordingly, a terminal having no communication function or a terminal having a communication function but unable to use a related communication facility may not only recognize the position and operation of the touch pen 602 using the receiver 600 but also share an image and a second device. The interlocking method of the present invention, such as display of an image, can be used.

According to another embodiment of the present invention, the receiver may be included in the user terminal 100 to be integrally implemented.

7 is a diagram illustrating a touch means sensing method according to another embodiment of the present invention.

Referring to FIG. 7, the user terminal 100 includes a touch panel 700 and an electromagnetic generator 702.

The electromagnetic generator 702 may be connected to the rear surface of the touch panel 700, and may be formed of a thin metal film to generate an electromagnetic field when electricity flows.

The touch pen 707 includes a body 710 and a touch unit 712, and the touch unit 712 may be preferably made of a small metal coil. As a result, when the touch pen 704 is close to the touch panel 700, an electromagnetic induction phenomenon occurs in the touch unit 712, and as a result, a deformation of the electromagnetic field generated from the electromagnetic generator 702 occurs. Therefore, the user terminal 100 detects the deformation of the electromagnetic field and recognizes the position of the touch pen 704. In particular, since the deformation of the electromagnetic field varies depending on the approach position and the touch intensity of the touch pen 704, the touch means sensing method detects the touch pen 700 and the touch pressure of the touch pen 700 in detail. can do. Therefore, the size of the second image displayed on the receiving terminal 102, the thickness of the line when represented by a line, and the like may be displayed differently according to the position or pressure of the touch pen 704. Of course, the shape, size or color of the second image may vary depending on the touch intensity.

5 to 7, the user terminal 100 may detect the touch pen through various methods and reflect the detection result to the receiving terminal 102.

As described above, the user terminal 100 detects the touch means close to the detectable area and transmits the location information of the touch means to the receiving terminal 102, where the detectable area is a predetermined distance from the user terminal 100. It may be spaced apart. That is, the user terminal 100 may detect a touch means close to the user terminal 100 located within a predetermined distance from the user terminal 100. The preset distance may vary depending on the sensing scheme described above. For example, when the capacitive touch panel is used in the user terminal 100, the preset distance may be a distance corresponding to the low level L1 described with reference to FIG. 3. In addition, the preset distance may be set based on the shortest distance between the touch means and the user terminal 100.

8 is a flowchart illustrating a terminal interworking and control method according to a second embodiment of the present invention.

Referring to FIG. 8, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S800).

Subsequently, the user terminal 100 transmits image data corresponding to the currently displayed first image to the receiving terminal 102, and the receiving terminal 102 displays the first image corresponding to the image data (S802). . That is, the user terminal 100 and the receiving terminal 102 share the first image.

Subsequently, the user terminal 100 detects a touch means such as a finger or a touch pen through various methods (S804).

Subsequently, the user terminal 100 generates a composite image displaying a second image corresponding to the position of the sensed touch means together with the currently displayed first image, and generates composite image data (synthesis information) corresponding to the composite image. ) Is transmitted to the receiving terminal 102, and the receiving terminal 102 displays the composite image corresponding to the composite image data (S806). As a result, the fisheries terminal 102 displays a screen on which the second image is displayed together with the first image. In this case, only the first image may be displayed on the user terminal 100, or a screen on which the second image is displayed together with the first image may be displayed.

Subsequently, when the user touches or moves in the touched state, the user terminal 100 transmits a control signal having the position information of the touch means to the receiving terminal 102, and the receiving terminal 102 transmits the touch signal. According to a control signal, a second image indicating the position of the touch means is displayed together with the corresponding first image (S808). That is, the movement of the touch means is reflected on the screen of the receiving terminal 102. However, the first image on the receiving terminal 102 may be the same image as the previous image or may be different from the previous image.

In summary, unlike the first embodiment in which the receiving terminal 102 generates the second image and displays the second image together, in the present embodiment, the user terminal 100 synthesizes the first image and the second image. A composite image is generated and the generated composite image is transmitted to the receiving terminal 102. As a result, the receiving terminal 102 displays a screen on which the second image is displayed together with the first image.

9 is a flowchart illustrating a terminal interworking and control method according to a third embodiment of the present invention.

9, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S900). The interworking may be automatically started when the user terminal 100 and the receiving terminal 102 are connected, or may be started depending on the user's selection, asking whether the user is selected.

Subsequently, the user or the user terminal 100 automatically sets a plurality of detection levels for detecting a touch means such as a finger or a touch pen (S902). Specifically, a first level for detecting the touch means when the touch means is close to the user terminal 100 and a second level for detecting the touch means when the touch means touches are set. Of course, the levels for sensing the touch means may be set more precisely.

Subsequently, the user terminal 100 detects the touch means and displays a second image on the receiving terminal 102 indicating the position of the touch means (S904).

Subsequently, it is determined whether or not there is a request for interlocking stop by the user (S906). The interworking stop may be requested by the user by selecting a menu or by turning off the connection between the user terminal 100 and the receiving terminal 102. When the user controls the user terminal 100 while viewing the receiving terminal 102 and wants to use only the user terminal 100 or only wants to view the receiving terminal 102, the user terminal 100 and the receiving terminal 102 You can request to stop interlocking while connected.

If there is no user stop request for interworking, step S904 is performed again.

On the other hand, when the user stops the interlocking request of the user, the user terminal 100 initializes the plurality of levels to change to the sensing levels to detect only one level is present or touched (S908). That is, when the touch means is not touched, the user terminal 100 does not sense the touch means but changes the level so as to detect the touch means only when the touch means is touched.

In summary, in the terminal interworking and control method of the present invention, a user may freely set and change an interworking request, an interlocking stop request, and a sensing level.

According to another embodiment of the present invention, the first detection level may be set when the user terminal 100 and the reception terminal 102 start interlocking, and the detection level may be differently set during the interlocking. For example, the user may set different sensing levels according to the approach distance and the touch intensity of the touch means during interlocking, or change the sensing level setting to sense only when the touch means is touched.

10 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.

Referring to FIG. 10, the user terminal 100 and the receiving terminal 102 interlock with each other, that is, display the second image 1000 indicating the position of the touch means while sharing the first image. However, when the first image is shared, the receiving terminal 102 transmits image data to the user terminal 100 to share the first image.

When the receiving terminal 102 is an IPTV, as shown in FIG. 10, user interfaces (1010s) for selecting a program such as a movie or a drama may be displayed, and the first image on which the UIs 1010 are displayed may be displayed. It may be shared with the user terminal 100. Of course, the second terminal 1000 is displayed on the receiving terminal 102 together with the first image.

Looking at the advantages of sharing the first image displayed on the receiving terminal 102 to the user terminal 100, the user is watching the receiving terminal 102, for example TV, the user terminal 100, for example smart The phone can be used to control the operation on the receiving terminal 102. For example, the user may use the user terminal 100 to select a specific program, send an e-mail, perform an e-commerce transaction, or play a game through the reception terminal 102.

In addition, since the same first image as that of the receiving terminal 102 is displayed on the user terminal 100, the user may use the user terminal 100 to receive the screen of the receiving terminal 102. You can enjoy various services provided. For example, the user may control the receiving terminal 102 in the living room from the room to perform electronic commerce or the like.

11 is a flowchart illustrating a terminal interworking and control method according to a fourth embodiment of the present invention.

Referring to FIG. 11, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S1100).

Subsequently, the receiving terminal 102 transmits the image data corresponding to the first image currently displayed or the first image to be displayed to the user terminal 100, and the user terminal 100 transmits the image data corresponding to the transmitted image data. 1 Display an image on the screen (S1102). That is, the user terminal 100 and the receiving terminal 102 share the first image.

Subsequently, when a user approaches a touch means such as a finger or a touch pen, the user terminal 100 detects the touch means (S1104). The user terminal 100 may detect the touch means by using various methods such as a capacitance sensing method and an electromagnetic sensing method.

Subsequently, the user terminal 100 transmits a control signal having the position information of the touch means to the receiving terminal 102 according to the detection result, and the receiving terminal 102 is connected with the first image according to the transmitted control signal. Together, a second image indicating the position of the touch means is displayed (S1106). According to another embodiment of the present invention, the user terminal 100 may transmit the composite image data corresponding to the composite image on which the second image is displayed together with the first image to the receiving terminal 102. However, since the reception terminal 102 basically transmits the image data corresponding to the first image to the user terminal 100, the user terminal 100 transmits the control signal to the reception terminal 102. The manner is preferred.

Subsequently, when the user touches or moves the touch means, the user terminal 100 transmits a control signal having the position information of the touch means to the receiving terminal 102 to display the second image on the receiving terminal 102 ( S1108).

12 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.

Referring to FIG. 12A, in the terminal interworking system of the present invention, the receiver 1200 may be installed or embedded in the receiving terminal 102 instead of the user terminal 100.

When the touch pen 1202 transmits ultrasonic waves and infrared rays to the receiver 1200 installed in the receiving terminal 102 as a touch means used in the user terminal 100, the receiver 1200 receives the ultrasonic waves and infrared rays and touch pens. Detects the position of 1202.

The receiver 1200 transmits information on the detected position of the touch pen 1202 to the receiving terminal 102, and the receiving terminal 102 transmits a second image indicating the position of the touch pen 1202 to the first terminal. Display with the image. Of course, the user terminal 100 and the receiving terminal 102 display the second image in a state in which the first image is shared.

According to another embodiment of the present invention, the receiver 1200 may perform not only the position detection of the touch pen 1202 but also a communication role. In detail, the receiver 1200 may include a touch means detector 1210, an image signal unit 1212, a control signal unit 1214, and a transceiver 1216. The touch means detector 1210 detects the position of the touch pen 1202. The image signal unit 1212 receives an image signal that has delivered image data transmitted from the user terminal 100 through the transceiver 1216, and receives image data of the received image signal through the transceiver 1216. Send to 102 to display the first image. Of course, the image signal unit 1212 may receive the image data from the receiving terminal 102 and transmit the image signal that has distributed the received image data to the user terminal 100. The control signal unit 1214 may receive a control signal related to an interlocking operation and the like transmitted from the user terminal 100 and transmit the received control signal to the receiving terminal 102. Of course, the control signal having the location information of the touch pen 1202 is not transmitted from the user terminal 100 to the receiver 1200. In conclusion, the receiver 1200 not only detects the position of the touch pen 1202 but also performs a communication function. Therefore, the receiving terminal 102 can use the interworking and control method of the present invention without having a communication function.

13 is a flowchart illustrating a terminal interworking and control method according to a fifth embodiment of the present invention.

Referring to FIG. 13, the receiver 1200 is installed in the receiving terminal 102 (S1300). According to another embodiment of the invention, the receiver may be embedded in the receiving terminal 102.

Subsequently, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S1302).

Subsequently, the user terminal 100 transmits image data corresponding to the first image to the receiving terminal 102 to share the first image, or the receiving terminal 102 transmits the image data to the user terminal 100. The first image is shared (S1304).

Subsequently, when the touch means, for example, the touch pen 1202 is close to the user terminal 100, the receiver 1200 detects the position of the touch pen 1202 through infrared and ultrasonic waves transmitted from the touch pen 1202. The transmission is transmitted to the reception terminal 102, and the reception terminal 102 displays a second image such as a shadow according to the detected position together with the first image (S1306).

Subsequently, when the user touches or moves after touching the user terminal 100 with the touch pen 1202, the receiver 1200 detects the movement of the touch pen 1202 and transmits the movement to the receiving terminal 102. 102 displays the corresponding second image together with the first image according to the movement (S1308).

14 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.

Referring to FIG. 14A, the terminal interworking system according to the present embodiment includes a user terminal 100, a receiving terminal 102, and a dongle 1400.

The dongle 1400 serves not only to assist communication of the user terminal 100 but also to sense the position of the touch pen 1402 as a touch means. Specifically, the dongle 1400 may include a communication unit 1410 and a touch means detector 1412.

The communication unit 1410 is a communication means for the user terminal 100, and may provide a specific communication function, for example, a Wibro communication function, and converts the specific communication function into another communication function, for example, Wi- It may be converted to Fi and used for the user terminal 100.

The touch means detecting unit 1412 may detect the position of the touch pen 1402 similarly to the receiver, and may transmit information about the detected position to a control signal and transmit the information to the receiving terminal 102.

In summary, the user terminal 100 has a communication function through the dongle 1400, and the dongle 1400 is also used as a means for detecting the position of the touch pen 1402.

Hereinafter, a receiver and a dongle in the case of being separated from the user terminal 100 will be collectively referred to as an external device.

15 is a block diagram showing the structure of a user terminal according to an embodiment of the present invention.

Referring to FIG. 15, the user terminal 100 of the present embodiment includes a control unit 1500, an interlocking unit 1502, an image unit 1504, a sensing unit 1506, a setting unit 1508, a display unit 1510, A signal unit 1512, a transceiver 1514, and a storage unit 1516 are included.

The interlocking unit 1502 manages all functions related to interworking with the receiving terminal 102.

The image unit 1504 may generate a first image to be shared with the receiving terminal 102 or generate a composite image of the second image and the first image. The image unit 1504 displays the first image through the display unit 1510.

The sensor 1506 detects a touch means such as a finger or a touch pen. Specifically, the sensing unit 1506 detects the position of the touch means by distinguishing when the touch means is close to when it is touched. The sensing method is not limited, such as capacitive and electromagnetic induction. Information about the position of the touch means sensed by the detector 1506 may be generated by a location information generator (not shown).

The setting unit 1508 manages setting of various functions such as setting of an interlocking function and setting of a detection level.

The display unit 1510 is a touch panel and may be implemented in various ways, such as a capacitive type, a resistive film type, and an electromagnetic induction type.

The signal unit 1512 generates an image signal that distributes image data corresponding to the first image and a control signal having information about the position of the touch means. In addition, the signal unit 1512 may transmit the image signal to the receiving terminal 102 through the data channel and the control signal to the receiving terminal 102 through the control channel.

The transceiver unit 1514 serves as a communication path with the receiving terminal 102.

The storage unit 1516 stores various data such as a first image, an image signal, a control signal, an application program, and the like.

The controller 1500 generally controls the operations of the components of the user terminal 100.

FIG. 16 is a block diagram illustrating a structure of the image unit of FIG. 15 according to an embodiment of the present invention. FIG.

Referring to FIG. 16, the image unit 1504 of the present embodiment includes a first image generator 1600, a second image generator (position information generator 1602), and an image synthesizer 1604.

The first image generator 1600 generates a shared first image, and the second image generator 1602 generates a second image such as a shadow image corresponding to the position of the touch means.

The image synthesizer 1604 generates a composite image on which the second image is displayed along with the generated first image.

According to another embodiment of the present invention, the user terminal 100 transmits an image signal that has delivered image data corresponding to the first image to the receiving terminal 102, and is not a control signal having information on the position of the touch means. The control signal in which the position indication data corresponding to the second image is distributed may be transmitted to the receiving terminal 102.

17 is a block diagram showing the structure of a user terminal according to a second embodiment of the present invention. 17 illustrates a structure of a user terminal 100 in a terminal interworking system in which a receiver is installed in the user terminal 100.

Referring to FIG. 17, the user terminal 100 according to the present embodiment includes a control unit 1700, an interlocking unit 1702, an image unit 1704, a touch unit 1706, a setting unit 1708, and a display unit 1710. It may include a signal unit 1712, a transceiver 1714, and a storage 1716.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image unit 1704 generates a first image to be shared with the receiving terminal 102 or generates a composite image of the first image and the second image.

The touch manager 1706 receives and manages information on the position of the touch means from the receiver. That is, the receiver detects the position of the touch means, and the touch manager 1706 detects the position of the touch means by analyzing a signal transmitted from the receiver.

The signal unit 1712 generates an image signal that delivers image data corresponding to the first image, a control signal having information about a position of the touch means, or a synthesized signal that distributes data corresponding to a composite image. In addition, the signal unit 1712 may transmit the image signal or the composite signal to the receiving terminal 102 through a data channel and transmit the control signal to the receiving terminal 102 through a control channel.

18 is a block diagram showing the structure of a user terminal according to a third embodiment of the present invention. 18 illustrates a structure of a user terminal 100 in a terminal interworking system in which a receiver is embedded in the user terminal 100.

Referring to FIG. 18, the user terminal 100 of the present embodiment includes a control unit 1800, an interlocking unit 1802, a transceiver 1804, an image unit 1806, a receiver unit 1808, a setting unit 1810, and the like. The display unit 1812, the signal unit 1814, and the storage unit 1816 may be included.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image unit 1806 generates a first image to be shared with the receiving terminal 102 or generates a composite image of the first image and the second image.

The receiver 1808 receives infrared rays and ultrasonic waves transmitted from the touch means when the touch means approaches or touches the user terminal 100, and analyzes the received infrared rays and ultrasonic waves to detect the position of the touch means.

The signal unit 1814 generates an image signal for distributing image data corresponding to the first image, a control signal having information about the position of the touch means, or a synthesized signal for distributing composite image data corresponding to the composite image. In addition, the signal unit 1814 may transmit the image signal or the composite signal to the receiving terminal 102 through a data channel and transmit the control signal to the receiving terminal 102 through a control channel.

19 is a block diagram showing the structure of a user terminal according to a fourth embodiment of the present invention. 19 illustrates a structure of a user terminal 100 using an electromagnetic induction method.

Referring to FIG. 19, the user terminal 100 of the present embodiment may include a control unit 1900, an interlocking unit 1902, an image unit 1904, an electromagnetic generator 1906, a detection unit 1908, and a setting unit 1910. The display unit 1912 may include a signal unit 1914, a transceiver 1916, and a storage unit 1920.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image unit 1904 generates a first image to be shared with the receiving terminal 102 or generates a composite image of the first image and the second image.

The electromagnetic generator 1906 serves to generate electromagnetic waves in order to sense the touch means by an electromagnetic induction method, and may be disposed on the rear surface of the display unit 1912.

The sensing unit 1908 senses the position of the touch means by sensing the electromagnetic change caused by the coil of the touch means.

The signal unit 1914 generates an image signal for distributing image data corresponding to the first image, a control signal having information on the position of the touch means, or a synthesized signal for distributing composite image data. In addition, the signal unit 1914 may transmit the image signal or the image signal to the receiving terminal 102 through a data channel and the control signal to the receiving terminal 102 through a control channel.

20 is a block diagram showing the structure of a user terminal according to a fifth embodiment of the present invention. 20 illustrates the structure of the user terminal 100 when the receiving terminal 102 transmits image data to the user terminal 100.

Referring to FIG. 20, the user terminal 100 of the present embodiment includes a control unit 2000, an interlocking unit 2002, a transceiver unit 2004, an image signal unit 2006, an image unit 2008, and a detection unit 2010. , A signal unit 2012, a display unit 2014, a setting unit 2016, and a storage unit 2018.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image signal unit 2006 receives an image signal transmitted from the receiving terminal 102 through the transceiver 2004 and analyzes the received image signal.

The image unit 2008 generates a first image corresponding to the image signal analyzed by the image signal unit 2006 or generates a composite image of the first image and the second image.

The detector 2010 detects the position of the touch means.

The signal unit 2012 generates a composite signal in which the control signal having the information on the position of the touch means or the composite image data is distributed. In addition, the signal unit 1914 may transmit the synthesized signal to the receiving terminal 102 through the data channel and the control signal to the receiving terminal 102 through the control channel.

21 is a block diagram showing the structure of a receiving terminal according to the first embodiment of the present invention.

The receiving terminal 102 of the present embodiment includes a control unit 2100, an interlocking unit 2102, a transmitting and receiving unit 2104, an image signal unit 2106, a control signal unit 2108, an image unit 2110, and a display unit 2112. ) And a storage unit 2114.

The linkage unit 2102 manages a linkage function with the user terminal 100.

The transceiver 2104 serves as a communication path of the user terminal 100.

The image signal unit 2106 receives an image signal that has delivered image data transmitted from the user terminal 100, and analyzes the received image signal to detect image data.

The control signal unit 2108 receives a control signal transmitted from the user terminal 100 and analyzes the received control signal to detect information on the position of the touch means.

The image unit 2110 displays the first image corresponding to the detected image data through the display unit 2112 and, together with the first image, a shadow corresponding to the information on the position of the detected touch means. Display the second image. According to another embodiment of the present invention, the image unit 2110 generates a composite image in which the second image is displayed together with the first image through the image data and the information, and displays the generated composite image. Display via 2112.

The display unit 2114 may be implemented with various display elements such as LCD, OLED, PDP, etc. without limiting to display an image, and does not need to have a touch function.

The storage unit 2114 stores various data such as a first image, a second image, a composite image, an application program, and the like.

The controller 2100 generally controls the operations of the components of the receiving terminal 102.

22 is a block diagram showing the configuration of a receiving terminal according to a second embodiment of the present invention. FIG. 22 shows the structure of the receiving terminal 102 when the composite image is transmitted from the user terminal 100 to the receiving terminal 102.

Referring to FIG. 22, the reception terminal 102 of the present embodiment includes a control unit 2200, an interlocking unit 2202, a transceiver 2204, a signal unit 2206, an image unit 2208, a display unit 2210, and the like. It may include a storage unit 2212.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The signal unit 2206 receives the image signal and the synthesized signal from the user terminal 100, and analyzes the received image signal and the synthesized signal. The signal unit 2206 receives the image signal at the start of interlocking, and receives a composite signal when the user terminal 100 detects the touch means. The composite signal includes data corresponding to the composite image of the first image and the second image. According to another exemplary embodiment of the present invention, the signal unit 2206 may receive an image signal that has delivered image data and a control signal that has delivered data corresponding to the second image.

The image unit 2208 displays the synthesized image corresponding to the data included in the synthesized signal through the display unit 2210. As a result, the reception terminal 102 displays a screen on which the second image is displayed together with the first image. According to another exemplary embodiment of the present disclosure, the image unit 2208 may display the first image corresponding to the image data through the display unit 2210 and display the second image together with the first image.

23 is a block diagram showing the configuration of a receiving terminal according to a third embodiment of the present invention. 23 shows the structure of the receiving terminal 102 when the receiver is installed in the receiving terminal 102.

Referring to FIG. 23, the receiving terminal 102 of the present embodiment includes a control unit 2300, an interlocking unit 2302, a transceiver 2304, a signal unit 2306, a receiver unit 2308, an image unit 2310, The display unit 2312 and the storage unit 2314 may be included.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The signal unit 2306 receives an image signal from which the image data has been distributed from the user terminal 100 and analyzes the received image signal.

The receiver unit 2308 receives and analyzes information on the position of the touch means from the receiver installed in the receiving terminal 102.

The image unit 2310 displays a first image corresponding to the image data through the display unit 2312, and displays a second image corresponding to the position of the touch unit together with the first image. According to another embodiment of the present invention, the image unit 2310 generates a composite image of the first image and the second image, and displays the generated composite image through the display unit 2312.

24 is a block diagram showing a configuration of a receiving terminal according to a fourth embodiment of the present invention. 24 shows the structure of the receiving terminal 102 when the receiver is built in the receiving terminal 102.

Referring to FIG. 24, the reception terminal 102 of the present embodiment includes a control unit 2400, an interlocking unit 2402, a transceiver 2404, a signal unit 2406, a receiver unit 2408, an image unit 2410, It may include a display unit 2412 and a storage unit 2414.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The receiver 2408 detects the position of the touch means by detecting infrared rays and ultrasonic waves transmitted from the touch means approaching or touching the user terminal 100.

The image unit 2410 displays the first image corresponding to the image data through the display unit 2412, and the second image corresponding to the position of the touch means detected by the receiver unit 2408 together with the first image. Display the image. According to another embodiment of the present invention, the image unit 2410 generates a composite image of the first image and the second image, and displays the generated composite image through the display unit 2312.

25 is a block diagram showing a configuration of a receiving terminal according to a fifth embodiment of the present invention. 25 shows the structure of the receiving terminal 102 when the receiving terminal 102 transmits the image data to the user terminal 100.

Referring to FIG. 25, the reception terminal 102 of the present embodiment includes a control unit 2500, an interlocking unit 2502, a signal unit 2504, an image unit 2506, a display unit 2508, a transceiver unit 2510, and the like. The storage unit 2512 may be included.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The signal unit 2504 generates an image signal in which image data corresponding to the first image to be displayed or displayed on the receiving terminal 102 is distributed, and the generated image signal is transmitted to the user terminal 100 through the transceiver unit 2510. To send). In addition, the signal unit 2504 receives and analyzes the information on the position of the touch means transmitted from the user terminal 100 or the control signal having data corresponding to the second image or the synthesized signal that has distributed the data for the synthesized image. .

The image unit 2506 generates a second image corresponding to the position of the touch means, and displays the second image together with the first image through the display unit 2508. According to another embodiment of the present invention, the image unit 2506 may generate a composite image of the first image and the second image, and display the generated composite image through the display unit 2508.

The embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art having ordinary knowledge of the present invention may make various modifications, changes, and additions within the spirit and scope of the present invention. Should be considered to be within the scope of the following claims.

100: user terminal 102: receiving terminal
400: second image 500: touch pen
600: receiver 602: touch pen
700: touch panel 702: electromagnetic generator
704: touch pen 1000: the second image
1010: UI 1200: Receiver
1202: Touch Pen 1400: Dongle
1402: touch pen

Claims (23)

A signal unit for receiving an image signal that has distributed the image data, and for receiving a control signal having position information of the touch means in proximity to the user terminal; And
And a display unit configured to display a second image corresponding to the position information of the touch means included in the received control signal. The method of claim 1,
And an image unit configured to display the first image and the second image corresponding to the image data of the received image signal through the display unit.
The first image is shared between the user terminal and the receiving terminal, and the second image is displayed together with the first image. The receiver terminal of claim 1, wherein the image signal is transmitted from the user terminal through a data channel, and the control signal is transmitted from the user terminal through a control channel. The method of claim 3, wherein the user terminal is a smartphone and the receiving terminal is a TV,
The receiving terminal has a larger size than the user terminal, and the control channel is CEC (Consumer Electronic Control), DDC (Display Data Channel), Utility, SCL / SDA, MBUS, CBUS, Auxiliary in Displayport. Receiving terminal used in the terminal interworking system, characterized in that the channel. The method of claim 1, wherein the image signal and the control signal are transmitted from the user terminal to the receiving terminal through the same channel.
The image signal is transmitted in data intervals, and the control signal is used in a terminal interworking system, characterized in that in the blank interval between the data intervals. The display apparatus of claim 1, wherein the display unit displays the second image in the current channel after stopping a program that is currently being broadcast, displays the second image in the other channel after switching the current channel to another channel, Receiving terminal, characterized in that for displaying the second image on the current channel while the program is being displayed in the form of a pop-up. The method of claim 1,
The method may further include an image unit configured to generate a composite image obtained by synthesizing the first image and the second image corresponding to the image data of the received image signal, and to display the generated composite image through the display unit.
And the second image is displayed together with the first image. The apparatus of claim 1, wherein the signal unit receives location information of the touch means when the touch means touches the user terminal from the user terminal,
The second image according to the position information when the touch has a shape different from the second image according to the position information when the touch means close to the receiving terminal used in the terminal interworking system. A control signal unit for receiving an image signal that has distributed image data, and for receiving a control signal having position indication data corresponding to a second image indicating a position of the touch means proximate to the user terminal; And
And a display unit configured to display a second image corresponding to the position indication data included in the received control signal. 10. The method of claim 9,
And an image unit configured to display the first image and the second image corresponding to the image data of the received image signal through the display unit.
The image signal is transmitted from the user terminal, the first image is shared between the user terminal and the receiving terminal, the second image is used in a terminal interworking system characterized in that displayed together with the first image. Receiving terminal. The method of claim 9, wherein the receiving terminal,
The method may further include an image unit configured to generate a composite image obtained by synthesizing the first image and the second image corresponding to the image data of the received image signal, and to display the generated composite image through the display unit.
And the second image is displayed together with the first image. In a receiving terminal,
A signal unit configured to receive composite image data corresponding to a composite image of a second image indicating a position of the touch means close to the user terminal and the first image to be displayed on the user terminal; And
And a display unit configured to display a composite image corresponding to the received composite image data. The method of claim 12, wherein the composite image data is transmitted from the user terminal to the receiving terminal, wherein the user terminal and the receiving terminal share the first image, and the second image is displayed together with the first image. Receiving terminal used in the terminal interworking system, characterized in that. A signal unit for transmitting an image signal having image data to a user terminal and receiving a control signal having position information of a touch means in proximity to the user terminal; And
And a display unit configured to display a second image corresponding to the position information of the touch means included in the received control signal. 15. The method of claim 14,
The display device may further include an image unit configured to display the first image and the second image corresponding to the image data through the display unit.
The second image is displayed together with the first image, and the user terminal displays the first image corresponding to the image data to share the first image with the receiving terminal. Receiving terminal. The method of claim 14, wherein the image signal is transmitted to the user terminal through a data channel, and the control signal is received to the receiving terminal through a control channel.
The control channel is CEC (Consumer Electronic Control), DDC (Display Data Channel), Utility, SCL / SDA, CHL in the MHL method, auxiliary channel in the DisplayPort method is used for the terminal interworking system terminal. A signal unit for transmitting an image signal having image data to a user terminal and receiving a control signal having position indication data corresponding to a second image indicating a position of a touch means proximate to the user terminal; And
And a display unit configured to display a second image corresponding to the position indication data included in the received control signal. 18. The method of claim 17,
The display apparatus may further include an image unit configured to display a first image corresponding to the image data and to display the second image.
The second image is displayed together with the first image, and the user terminal displays the first image corresponding to the image data to share the first image with the receiving terminal. Receiving terminal. A signal for transmitting an image signal having image data to a user terminal and receiving composite image data corresponding to a composite image of a second image representing a position of a touch means proximate to the user terminal and a first image to be displayed on the user terminal part; And
And a display unit configured to display a composite image corresponding to the received composite image data. 20. The method of claim 19,
Further comprising an image unit for displaying the composite image,
The second image is displayed together with the first image, and the user terminal displays the first image corresponding to the image data to share the first image with the receiving terminal. Receiving terminal. A signal unit for receiving image data from the user terminal and position information of the touch means in proximity to the user terminal; And
And a display configured to display a second image of the location information. The method of claim 21,
The display apparatus may further include an image unit configured to generate the second image by using the location information.
And the display unit displays the image data and the second image together. A signal unit for receiving composite image data from a user terminal; And
Including a display unit for displaying the composite image data,
And the composite image data has first image data and second image data indicating a position of a touch means proximate to the user terminal.
KR1020120022986A 2012-03-06 2012-03-06 System for interworking and controlling devices and reception device used in the same KR20130101886A (en)

Priority Applications (9)

Application Number Priority Date Filing Date Title
KR1020120022986A KR20130101886A (en) 2012-03-06 2012-03-06 System for interworking and controlling devices and reception device used in the same
US13/785,266 US20130244730A1 (en) 2012-03-06 2013-03-05 User terminal capable of sharing image and method for controlling the same
US13/785,600 US20130234984A1 (en) 2012-03-06 2013-03-05 System for linking and controlling terminals and user terminal used in the same
US13/785,370 US20130234959A1 (en) 2012-03-06 2013-03-05 System and method for linking and controlling terminals
US13/785,498 US8913026B2 (en) 2012-03-06 2013-03-05 System for linking and controlling terminals and user terminal used in the same
US13/785,302 US20130241854A1 (en) 2012-03-06 2013-03-05 Image sharing system and user terminal for the system
US14/155,204 US20140125622A1 (en) 2012-03-06 2014-01-14 System for linking and controlling terminals and user terminal used in the same
US15/052,803 US20160170703A1 (en) 2012-03-06 2016-02-24 System and method for linking and controlling terminals
US16/142,998 US10656895B2 (en) 2012-03-06 2018-09-26 System for linking and controlling terminals and user terminal used in the same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020120022986A KR20130101886A (en) 2012-03-06 2012-03-06 System for interworking and controlling devices and reception device used in the same

Publications (1)

Publication Number Publication Date
KR20130101886A true KR20130101886A (en) 2013-09-16

Family

ID=49451868

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020120022986A KR20130101886A (en) 2012-03-06 2012-03-06 System for interworking and controlling devices and reception device used in the same

Country Status (1)

Country Link
KR (1) KR20130101886A (en)

Similar Documents

Publication Publication Date Title
US10656895B2 (en) System for linking and controlling terminals and user terminal used in the same
CN105278674B (en) Radar-based gesture recognition through wearable devices
US10338776B2 (en) Optical head mounted display, television portal module and methods for controlling graphical user interface
US20120274547A1 (en) Techniques for content navigation using proximity sensing
US9513802B2 (en) Methods for displaying a user interface on a remote control device and a remote control device applying the same
US20160091979A1 (en) Interactive displaying method, control method and system for achieving displaying of a holographic image
KR100980741B1 (en) A remote controller and a method for remote contrlling a display
CN111052063B (en) Electronic device and control method thereof
KR20140134453A (en) Input apparatus, display apparatus and control method thereof
KR101384493B1 (en) System for interworking and controlling devices and user device used in the same
KR20110134810A (en) A remote controller and a method for remote contrlling a display
US10386932B2 (en) Display apparatus and control method thereof
KR101212364B1 (en) System for interworking and controlling devices and user device used in the same
US20130244730A1 (en) User terminal capable of sharing image and method for controlling the same
KR101151549B1 (en) System for interworking and controlling devices and user device used in the same
WO2022017421A1 (en) Interaction method, display device, emission device, interaction system, and storage medium
KR101951484B1 (en) System for interworking and controlling devices and reception device used in the same
KR20130101886A (en) System for interworking and controlling devices and reception device used in the same
KR101515912B1 (en) User Terminal Capable of Sharing Image and Method for Controlling the Same
KR20130129684A (en) System for interworking and controlling devices and user device used in the same
US20130314318A1 (en) Method of improving cursor operation of handheld pointer device in a display and handheld pointer device with improved cursor operation
US20120212513A1 (en) Method Of Improving Operation Of Handheld Pointer Device In A Display Screen
KR20130129693A (en) System for interworking and controlling devices and user device used in the same

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
AMND Amendment
E601 Decision to refuse application
AMND Amendment
E902 Notification of reason for refusal
AMND Amendment