KR20130129693A - System for interworking and controlling devices and user device used in the same - Google Patents

Abstract

The present invention discloses a system for interlocking and controlling a terminal. A user terminal used in the system includes an image unit which outputs image data which is shared with a receiving terminal and a signal unit which transmits an image signal which includes the image data which is outputted from the image unit and a control signal which includes the location information of a touch unit which approaches the user terminal to the receiving terminal.

Description

Terminal interworking and control system and user terminal used in the same {SYSTEM FOR INTERWORKING AND CONTROLLING DEVICES AND USER DEVICE USED IN THE SAME}

Terminal interworking and control system and user terminal used in the same {SYSTEM FOR INTERWORKING AND CONTROLLING DEVICES AND USER DEVICE USED IN THE SAME}

Recently, people enjoy playing games, checking e-mails or surfing the web by using electronic terminals such as smartphones. The electronic terminal has the advantage of being convenient to carry because it is light and small, but there is also a disadvantage that the screen is not small enough to enjoy games. That is, due to the small size of the electronic terminal, the enjoyment of games, web surfing, etc. is halved and it is inconvenient to check e-mail.

The present invention provides a method and system for interlocking a large sized terminal with a small sized terminal and controlling the small sized terminal while viewing the large sized terminal.

In order to achieve the above object, the user terminal used in the terminal interworking system according to an embodiment of the present invention includes an image unit for outputting image data to be shared with the receiving terminal; And a signal unit for transmitting an image signal having the image data output from the image unit and a control signal having position information of the touch means close to the user terminal to the receiving terminal.

A user terminal used in a terminal interworking system according to another embodiment of the present invention displays an image data corresponding to a first image to be shared with a receiving terminal and a position of the touch means when the touch means approaches the user terminal. An image unit generating position indication data corresponding to the second image; And a signal unit for transmitting the image data and the position indication data generated by the image unit to the receiving terminal.

A user terminal used in a terminal interworking system according to another embodiment of the present invention includes a first image to be shared with a receiving terminal and a second image indicating a position of the touch means when the touch means is close to the user terminal. An image unit configured to output synthesized image data corresponding to the synthesized synthesized image; And a signal unit which transmits the synthesized signal which has distributed the composite image data output from the image unit to the receiving terminal.

A user terminal used in a terminal interworking system according to another embodiment of the present invention includes an image signal unit for receiving image data corresponding to a first image; And a signal unit for transmitting a control signal having position information of the touch means close to the user terminal to the receiving terminal.

A user terminal used in a terminal interworking system according to another embodiment of the present invention includes an image signal unit for receiving image data corresponding to a first image; And a signal unit for transmitting the position indication data corresponding to the second image indicating the position information of the touch means close to the user terminal to the receiving terminal.

A user terminal used in a terminal interworking system according to another embodiment of the present invention includes an image signal unit for receiving image data corresponding to a first image; And a signal unit configured to transmit the composite image data corresponding to the composite image of the first image corresponding to the second image representing the position information of the touch means close to the user terminal and the received image data to the receiving terminal.

A user terminal used in a terminal interworking system according to another embodiment of the present invention includes a sensing unit for detecting the position of the touch means in the proximity of the touch means; And a signal unit for transmitting a control signal having information about the position of the detected touch means to the receiving terminal.

A user terminal according to another embodiment of the present invention includes a display unit; And

It includes a signal unit for transmitting the image information and the location information of the touch means close to the user terminal to the receiving terminal.

A user terminal according to another embodiment of the present invention includes a display unit; And

And a signal unit for transmitting the composite image data to the receiving terminal. Here, the composite image data has first image data and second image data indicating a position of a touch means proximate to the user terminal.

Since the method and system for connecting and controlling a terminal according to the present invention share a first image with terminals, a second image related to a user's control of a small terminal is displayed on the first image of the large terminal. The user can control the operation of the small size terminal while looking at the large size terminal. Therefore, the user can fully enjoy games, surfing the web, and the like through a small size terminal.

1 is a view schematically showing a terminal interworking and control system according to an embodiment of the present invention.
2 is a flowchart illustrating a terminal interworking and control method according to a first embodiment of the present invention.
3 is a diagram illustrating a control method used in the terminal interworking and control method of FIG. 2.
4 is a plan view illustrating an image sharing screen according to an exemplary embodiment of the present invention.
5 is a diagram illustrating a touch means sensing method according to an embodiment of the present invention.
6 is a view showing a touch means detection method according to another embodiment of the present invention.
7 is a diagram illustrating a touch means sensing method according to another embodiment of the present invention.
8 is a flowchart illustrating a terminal interworking and control method according to a second embodiment of the present invention.
9 is a flowchart illustrating a terminal interworking and control method according to a third embodiment of the present invention.
10 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.
11 is a flowchart illustrating a terminal interworking and control method according to a fourth embodiment of the present invention.
12 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.
13 is a flowchart illustrating a terminal interworking and control method according to a fifth embodiment of the present invention.
14 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.
15 is a block diagram showing the structure of a user terminal according to an embodiment of the present invention.
FIG. 16 is a block diagram illustrating a structure of the image unit of FIG. 15 according to an embodiment of the present invention. FIG.
17 is a block diagram showing the structure of a user terminal according to a second embodiment of the present invention.
18 is a block diagram showing the structure of a user terminal according to a third embodiment of the present invention.
19 is a block diagram showing the structure of a user terminal according to a fourth embodiment of the present invention.
20 is a block diagram showing the structure of a user terminal according to a fifth embodiment of the present invention.
21 is a block diagram showing the structure of a receiving terminal according to the first embodiment of the present invention.
22 is a block diagram showing the configuration of a receiving terminal according to a second embodiment of the present invention.
23 is a block diagram showing the configuration of a receiving terminal according to a third embodiment of the present invention.
24 is a block diagram showing a configuration of a receiving terminal according to a fourth embodiment of the present invention.
25 is a block diagram showing a configuration of a receiving terminal according to a fifth embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Terminal interworking and control system of the present invention relates to a system for interlocking and controlling the terminals, in particular a system for controlling the operation of the terminal based on the image sharing and the shared image. In particular, the terminal interworking and control system of the present invention, after interlocking a small size terminal directly controlled by the user, for example, a touch method and a large size terminal, while the user looks at the large size terminal of the small size Various methods for controlling the terminal are proposed. However, for convenience of description, a small sized terminal directly controlled by a user is called a user terminal, and a large sized terminal that receives location information of the touch means from the user terminal is called a receiving terminal. Of course, the user terminal preferably has a smaller size than the receiving terminal, but the size of the terminals is not limited to the above size.

The terminal used in the system of the present invention has a function of displaying an image and a wired / wireless communication function, but the communication function is not an essential function. However, considering the real life application, it is preferable that each terminal has an image display function and a communication function. The terminal is not limited to a specific device as long as it can display an image and transmit or receive a signal with another terminal, for example, a smartphone, a smart TV, a PC, a tablet PC, a laptop, a touch pad, a game console, a cloud PC, or the like. Various devices can be used as the terminal of the present invention. However, it is preferable that a terminal having a small size has a touch function.

Hereinafter, a terminal interworking and control system according to various embodiments of the present invention will be described with reference to the accompanying drawings. However, for convenience of description, an image shared by the terminals will be referred to as a first image, and an image representing position information of the touch means will be referred to as a second image (location display image).

1 is a view schematically showing a terminal interworking and control system according to an embodiment of the present invention.

Referring to FIG. 1, the terminal interworking and control system of the present embodiment includes a user terminal 100 and a receiving terminal 102.

The user terminal 100 may be an image display terminal directly controlled by the user, and may be, for example, a smartphone capable of executing a game such as Angry Birds. Preferably, the user terminal 100 is a terminal having a touch function and having a relatively small size.

The receiving terminal 102 is a terminal which is not directly manipulated by a user but interlocks with the user terminal 100 and which can display an image. The reception terminal 102 may be an element used for a different purpose than the user terminal 100, and may be, for example, a TV that broadcasts a broadcast program such as a drama. Preferably, the receiving terminal 102 is a terminal having a relatively large size, but does not necessarily have a touch function.

When the user terminal 100 distributes the image data corresponding to the first image displayed by the user terminal to the image signal and transmits the image data to the receiving terminal 102 through the corresponding channel, the receiving terminal 102 transmits the image data through the image signal. Acquire and display a first image corresponding to the image data. As a result, the user terminal 100 and the receiving terminal 102 display the same first image, that is, share the first image. However, the first image displayed on the receiving terminal 102 is substantially the same as the first image displayed on the user terminal 100, but due to the characteristics of the terminal of the receiving terminal 102, the color is different or the display ratio is different. There is a number. Although the receiving terminal 102 may display the transmitted image data as a corresponding image without special processing, the receiving terminal 102 may display the first image by processing the transmitted image data according to its characteristics. That is, as long as the receiving terminal 102 can display an image substantially the same as the first image of the user terminal 100, the method of processing the transmitted image data may be variously modified.

When the same image as the first image of the user terminal 100 is displayed on the receiving terminal 102, the receiving terminal 102 may stop the existing executing program and display the first image or execute the existing executing program. It can also be run through a separate channel. In addition, the receiving terminal 102 may display the existing execution program together on the screen on which the first image is displayed. For example, the receiving terminal 102 may stop the drama, which is an existing execution program, and display the first image of the Angry Bird game that is executed in the user terminal 100, for example, by switching the drama to a separate channel. The first image may be broadcasted and the first image may be displayed on a corresponding channel, or the drama may be continuously broadcasted on the current channel, but the first image may be displayed on another channel. In addition, the receiving terminal 102 may display the first image on the current channel, but display the drama that was being aired in a separate pop-up window. However, since the present invention is intended for a user to view the first image of the user terminal 100 through the receiving terminal 102, the receiving terminal 102 stops a program executed in the current channel or displays the display through another channel. It is preferable to switch the screen of the current channel to the screen on which the first image is displayed.

According to another embodiment of the present invention, the receiving terminal 102 may share the first image by transmitting an image signal to the user terminal 100. Even in this case, the control signal reflecting the user's operation (touch operation) is transmitted from the user terminal 100 to the receiving terminal 102.

Image sharing between the user terminal 100 and the receiving terminal 102 is not performed only once, but the same image is displayed to the receiving terminal 102 whenever the first image of the user terminal 100 is switched. Therefore, when the user operates the user terminal 100, the first image corresponding to the operation is displayed on the receiving terminal 102. In particular, since the receiving terminal 102 has a relatively larger size than the user terminal 100, the user may control the user terminal 100, but the gaze may look at the receiving terminal 102. That is, the user may manipulate the user terminal 100 while looking at the receiving terminal 102.

However, if the user terminal 100 is not viewed, it is difficult to grasp its own operation, and if the receiver terminal 102 is not present, the screen is small, so that the present invention can easily control the user terminal 100 while watching the receiver terminal 102. Various methods and systems are proposed.

According to an embodiment of the present invention, when a touch means such as a user's finger is close to a touch sensing area of the user terminal 100, for example, a screen, the touch means may be displayed on the first image displayed on the receiving terminal 102. A corresponding second image, for example a shadow image, may be displayed at that location, ie the position of the touch means is specified above the first image. In the case of a general PC, not an interlocking system, if the mouse is moved, the position of the mouse is not specified to the exact position, and in the case of a smartphone, the image indicating the position is not displayed even if it is touched.

On the other hand, the present invention specifies the position of the touch means on the user terminal 100 to the receiving terminal 102 as a shadow image, while interlocking the user terminal 100 and the receiving terminal 102. Therefore, the user can check the position and the operation touched by the user while not looking at the user terminal 100 but can control the receiver terminal 102 without looking at the receiver terminal 102. Subsequently, when the user touches and moves the screen of the user terminal 100, the position of the second image may change according to the movement. In this case, the second image may be maintained as a shadow image, or the finger image may move on the first image according to the movement after the shadow image as the second image is changed to a finger image. Therefore, the user can check the operation status of the touch means in the user terminal 100 through the receiving terminal 102, and as a result, the user does not look at the user terminal 100, but only while looking at the receiving terminal 102. 100 can be easily controlled.

For example, when a user is playing an Angry Birds game with a smartphone, the user may enjoy the Angry Birds game by operating the smartphone while watching only a TV screen having a large size. If the first image displayed on the user terminal 100 is changed to another first image according to a user's operation, the same first image is displayed on the receiving terminal 102, in which case the second image is displayed on the new first image. It is displayed at that location. The second image may be displayed on the screen of the receiving terminal 102 based on the coordinates of the user terminal 100. Of course, the first image may be shared according to image data transmission of the user terminal 100 or may be shared according to image data transmission of the receiving terminal 102.

According to another exemplary embodiment of the present invention, when the user approaches the touch sensing unit from the user terminal 100 having a touch sensing unit, for example, a touch pad, separately from the display unit, such as a laptop, the sensing unit 102 detects this. The second image may be displayed on the first image. That is, as long as the user terminal 100 can detect the touch means, the area for detecting the touch means is not limited to a specific portion of the user terminal 100.

In summary, the user terminal 100 and the receiving terminal 102 share the first image, and the user can easily manipulate the user terminal 100 having the small size while viewing the receiving terminal 102 having the large size. have. Accordingly, the user may move the touch means such as a finger or a touch pen on the image of the user terminal 100 while looking at the receiving terminal 102 or click a menu icon displayed on the first image of the user terminal 100 to specify You can also run the function. Of course, according to the above execution, an image substantially identical to the new first image displayed on the user terminal 100 is displayed on the receiving terminal 102.

Although not described above, once the second image is specified on the receiving terminal 102, the second image may reflect the movement of the touch means, but the second image may be moved by manipulating a physical keyboard, a keyboard displayed on the screen, and the like. It may be.

According to another embodiment of the present invention, the companion system may include a plurality of user terminals 100 and one receiving terminal 102. For example, when a plurality of users are playing the same game, the user terminals 100 are wired or wirelessly connected to the receiving terminal 102. When the user terminals 100 respectively transmit the position information of the corresponding touch means to the receiving terminal 102, the receiving terminal 102 displays second images indicating the positions of the touch means on the same first image. Can be. In this case, the first image shared by the user terminals 100 and the receiving terminal 102 may be transmitted by the specific user terminal 100 to the receiving terminal 102.

According to another embodiment of the present invention, when a plurality of touch means touches one user terminal 100, for example, a user touches the user terminal 100 with one left and right fingers or a plurality of users When the user terminal 100 is touched, the user terminal 100 transmits a control signal including the location information of the touch means to the receiving terminal 102, and the receiving terminal 102 according to the control signal has a first image. Second images corresponding to the location information may be displayed. Of course, shapes, colors or sizes may be different so that the second images are distinguished from each other. Meanwhile, the user terminal 100 may transmit a plurality of control signals having location information of each touch means to the receiving terminal 102.

According to another embodiment of the present invention, the user terminal 100 may sense the touch means by another method by realizing a plurality of sensing methods. For example, the user terminal 100 may sense a finger through a capacitive method and a touch pen using a receiver. Therefore, the user may implement multi-touch by simultaneously using a finger and a touch pen. As a result, the user can realize a separate operation by using a plurality of touch means.

According to another embodiment of the present invention, the companion system may include a user terminal, a receiver terminal, and a companion server. The companion server transmits the first image to both the user terminal and the receiving terminal to display the same first image, and the user terminal transmits the position information of the touch means to the receiving terminal to display the second image on the first image. May be displayed.

Hereinafter, various embodiments of the terminal interworking and control method of the present invention will be described with reference to the accompanying drawings.

2 is a flowchart illustrating a terminal interworking and control method according to a first embodiment of the present invention, FIG. 3 is a diagram illustrating a control method used in the terminal interworking and control method of FIG. 2, and FIG. FIG. 1 is a plan view illustrating an image sharing screen according to an exemplary embodiment. This embodiment proposes a linkage and control method when the user terminal 100 transmits image data (image signal) to the reception terminal 102.

Referring to FIG. 2, the user terminal 100 requests the linkage of an image to the receiving terminal 102 (S200). Specifically, when the user connects the user terminal 100 to the receiving terminal 102 by wire or wirelessly and requests interlocking, or when the user terminal 100 is connected to the receiving terminal 102, the user terminal 100 or the receiving terminal 102. After the terminal 102 identifies the counterpart terminal, the terminal 102 may inquire whether to interwork with the counterpart terminal. Interworking of the terminals 100 and 102 may be performed at the request of a user, or may be automatically performed when the terminals 100 and 102 mutually identify each other. According to an embodiment of the present invention, as shown in FIG. 3A, the user terminal 100 and the receiving terminal 102 may be connected to two channels, that is, a data channel and a control channel, and may request an interworking request or Control signals for permission can be transmitted and received through the control channel.

Subsequently, the user terminal 100 distributes image data corresponding to the first image displayed on the screen to an image signal and transmits the image data to the receiving terminal 102 through the data channel, and the receiving terminal 102 transmits the transmitted image. The first image corresponding to the data is displayed (S202). That is, the user terminal 100 and the receiving terminal 102 share the first image. Since it is useful to display the first image of the user terminal 100 and the first image of the receiving terminal 102 at substantially the same time, the image portion of the user terminal 100 transmits the image data to the display unit when the image data is transmitted. It is desirable to transmit simultaneously to the receiving terminal 102 through the channel.

Subsequently, when a user approaches a touch means such as a finger or a touch pen to the user terminal 100, the user terminal 100 detects the touch means (S204). Specifically, the user terminal 100 detects the touch means when the touch means enters a detectable area. For example, the user terminal 100 may use a capacitive touch panel, and when the user approaches the finger to the user terminal 100, the user terminal 100 may detect the capacitance change caused by the finger to detect the position of the finger. have.

However, for this sensing, the user terminal 100 may set the sensing level to a plurality of levels, for example, two levels, as shown in FIG. 3B. When not performing the terminal interworking process, that is, when the user uses only the user terminal 100, the detection level of the user terminal 100 is set to a high level (L2) so that the touch position is detected only when the user touches. . On the other hand, when the user performs the terminal interworking process, that is, also using the receiving terminal 102, the user terminal 100 changes the detection level to a low level (L1). Therefore, the user terminal 100 may detect the touch means even when the touch means is not in touch but in close proximity. That is, the user terminal 100 may set different sensing levels depending on whether or not the terminal interworking process is used. According to another embodiment of the present invention, the user terminal 100 may use a variety of methods, such as electromagnetic induction, resistive film, optical, ultrasonic method, in addition to the capacitive method, the setting conditions may vary depending on the usage method .

Subsequently, the user terminal 100 transmits a control signal including position information of the touch means according to the detection result to the receiving terminal 102 through a control channel, and the receiving terminal 102 transmits the control signal according to the control signal. A second image according to the location information, for example, a shadow image 400 as shown in FIG. 4 is displayed on the first image (S206). In the above, signals are transmitted differently by being divided into a data channel and a control channel, but both image data and a control signal may be transmitted through the data channel. For example, the user terminal 100 sets the control section 112 between the data sections 110 and 114 as shown in FIG. 3C to control the signal C1 during the control section 112. Can transmit

Subsequently, when the user touches and moves the user terminal 100 by using a touch means, the user terminal 100 generates a control signal including the position information according to the movement and transmits it to the receiving terminal 102, and the receiving terminal 102. 102 displays the second image on the first image according to the transmitted control signal (S208). As a result, the second image is displayed on the first image of the receiving terminal 102 to reflect the movement of the touch means. Of course, when the user clicks a menu for performing a specific operation on the first image on the user terminal 100, only image switching is performed, and thus, the receiving terminal 102 maintains the second image at the current position. Only an image may be changed into a new image of the user terminal 100.

In this case, the position information of the touch means may be information in the form of an image or information in the form of a coordinate. That is, the user terminal 100 directly generates the position information on the second image in the form of image data and transmits it to the receiving terminal 102 or transmits only the position information of the touch means to the receiving terminal 102 in the form of a control signal. Can be. In this case, the position of the touch means may not be displayed on the user terminal 100, but the position of the touch means may be displayed only on the receiving terminal 102.

In summary, the terminal interworking and control method of the present invention detects the proximity of the touch means in the state where the first image is shared, displays a second image on the first image of the receiving terminal 102, and the user selects the user terminal ( When touching or moving after the touch, the second image reflecting the movement is displayed on the first image of the receiving terminal 102.

The first image and the second image displayed on the receiving terminal 102 may overlap as separate images, or a composite image generated by combining the first image and the second image may be displayed.

Although the second image is referred to as being displayed on the first image, the display method may be variously modified as long as the second image is displayed together with the first image. For example, a portion of the first image corresponding to the position of the touch unit may be darkly expressed, and the darkly expressed portion may be the second image.

Hereinafter, various methods of sensing the touch means will be described with reference to the accompanying drawings.

As a basic method of detecting the touch means, the display unit of the user terminal 100 is configured by a capacitive touch panel, and a touch means such as a finger, or the like, is used. In this case, the sensing unit of the user terminal 100 detects the position of the touch means by changing the capacitance when approaching and touching the touch means.

There are other methods of sensing the touch means, which will be described with reference to FIGS. 5 to 7 below.

5 is a diagram illustrating a touch means sensing method according to an embodiment of the present invention.

Referring to FIG. 5, the touch pen 500 may be used as a touch means for touching the user terminal 100. The user terminal 100 uses a capacitive touch panel.

The touch pen 500 includes a body 510 and a touch unit 512. Body 510 is made of a material that does not conduct electricity, the touch unit 512 is a conductor. Therefore, due to the touch unit 512, a change in capacitance occurs when the touch pen 500 approaches the user terminal 100 or when the user touches the user terminal 100. As a result, the user terminal 100 touches. The pen 500 may be sensed. That is, unlike the general touch pen that could not be used in the capacitive method, the touch pen 500 of the present invention can be used in the capacitive method. Of course, the entire touch pen 500 may be made of a conductor. However, in consideration of cost, it is preferable to manufacture only the touch unit 512 that is actually touched with a conductor.

6 is a view showing a touch means detection method according to another embodiment of the present invention.

Referring to FIG. 6A, a receiver 600 may be mounted on a part of the user terminal 100, and a touch pen 602 may be used.

The receiver 600 may include one infrared sensor and two ultrasonic sensors. The receiver 600 may detect the movement of the touch pen 602 by receiving infrared rays and ultrasonic waves transmitted from the touch unit (tip) of the touch pen 602. The control signal having the position information of the touch pen 602 according to the detection result is transmitted to the receiving terminal 102. The receiving terminal 102 analyzes the control signal to detect the position of the touch pen 602 and displays a second image at the detected position. As a result, the second image is displayed together with the first image. In this case, the control signal may be transmitted by the receiver 600 to the receiving terminal 102 or the user terminal 100 may be transmitted to the receiving terminal 102.

Looking at how the receiver 600 detects the position of the touch pen 602, three sensors respectively measure the transmission time of the infrared ray and the transmission time of the ultrasonic wave, converts the transmission times into distances, and converts the converted distances. The position of the touch pen 602 is detected through a method similar to triangulation. Therefore, even if the touch pen 602 is close without directly touching the user terminal 100, the position of the touch pen 602 may be sensed, and the touch and movement of the touch pen 602 may be accurately sensed. Of course, signals other than infrared and ultrasonic may also be used.

The second image reflecting the position of the touch pen 602 may not be displayed on the user terminal 100 but only on the receiving terminal 102. Of course, the user terminal 100 operates in response to the movement of the touch pen 602.

According to another embodiment of the present invention, the receiver 600 not only detects the position of the touch pen 602 but also transmits an image signal and a control signal including image data to the receiving terminal 102. Can be. In detail, the receiver 600 may include a touch controller 610, an image signal unit 612, a control signal unit 614, and a transceiver unit 616. The touch controller 610 detects the position of the touch pen 602 by using the received infrared rays and ultrasonic waves, and provides information about the detected position to the user terminal 100. The user terminal 100 displays the location of the touch pen 602 or performs a related operation according to the provided information. The image signal unit 612 receives image data from the user terminal 100, and transmits the provided image data to the receiving terminal 102 through the transceiver 616. The control signal unit 614 transmits a control signal having the detected position information of the touch pen 602 to the receiving terminal 102. That is, since the receiver 600 transmits a control signal having the image data transmitted from the user terminal 100 and the position information of the touch pen 602 to the receiving terminal 102, the user terminal 100 does not have a communication function. Not enough. Accordingly, a terminal having no communication function or a terminal having a communication function but unable to use a related communication facility may not only recognize the position and operation of the touch pen 602 using the receiver 600 but also share an image and a second device. The interlocking method of the present invention, such as display of an image, can be used.

According to another embodiment of the present invention, the receiver may be included in the user terminal 100 to be integrally implemented.

7 is a diagram illustrating a touch means sensing method according to another embodiment of the present invention.

Referring to FIG. 7, the user terminal 100 includes a touch panel 700 and an electromagnetic generator 702.

The electromagnetic generator 702 may be connected to the rear surface of the touch panel 700, and may be formed of a thin metal film to generate an electromagnetic field when electricity flows.

The touch pen 707 includes a body 710 and a touch unit 712, and the touch unit 712 may be preferably made of a small metal coil. As a result, when the touch pen 704 is close to the touch panel 700, an electromagnetic induction phenomenon occurs in the touch unit 712, and as a result, a deformation of the electromagnetic field generated from the electromagnetic generator 702 occurs. Therefore, the user terminal 100 detects the deformation of the electromagnetic field and recognizes the position of the touch pen 704. In particular, since the deformation of the electromagnetic field varies depending on the approach position and the touch intensity of the touch pen 704, the touch means sensing method detects the touch pen 700 and the touch pressure of the touch pen 700 in detail. can do. Therefore, the size of the second image displayed on the receiving terminal 102, the thickness of the line when represented by a line, and the like may be displayed differently according to the position or pressure of the touch pen 704. Of course, the shape, size or color of the second image may vary depending on the touch intensity.

5 to 7, the user terminal 100 may detect the touch pen through various methods and reflect the detection result to the receiving terminal 102.

As described above, the user terminal 100 detects the touch means close to the detectable area and transmits the location information of the touch means to the receiving terminal 102, where the detectable area is a predetermined distance from the user terminal 100. It may be spaced apart. That is, the user terminal 100 may detect a touch means close to the user terminal 100 located within a predetermined distance from the user terminal 100. The preset distance may vary depending on the sensing scheme described above. For example, when the capacitive touch panel is used in the user terminal 100, the preset distance may be a distance corresponding to the low level L1 described with reference to FIG. 3. In addition, the preset distance may be set based on the shortest distance between the touch means and the user terminal 100.

8 is a flowchart illustrating a terminal interworking and control method according to a second embodiment of the present invention.

Referring to FIG. 8, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S800).

Subsequently, the user terminal 100 transmits image data corresponding to the currently displayed first image to the receiving terminal 102, and the receiving terminal 102 displays the first image corresponding to the image data (S802). . That is, the user terminal 100 and the receiving terminal 102 share the first image.

Subsequently, the user terminal 100 detects a touch means such as a finger or a touch pen through various methods (S804).

Subsequently, the user terminal 100 generates a composite image displaying a second image corresponding to the position of the sensed touch means together with the currently displayed first image, and generates composite image data (synthesis information) corresponding to the composite image. ) Is transmitted to the receiving terminal 102, and the receiving terminal 102 displays the composite image corresponding to the composite image data (S806). As a result, the fisheries terminal 102 displays a screen on which the second image is displayed together with the first image. In this case, only the first image may be displayed on the user terminal 100, or a screen on which the second image is displayed together with the first image may be displayed.

Subsequently, when the user touches or moves in the touched state, the user terminal 100 transmits a control signal having the position information of the touch means to the receiving terminal 102, and the receiving terminal 102 transmits the touch signal. According to a control signal, a second image indicating the position of the touch means is displayed together with the corresponding first image (S808). That is, the movement of the touch means is reflected on the screen of the receiving terminal 102. However, the first image on the receiving terminal 102 may be the same image as the previous image or may be different from the previous image.

In summary, unlike the first embodiment in which the receiving terminal 102 generates the second image and displays the second image together, in the present embodiment, the user terminal 100 synthesizes the first image and the second image. A composite image is generated and the generated composite image is transmitted to the receiving terminal 102. As a result, the receiving terminal 102 displays a screen on which the second image is displayed together with the first image.

9 is a flowchart illustrating a terminal interworking and control method according to a third embodiment of the present invention.

9, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S900). The interworking may be automatically started when the user terminal 100 and the receiving terminal 102 are connected, or may be started depending on the user's selection, asking whether the user is selected.

Subsequently, the user or the user terminal 100 automatically sets a plurality of detection levels for detecting a touch means such as a finger or a touch pen (S902). Specifically, a first level for detecting the touch means when the touch means is close to the user terminal 100 and a second level for detecting the touch means when the touch means touches are set. Of course, the levels for sensing the touch means may be set more precisely.

Subsequently, the user terminal 100 detects the touch means and displays a second image on the receiving terminal 102 indicating the position of the touch means (S904).

Subsequently, it is determined whether or not there is a request for interlocking stop by the user (S906). The interworking stop may be requested by the user by selecting a menu or by turning off the connection between the user terminal 100 and the receiving terminal 102. When the user controls the user terminal 100 while viewing the receiving terminal 102 and wants to use only the user terminal 100 or only wants to view the receiving terminal 102, the user terminal 100 and the receiving terminal 102 You can request to stop interlocking while connected.

If there is no user stop request for interworking, step S904 is performed again.

On the other hand, when the user stops the interlocking request of the user, the user terminal 100 initializes the plurality of levels to change to the sensing levels to detect only one level is present or touched (S908). That is, when the touch means is not touched, the user terminal 100 does not sense the touch means but changes the level so as to detect the touch means only when the touch means is touched.

In summary, in the terminal interworking and control method of the present invention, a user may freely set and change an interworking request, an interlocking stop request, and a sensing level.

According to another embodiment of the present invention, the first detection level may be set when the user terminal 100 and the reception terminal 102 start interlocking, and the detection level may be differently set during the interlocking. For example, the user may set different sensing levels according to the approach distance and the touch intensity of the touch means during interlocking, or change the sensing level setting to sense only when the touch means is touched.

That is, in relation to the detection level setting, the user terminal according to the present invention may set the detection level differently according to the user's request even when the user terminal 100 and the receiving terminal 102 are not linked.

In this regard, more specifically, the user terminal according to the present invention includes a display unit; A mode changer for changing a touch mode according to a user request of a user terminal; According to the touch mode, it may include a location information generation unit for generating location information of the touch means in close proximity to the display unit. The touch mode may include a direct touch mode for recognizing a touch of a touch means touching the display unit, an indirect touch mode for recognizing touch of a touch means close to the display unit and a touch means touching the display unit. In the direct touch mode, the sensing level may be set to a high level L2, and in the indirect touch mode, the sensing level may be set to a level L1 lower than L2.

The mode changer may change the touch mode according to a user's touch, voice data, or image data. That is, the user may change the touch mode through a touch on the menu screen of the user terminal, or may change the touch mode through an image such as a voice or a motion of the user.

In addition, the user terminal according to the present invention may further include a switch for receiving a user request. The switch can be configured like the iPhone's bell / vibration switch.

In the indirect touch mode, the position information generation unit generates position information of the touch unit located within a predetermined distance from the display unit. In detail, the location information generator may generate location information of the touch means when the touch means does not directly touch the display in the indirect touch mode.

For the above-described interworking system, the user terminal according to the present invention may further include a signal unit for transmitting the location information to the receiving terminal linked with the user terminal, wherein the image corresponding to the location information is not displayed on the user terminal. The receiving terminal may display an image corresponding to the location information.

10 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.

Referring to FIG. 10, the user terminal 100 and the receiving terminal 102 interlock with each other, that is, display the second image 1000 indicating the position of the touch means while sharing the first image. However, when the first image is shared, the receiving terminal 102 transmits image data to the user terminal 100 to share the first image.

When the receiving terminal 102 is an IPTV or a smart TV, as illustrated in FIG. 10, a user interface (1010) for selecting a program such as a movie or a drama may be displayed, and the UI 1010 may be displayed. One image may be shared with the user terminal 100. Of course, the second terminal 1000 is displayed on the receiving terminal 102 together with the first image.

Looking at the advantages of sharing the first image displayed on the receiving terminal 102 to the user terminal 100, the user is watching the receiving terminal 102, for example TV, the user terminal 100, for example smart The phone may control an operation on the receiving terminal 102 on which the second image is displayed. For example, the user may use the user terminal 100 to select a specific program, send an e-mail, perform an e-commerce transaction, or play a game through the reception terminal 102.

In addition, since the same first image as that of the receiving terminal 102 is displayed on the user terminal 100, the user may use the user terminal 100 to receive the screen of the receiving terminal 102. You can enjoy various services provided. For example, the user may control the receiving terminal 102 in the living room from the room to perform electronic commerce or the like.

More specifically, the user terminal according to the present invention receives the image data from the receiving terminal, and displays the image data. Here, the image data may be a UI of the receiving terminal. The user terminal transmits the first position information of the touch means close to the display unit of the user terminal and the second position information of the touch means touching the display unit to the TV. That is, the TV UI is displayed on the user terminal, and the user moves his hand while bringing his hand (touch means) close to the user terminal display unit for TV operation. The first location information is transmitted to the TV as the hand moves, and the position display image moves on the TV as the hand moves. When the user touches an icon, the second location information is transmitted to the TV, and the TV may execute an icon corresponding to the second location information.

Alternatively, the user terminal according to the present invention may transmit the position information of the touch means close to the display unit and the touch information of the touch means for TV control to the TV. That is, when the location display image is moved on the TV according to the movement of the user's hand, and the user touches the desired icon, the touch information about the icon touch is transmitted to the TV so that the TV can execute the icon touched by the user. The touch information may be a control signal for directly executing an icon in comparison with the second location information.

Meanwhile, even in this case, the user terminal preferably receives image data such as a UI from the TV. The TV may transmit tag information, location information, etc. for each of the selected items of the UI to the user terminal so that the user terminal may generate a control signal such as touch information.

Here, the operation of the user terminal may be performed according to the execution of the remote control application stored in the user terminal. When the remote control application is executed, the user terminal may request transmission of image data to the TV, and the user terminal may receive image data from the TV.

The above-described TV control method is applicable not only in the indirect touch mode, but also in a touch method using a pen such as a Galaxy Note and an Optimus view. The difference in the touch pressure or the touch area of the pen may be set to correspond to the indirect touch mode and the direct touch mode, respectively. For example, it may correspond to an indirect touch mode when the pen is weakly touched or the touch area is small, and a direct touch mode when the pen is strongly touched or the touch area is wide.

Hereinafter, various embodiments will be described.

11 is a flowchart illustrating a terminal interworking and control method according to a fourth embodiment of the present invention.

Referring to FIG. 11, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S1100).

Subsequently, the receiving terminal 102 transmits the image data corresponding to the first image currently displayed or the first image to be displayed to the user terminal 100, and the user terminal 100 transmits the image data corresponding to the transmitted image data. 1 Display an image on the screen (S1102). That is, the user terminal 100 and the receiving terminal 102 share the first image.

Subsequently, when a user approaches a touch means such as a finger or a touch pen, the user terminal 100 detects the touch means (S1104). The user terminal 100 may detect the touch means by using various methods such as a capacitance sensing method and an electromagnetic sensing method.

Subsequently, the user terminal 100 transmits a control signal having the position information of the touch means to the receiving terminal 102 according to the detection result, and the receiving terminal 102 is connected with the first image according to the transmitted control signal. Together, a second image indicating the position of the touch means is displayed (S1106). According to another embodiment of the present invention, the user terminal 100 may transmit the composite image data corresponding to the composite image on which the second image is displayed together with the first image to the receiving terminal 102. However, since the reception terminal 102 basically transmits the image data corresponding to the first image to the user terminal 100, the user terminal 100 transmits the control signal to the reception terminal 102. The manner is preferred.

Subsequently, when the user touches or moves the touch means, the user terminal 100 transmits a control signal having the position information of the touch means to the receiving terminal 102 to display the second image on the receiving terminal 102 ( S1108).

12 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.

Referring to FIG. 12A, in the terminal interworking system of the present invention, the receiver 1200 may be installed or embedded in the receiving terminal 102 instead of the user terminal 100.

When the touch pen 1202 transmits ultrasonic waves and infrared rays to the receiver 1200 installed in the receiving terminal 102 as a touch means used in the user terminal 100, the receiver 1200 receives the ultrasonic waves and infrared rays and touch pens. Detects the position of 1202.

The receiver 1200 transmits information on the detected position of the touch pen 1202 to the receiving terminal 102, and the receiving terminal 102 transmits a second image indicating the position of the touch pen 1202 to the first terminal. Display with the image. Of course, the user terminal 100 and the receiving terminal 102 display the second image in a state in which the first image is shared.

According to another embodiment of the present invention, the receiver 1200 may perform not only the position detection of the touch pen 1202 but also a communication role. In detail, the receiver 1200 may include a touch means detector 1210, an image signal unit 1212, a control signal unit 1214, and a transceiver 1216. The touch means detector 1210 detects the position of the touch pen 1202. The image signal unit 1212 receives an image signal that has delivered image data transmitted from the user terminal 100 through the transceiver 1216, and receives image data of the received image signal through the transceiver 1216. Send to 102 to display the first image. Of course, the image signal unit 1212 may receive the image data from the receiving terminal 102 and transmit the image signal that has distributed the received image data to the user terminal 100. The control signal unit 1214 may receive a control signal related to an interlocking operation and the like transmitted from the user terminal 100 and transmit the received control signal to the receiving terminal 102. Of course, the control signal having the location information of the touch pen 1202 is not transmitted from the user terminal 100 to the receiver 1200. In conclusion, the receiver 1200 not only detects the position of the touch pen 1202 but also performs a communication function. Therefore, the receiving terminal 102 can use the interworking and control method of the present invention without having a communication function.

13 is a flowchart illustrating a terminal interworking and control method according to a fifth embodiment of the present invention.

Referring to FIG. 13, the receiver 1200 is installed in the receiving terminal 102 (S1300). According to another embodiment of the invention, the receiver may be embedded in the receiving terminal 102.

Subsequently, the user terminal 100 and the receiving terminal 102 are connected to start interlocking (S1302).

Subsequently, the user terminal 100 transmits image data corresponding to the first image to the receiving terminal 102 to share the first image, or the receiving terminal 102 transmits the image data to the user terminal 100. The first image is shared (S1304).

Subsequently, when the touch means, for example, the touch pen 1202 is close to the user terminal 100, the receiver 1200 detects the position of the touch pen 1202 through infrared and ultrasonic waves transmitted from the touch pen 1202. The transmission is transmitted to the reception terminal 102, and the reception terminal 102 displays a second image such as a shadow according to the detected position together with the first image (S1306).

Subsequently, when the user touches or moves after touching the user terminal 100 with the touch pen 1202, the receiver 1200 detects the movement of the touch pen 1202 and transmits the movement to the receiving terminal 102. 102 displays the corresponding second image together with the first image according to the movement (S1308).

14 is a diagram illustrating a terminal interworking system according to another embodiment of the present invention.

Referring to FIG. 14A, the terminal interworking system according to the present embodiment includes a user terminal 100, a receiving terminal 102, and a dongle 1400.

The dongle 1400 serves not only to assist communication of the user terminal 100 but also to sense the position of the touch pen 1402 as a touch means. Specifically, the dongle 1400 may include a communication unit 1410 and a touch means detector 1412.

The communication unit 1410 is a communication means for the user terminal 100, and may provide a specific communication function, for example, a Wibro communication function, and converts the specific communication function into another communication function, for example, Wi- It may be converted to Fi and used for the user terminal 100.

The touch means detecting unit 1412 may detect the position of the touch pen 1402 similarly to the receiver, and may transmit information about the detected position to a control signal and transmit the information to the receiving terminal 102.

In summary, the user terminal 100 has a communication function through the dongle 1400, and the dongle 1400 is also used as a means for detecting the position of the touch pen 1402.

Hereinafter, a receiver and a dongle in the case of being separated from the user terminal 100 will be collectively referred to as an external device.

15 is a block diagram showing the structure of a user terminal according to an embodiment of the present invention.

Referring to FIG. 15, the user terminal 100 of the present embodiment includes a control unit 1500, an interlocking unit 1502, an image unit 1504, a sensing unit 1506, a setting unit 1508, a display unit 1510, A signal unit 1512, a transceiver 1514, and a storage unit 1516 are included.

The interlocking unit 1502 manages all functions related to interworking with the receiving terminal 102.

The image unit 1504 may generate a first image to be shared with the receiving terminal 102 or generate a composite image of the second image and the first image. The image unit 1504 displays the first image through the display unit 1510.

The sensor 1506 detects a touch means such as a finger or a touch pen. Specifically, the sensing unit 1506 detects the position of the touch means by distinguishing when the touch means is close to when it is touched. The sensing method is not limited, such as capacitive and electromagnetic induction. Information about the position of the touch means sensed by the detector 1506 may be generated by a location information generator (not shown).

The setting unit 1508 manages setting of various functions such as setting of an interlocking function and setting of a detection level.

The display unit 1510 is a touch panel and may be implemented in various ways, such as a capacitive type, a resistive film type, and an electromagnetic induction type.

The signal unit 1512 generates an image signal that distributes image data corresponding to the first image and a control signal having information about the position of the touch means. In addition, the signal unit 1512 may transmit the image signal to the receiving terminal 102 through the data channel and the control signal to the receiving terminal 102 through the control channel.

The transceiver unit 1514 serves as a communication path with the receiving terminal 102.

The storage unit 1516 stores various data such as a first image, an image signal, a control signal, an application program, and the like.

The controller 1500 generally controls the operations of the components of the user terminal 100.

FIG. 16 is a block diagram illustrating a structure of the image unit of FIG. 15 according to an embodiment of the present invention. FIG.

Referring to FIG. 16, the image unit 1504 of the present embodiment includes a first image generator 1600, a second image generator (position information generator 1602), and an image synthesizer 1604.

The first image generator 1600 generates a shared first image, and the second image generator 1602 generates a second image such as a shadow image corresponding to the position of the touch means.

The image synthesizer 1604 generates a composite image on which the second image is displayed along with the generated first image.

According to another embodiment of the present invention, the user terminal 100 transmits an image signal that has delivered image data corresponding to the first image to the receiving terminal 102, and is not a control signal having information on the position of the touch means. The control signal in which the position indication data corresponding to the second image is distributed may be transmitted to the receiving terminal 102.

17 is a block diagram showing the structure of a user terminal according to a second embodiment of the present invention. 17 illustrates a structure of a user terminal 100 in a terminal interworking system in which a receiver is installed in the user terminal 100.

Referring to FIG. 17, the user terminal 100 according to the present embodiment includes a control unit 1700, an interlocking unit 1702, an image unit 1704, a touch unit 1706, a setting unit 1708, and a display unit 1710. It may include a signal unit 1712, a transceiver 1714, and a storage 1716.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image unit 1704 generates a first image to be shared with the receiving terminal 102 or generates a composite image of the first image and the second image.

The touch manager 1706 receives and manages information on the position of the touch means from the receiver. That is, the receiver detects the position of the touch means, and the touch manager 1706 detects the position of the touch means by analyzing a signal transmitted from the receiver.

The signal unit 1712 generates an image signal that delivers image data corresponding to the first image, a control signal having information about a position of the touch means, or a synthesized signal that distributes data corresponding to a composite image. In addition, the signal unit 1712 may transmit the image signal or the composite signal to the receiving terminal 102 through a data channel and transmit the control signal to the receiving terminal 102 through a control channel.

18 is a block diagram showing the structure of a user terminal according to a third embodiment of the present invention. 18 illustrates a structure of a user terminal 100 in a terminal interworking system in which a receiver is embedded in the user terminal 100.

Referring to FIG. 18, the user terminal 100 of the present embodiment includes a control unit 1800, an interlocking unit 1802, a transceiver 1804, an image unit 1806, a receiver unit 1808, a setting unit 1810, and the like. The display unit 1812, the signal unit 1814, and the storage unit 1816 may be included.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image unit 1806 generates a first image to be shared with the receiving terminal 102 or generates a composite image of the first image and the second image.

The receiver 1808 receives infrared rays and ultrasonic waves transmitted from the touch means when the touch means approaches or touches the user terminal 100, and analyzes the received infrared rays and ultrasonic waves to detect the position of the touch means.

The signal unit 1814 generates an image signal for distributing image data corresponding to the first image, a control signal having information about the position of the touch means, or a synthesized signal for distributing composite image data corresponding to the composite image. In addition, the signal unit 1814 may transmit the image signal or the composite signal to the receiving terminal 102 through a data channel and transmit the control signal to the receiving terminal 102 through a control channel.

19 is a block diagram showing the structure of a user terminal according to a fourth embodiment of the present invention. 19 illustrates a structure of a user terminal 100 using an electromagnetic induction method.

Referring to FIG. 19, the user terminal 100 of the present embodiment may include a control unit 1900, an interlocking unit 1902, an image unit 1904, an electromagnetic generator 1906, a detection unit 1908, and a setting unit 1910. The display unit 1912 may include a signal unit 1914, a transceiver 1916, and a storage unit 1920.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image unit 1904 generates a first image to be shared with the receiving terminal 102 or generates a composite image of the first image and the second image.

The electromagnetic generator 1906 serves to generate electromagnetic waves in order to sense the touch means by an electromagnetic induction method, and may be disposed on the rear surface of the display unit 1912.

The sensing unit 1908 senses the position of the touch means by sensing the electromagnetic change caused by the coil of the touch means.

The signal unit 1914 generates an image signal for distributing image data corresponding to the first image, a control signal having information on the position of the touch means, or a synthesized signal for distributing composite image data. In addition, the signal unit 1914 may transmit the image signal or the image signal to the receiving terminal 102 through a data channel and the control signal to the receiving terminal 102 through a control channel.

20 is a block diagram showing the structure of a user terminal according to a fifth embodiment of the present invention. 20 illustrates the structure of the user terminal 100 when the receiving terminal 102 transmits image data to the user terminal 100.

Referring to FIG. 20, the user terminal 100 of the present embodiment includes a control unit 2000, an interlocking unit 2002, a transceiver unit 2004, an image signal unit 2006, an image unit 2008, and a detection unit 2010. , A signal unit 2012, a display unit 2014, a setting unit 2016, and a storage unit 2018.

Hereinafter, descriptions of the components that perform the same functions as those in the first embodiment will be omitted.

The image signal unit 2006 receives an image signal transmitted from the receiving terminal 102 through the transceiver 2004 and analyzes the received image signal.

The image unit 2008 generates a first image corresponding to the image signal analyzed by the image signal unit 2006 or generates a composite image of the first image and the second image.

The detector 2010 detects the position of the touch means.

The signal unit 2012 generates a composite signal in which the control signal having the information on the position of the touch means or the composite image data is distributed. In addition, the signal unit 1914 may transmit the synthesized signal to the receiving terminal 102 through the data channel and the control signal to the receiving terminal 102 through the control channel.

21 is a block diagram showing the structure of a receiving terminal according to the first embodiment of the present invention.

The receiving terminal 102 of the present embodiment includes a control unit 2100, an interlocking unit 2102, a transmitting and receiving unit 2104, an image signal unit 2106, a control signal unit 2108, an image unit 2110, and a display unit 2112. ) And a storage unit 2114.

The linkage unit 2102 manages a linkage function with the user terminal 100.

The transceiver 2104 serves as a communication path of the user terminal 100.

The image signal unit 2106 receives an image signal that has delivered image data transmitted from the user terminal 100, and analyzes the received image signal to detect image data.

The control signal unit 2108 receives a control signal transmitted from the user terminal 100 and analyzes the received control signal to detect information on the position of the touch means.

The image unit 2110 displays the first image corresponding to the detected image data through the display unit 2112 and, together with the first image, a shadow corresponding to the information on the position of the detected touch means. Display the second image. According to another embodiment of the present invention, the image unit 2110 generates a composite image in which the second image is displayed together with the first image through the image data and the information, and displays the generated composite image. Display via 2112.

The display unit 2114 may be implemented with various display elements such as LCD, OLED, PDP, etc. without limiting to display an image, and does not need to have a touch function.

The storage unit 2114 stores various data such as a first image, a second image, a composite image, an application program, and the like.

The controller 2100 generally controls the operations of the components of the receiving terminal 102.

22 is a block diagram showing the configuration of a receiving terminal according to a second embodiment of the present invention. FIG. 22 shows the structure of the receiving terminal 102 when the composite image is transmitted from the user terminal 100 to the receiving terminal 102.

Referring to FIG. 22, the reception terminal 102 of the present embodiment includes a control unit 2200, an interlocking unit 2202, a transceiver 2204, a signal unit 2206, an image unit 2208, a display unit 2210, and the like. It may include a storage unit 2212.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The signal unit 2206 receives the image signal and the synthesized signal from the user terminal 100, and analyzes the received image signal and the synthesized signal. The signal unit 2206 receives the image signal at the start of interlocking, and receives a composite signal when the user terminal 100 detects the touch means. The composite signal includes data corresponding to the composite image of the first image and the second image. According to another exemplary embodiment of the present invention, the signal unit 2206 may receive an image signal that has delivered image data and a control signal that has delivered data corresponding to the second image.

The image unit 2208 displays the synthesized image corresponding to the data included in the synthesized signal through the display unit 2210. As a result, the reception terminal 102 displays a screen on which the second image is displayed together with the first image. According to another exemplary embodiment of the present disclosure, the image unit 2208 may display the first image corresponding to the image data through the display unit 2210 and display the second image together with the first image.

23 is a block diagram showing the configuration of a receiving terminal according to a third embodiment of the present invention. 23 shows the structure of the receiving terminal 102 when the receiver is installed in the receiving terminal 102.

Referring to FIG. 23, the receiving terminal 102 of the present embodiment includes a control unit 2300, an interlocking unit 2302, a transceiver 2304, a signal unit 2306, a receiver unit 2308, an image unit 2310, The display unit 2312 and the storage unit 2314 may be included.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The signal unit 2306 receives an image signal from which the image data has been distributed from the user terminal 100 and analyzes the received image signal.

The receiver unit 2308 receives and analyzes information on the position of the touch means from the receiver installed in the receiving terminal 102.

The image unit 2310 displays a first image corresponding to the image data through the display unit 2312, and displays a second image corresponding to the position of the touch unit together with the first image. According to another embodiment of the present invention, the image unit 2310 generates a composite image of the first image and the second image, and displays the generated composite image through the display unit 2312.

24 is a block diagram showing a configuration of a receiving terminal according to a fourth embodiment of the present invention. 24 shows the structure of the receiving terminal 102 when the receiver is built in the receiving terminal 102.

Referring to FIG. 24, the reception terminal 102 of the present embodiment includes a control unit 2400, an interlocking unit 2402, a transceiver 2404, a signal unit 2406, a receiver unit 2408, an image unit 2410, It may include a display unit 2412 and a storage unit 2414.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The receiver 2408 detects the position of the touch means by detecting infrared rays and ultrasonic waves transmitted from the touch means approaching or touching the user terminal 100.

The image unit 2410 displays the first image corresponding to the image data through the display unit 2412, and the second image corresponding to the position of the touch means detected by the receiver unit 2408 together with the first image. Display the image. According to another embodiment of the present invention, the image unit 2410 generates a composite image of the first image and the second image, and displays the generated composite image through the display unit 2312.

25 is a block diagram showing a configuration of a receiving terminal according to a fifth embodiment of the present invention. 25 shows the structure of the receiving terminal 102 when the receiving terminal 102 transmits the image data to the user terminal 100.

Referring to FIG. 25, the reception terminal 102 of the present embodiment includes a control unit 2500, an interlocking unit 2502, a signal unit 2504, an image unit 2506, a display unit 2508, a transceiver unit 2510, and the like. The storage unit 2512 may be included.

Hereinafter, detailed description of the same components as in the first embodiment will be omitted.

The signal unit 2504 generates an image signal in which image data corresponding to the first image to be displayed or displayed on the receiving terminal 102 is distributed, and the generated image signal is transmitted to the user terminal 100 through the transceiver unit 2510. To send). In addition, the signal unit 2504 receives and analyzes the information on the position of the touch means transmitted from the user terminal 100 or the control signal having data corresponding to the second image or the synthesized signal that has distributed the data for the synthesized image. .

The image unit 2506 generates a second image corresponding to the position of the touch means, and displays the second image together with the first image through the display unit 2508. According to another embodiment of the present invention, the image unit 2506 may generate a composite image of the first image and the second image, and display the generated composite image through the display unit 2508.

Meanwhile, in the above-described embodiment, the user terminal transmits the position information or the position image of the touch means to the receiving terminal, but the user terminal may display the position image indicating the position of the touch means on its display unit. That is, the shadow image 400 as shown in FIG. 4 may also be displayed on the user terminal.

In more detail, when surfing the web on a smart phone equipped with a small and medium touch panel display, a user needs to click a desired item. By the way, since a finger cannot click precisely, if the item to be clicked (for example, an article) is described very densely, the wrong item may be selected.

This problem can be solved by the user terminal displaying the position image of the touch means on its display unit. The following description will be made in detail.

The user terminal according to the present invention recognizes a touch means close to the user terminal (for example, a smart phone) and displays a position image indicating the position of the touch means on the display. In more detail, the display unit may be a touch panel display unit, and the user terminal may recognize a touch unit proximate to the touch panel display unit of the user terminal in various ways described above. In this case, the position image may be, for example, an image whose shade is adjusted at the position of the cursor, the pointer, or the touch means. The position image may be generated by the image generator of the user terminal.

As described above, the proximity may be within a preset distance from the user terminal, and when a capacitive touch panel is used, it may be a case where a capacitance value change more than a preset threshold (level) occurs. Here, the threshold value may be lower than the threshold value when the touch means directly contacts the capacitive touch panel as described with reference to FIG. 3.

When the touch means (for example, a finger) is close to the user terminal and the touch means continues to move while the user terminal recognizes it, the position image is displayed continuously according to the touch means to be moved. That is, the location image moves along the touch means moving in a state close to the user terminal.

The user terminal user may check the position image displayed on the display unit and select the corresponding item by directly touching the user terminal. That is, when the touch means is recognized in proximity to the touch panel display, the position image is generated and moved. When the touch means touches the touch panel display unit after generating the position image, the user terminal operates to correspond to a touch operation in a general mobile phone including the touch panel display unit. In this case, the position image is displayed on the display unit according to the user terminal setting. It may be displayed, and may disappear when the touch means directly touches the touch panel display.

According to the present invention, in a touch method using a touch device that is not as sophisticated as a finger, an error of a selection operation can be reduced.

In addition, in order to easily provide the terminal user with the occurrence of the above-described proximity touch, the user terminal may further include an information providing unit for outputting generation information (proximity information) of the proximity touch to the user. For example, the information provider may provide proximity touch generation information to the user in a form of stimulating the user's sense of touch or hearing.

When the generation information of the proximity touch is provided in the form of stimulating the user's tactile sense, the information providing unit may be a vibration motor, and when the user terminal recognizes the proximity touch, the information providing unit vibrates the generation information of the proximity touch to the user. Can provide. Alternatively, when the generation information of the proximity touch is provided in a form that stimulates the user's hearing, the information provider may provide the generation information of the proximity touch to the user through the speaker of the user terminal.

On the other hand, the information providing unit may provide the user with a variety of types of touch or hearing so that the user of the user terminal can feel a variety of touch or hearing, depending on the proximity touch type of the user of the user terminal.

For example, while the user terminal recognizes a proximity touch of the touch means, while the touch means moves, the information provider may generate continuous vibration or sound. That is, when the proximity touch of the first touch means is recognized, one short vibration may be generated, and when the touch means moves later, vibration may be continuously generated. In other words, when the proximity touch of the first touch means is recognized, vibration may occur during the first time, and when the touch means moves later, vibration may occur during the second time. The second time may be longer than the first time.

Alternatively, when the first proximity touch is recognized, vibration may be provided, and when the touch means moves, a sound may be provided from the information providing unit.

The user terminal according to the embodiment of the present invention described above includes a touch panel display unit, a signal unit for transmitting the position information of the touch means adjacent to the touch panel display unit to the receiving terminal at all times, and the touch means is the touch panel display unit. It may include an information providing unit for providing proximity information indicating that the state is close to.

Here, the proximity information provided by the information provider may be information in the form of vibration or sound, and the user terminal may further include a recognition unit recognizing that the touch means is in proximity to the touch panel display.

The information providing unit may provide proximity information for a first time when the recognition unit recognizes the proximity state of the touch means, and when the touch means moves while the recognition unit recognizes the proximity state of the touch means, the second time period. Proximity information can be provided while The second time may be a time corresponding to the time when the touch means moves in a state where the recognizer recognizes a proximity state of the touch means.

The information providing unit may provide different types of proximity information in a first case where the recognizer recognizes a proximity state of the touch means and a second case in which the touch means moves while the recognizer recognizes the proximity state of the touch means. have. As described above, in the first case, vibration may be provided in the second case, or in the second case, the first pattern may be vibrated. In the second case, the second pattern may be vibrated so that the user can feel the movement of the touch means. Can provide.

On the other hand, when the touch means touches the touch panel display after the recognition of the proximity state of the recognition unit, the information providing unit does not provide the proximity information, so that the user terminal user can distinguish the proximity state from the touch directly.

In addition, the information providing unit may provide proximity information when the recognition unit recognizes the proximity state of the touch unit for the second time after the touch unit touches the touch panel display unit.

When the touch means is close to the user terminal and touches the user terminal, the touch means is spaced apart from the user terminal. That is, since the touch purpose is achieved, the touch means is spaced apart from the user terminal for the next operation. At this time, the proximity state occurs again. In this case, since the user does not intentionally generate the proximity state as the first proximity state after the touch means touches the touch panel display, the information providing unit does not provide the proximity information. When the second proximity state occurs, that is, when the user intentionally generates the proximity state, the information provider may provide the proximity information.

Meanwhile, the interworking system between the various terminals described above may be applied as follows.

While using a tablet PC at home or on the go, a call or message may come to your smartphone. However, the smart phone may be located at a location where the tablet PC user is bothered to take out or find the smart phone, such as the smart phone in a bag or a pocket.

The present invention transmits event occurrence information to the tablet PC when a specific event related to communication such as a call or a message comes to the smart phone. Herein, a smartphone and a tablet PC are described as an example, but the present invention is applicable to both mobile terminals. For example, when a specific event occurs such as a call or a message to the tablet PC, the tablet PC may transmit the event occurrence information to the notebook. For convenience of description below, a smartphone and a tablet PC will be described.

The event occurrence information of the smartphone may be transmitted to the tablet PC in the form of a message, or the image displayed on the smartphone where the event occurs may be transmitted to the tablet PC. When event occurrence information is transmitted in a message format, the event occurrence information may include a call reception fact, a reception telephone number, and a message content.

In addition, the present invention not only transmits event occurrence information, but also enables event occurrence in a tablet PC. First, when a call comes to a smartphone, event occurrence information is transmitted to the tablet PC. When event generation information of the smart phone is transmitted in a message format, the tablet PC may provide an interface for providing a call acceptance function. Alternatively, when event occurrence information is transmitted to the tablet PC in the form of an image displayed on the smartphone, the tablet PC may operate like a smartphone.

Tablet PC users can select a call from the screen that is displayed, in which case the data according to the call selection is sent to the smartphone. And the smartphone is activated call function.

The call receiver, that is, the tablet PC user, can make a call using the tablet PC, not the smartphone. Voice data entered through the tablet PC microphone is sent to the smartphone, and sent to the caller phone through the smartphone. Caller voice data is sent to the tablet PC through the smartphone and output through the tablet PC speaker.

In the case of a video call, video data other than audio data may be added and transmitted and received as described above, and the video data is output through the display unit.

When a message is received by the smartphone, event occurrence information is transmitted to the tablet PC. When the event occurrence information of the smartphone is transmitted in a message format, the tablet PC may provide a message input interface. Alternatively, when event occurrence information is transmitted to the tablet PC in the form of an image displayed on the smartphone, the tablet PC may operate like a smartphone.

The message receiver, i.e. the tablet PC user, may reply to the sender using the tablet PC, not the smartphone. The tablet PC user enters a message on the displayed screen and the entered message is sent to the caller via the smartphone.

That is, according to the present invention, the user who used the tablet PC on his or her bed may make a call using the tablet PC that was used without having to go to the living room when the phone came to the smart phone in the living room.

On the other hand, the components of the above-described embodiment can be easily identified from a process point of view. That is, each component can be identified as a respective process.

In addition, the above-described technical features may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art having ordinary knowledge of the present invention may make various modifications, changes, and additions within the spirit and scope of the present invention. Should be considered to be within the scope of the following claims.

100: user terminal 102: receiving terminal
400: second image 500: touch pen
600: receiver 602: touch pen
700: touch panel 702: electromagnetic generator
704: touch pen 1000: the second image
1010: UI 1200: Receiver
1202: Touch Pen 1400: Dongle
1402: touch pen

Claims (2)

A transmitter to transmit event information according to a communication related event to a user terminal; Receiving unit for receiving the response data for the event from the user terminal;
The mobile terminal comprising: The method of claim 1,
The communication related event
Incoming or incoming call event.
The response data
Acknowledgment of receipt of the call or reply data of receipt of the message.
Mobile terminal.

Images