KR20170085515A - A terminal device, a video control system, and a control method of a terminal device - Google Patents

Abstract

A terminal device, a video control system, and a control method of the terminal device are disclosed. The terminal device includes a display unit for displaying moving images captured by the external device, a communication unit for transmitting and receiving signals through the network, and a control unit for sensing information during playback of the moving image. The external device includes a sensor unit to sense information The moving image is transmitted to the network, and the moving image is detected and transmitted by the external device and includes the operation information of the external device inserted by the moving image production apparatus, and the control unit controls the operation of the inserted external device during the reproduction of the moving image Information on the first external device related to the external device and receives the operation information of the first external device associated with the external device and controls the reproduction of the moving image when the received operation information of the first external device corresponds to the operation information of the inserted external device.

Description

A terminal device, a video control system, and a control method of a terminal device

TECHNICAL FIELD The present invention relates to a terminal device, a video control system, and a control method of a terminal device.

As technology advances, various electronic devices are being developed. In addition, electronic devices can perform various functions as they become smaller and higher in performance. On the other hand, in recent years, research has been made on a technique for connecting all objects to a network. This technology, called Internet Of Things (IoT), refers to a technology that connects all things based on the Internet and communicates information between people, things, things and things. Objects connected to the network are not electronic devices in a general sense, but they can be called electronic devices in a broad sense because they can sense peripheral information and transmit sensed information.

Object Internet technology can provide various services in electronic devices that can perform various functions. Accordingly, a variety of useful user interfaces (UIs) or user experiences (UXs) are required based on the object Internet technology in an electronic device.

According to an aspect of the present invention, there is provided a terminal device, an image control system, and a control method of a terminal device capable of controlling a moving image by interacting with an operation of an external device connected to a network .

According to an embodiment of the present invention, a terminal device includes a display unit for displaying moving images captured by an external device, a communication unit for transmitting and receiving signals through a network, and a controller for sensing information during reproduction of the moving image The external device senses information including a sensor unit, and is connected to a network to transmit the sensed information. The moving image is sensed and transmitted by the external device, Wherein the control unit detects operation information of the inserted external device during reproduction of the moving image, receives operation information of the first external device related to the external device, When the operation information corresponds to the operation information of the inserted external device, can do.

The portion where the operation information of the external device is inserted may be set as a forced attribute value or an arbitrary attribute value.

If the portion of the external device in which the operation information of the external device is inserted is set to the forced attribute value and the portion is reproduced, the operation information of the received external device may include operation information of the inserted external device, When it is coped with, the reproduction portion of the moving picture can be moved to the next portion of the reproduced portion.

When the selection signal of the second external device is received and the currently reproduced portion of the moving image is set to the arbitrary attribute value, the control portion may insert the operation information of the photographed external device related to the second external device The moving part of the moving image can be moved.

In addition, the control unit can maintain the currently reproduced portion when the selection signal of the second external device is received and the currently reproduced portion of the moving image is set as the forced attribute value.

The moving image may include a plurality of photographed external apparatuses and a plurality of portions in which operation information of the plurality of external apparatuses is inserted.

Meanwhile, the control unit may suspend the reproduction of the moving picture when the first external apparatus associated with the external apparatus included in the moving picture exists.

If the received operation information of the first external device corresponds to the operation information of the inserted external device, the controller may reproduce the paused video from a point in time at which the paused video is stopped.

In addition, the controller may determine whether the first external apparatus is present by receiving the information of the first external apparatus within a predetermined distance when starting the reproduction of the moving picture.

In addition, the controller may continuously reproduce the moving picture when the first external device associated with the external device included in the moving picture does not exist.

Meanwhile, the first external device may include type information of the same attribute as the captured external device.

The portion including the operation image of the external apparatus may include a plurality of frames, and the operation information of the external apparatus may be inserted into the last frame of the plurality of frames.

The terminal device may further include a camera unit for photographing an external device connected to the network, wherein the control unit receives operation information of the external device when the operation of the external device is sensed, And inserts the operation information of the received external device into the included portion to generate a moving image.

The controller may control the reproduction of the moving picture by further considering size information of the external apparatus and size information of the first external apparatus.

In addition, the moving picture may further include additional information of the external apparatus in a portion where operation information of the external apparatus is inserted.

The additional information of the external device may include at least one of unique information, location information, authority information, type information, size information, component information, and temperature information.

In addition, the controller may control the display unit to display at least one of additional information and operation information of the inserted external device.

The control unit senses the operation information of the inserted external device during reproduction of the moving image and transmits the notification signal to the first external device when the first external device related to the external device exists Can be controlled.

According to an embodiment of the present invention, an image control system displays a moving image in which an external device connected to a network is photographed and included and operation information of an external device sensed by the sensor unit of the external device is inserted, A terminal device for requesting operation information of a first external device associated with the external device by sensing operation information of the inserted external device during reproduction of the moving image and a second external device for transmitting operation information at the request of the terminal device Wherein the terminal device receives the operation information of the first external device and controls the reproduction of the moving image when the operation information of the first external device corresponds to the operation information of the inserted external device .

According to an embodiment of the present invention, there is provided a method of controlling a terminal device, the method comprising: displaying a moving image in which an external device connected to a network is photographed and included and operation information of an external device sensed by the external device is embedded; Detecting operation information of the inserted external device during reproduction of the moving image, receiving operation information of a first external device related to the external device, and receiving operation information of the first external device, And controlling the reproduction of the moving picture when it corresponds to the operation information of the apparatus.

According to the above-described various embodiments, the terminal device, the image control system, and the control method of the terminal device can control the moving image by sensing the information of the external device.

The terminal device, the image control system, and the control method of the terminal device can generate a moving image in which information about the photographed external device is inserted.

Also, the terminal device, the video control system, and the control method of the terminal device can control the moving picture according to the property value of the moving picture.

Also, the terminal device, the video control system, and the control method of the terminal device can control the moving picture according to whether there is a real external device associated with the photographed external device in the moving picture.

Also, the terminal device, the image control system, and the control method of the terminal device can transmit a signal to a real external device related to the photographed external device in the moving image.

1 is a block diagram of a terminal device according to an embodiment.
2 is a block diagram of a terminal device according to another embodiment.
3 is a view for explaining an image control system according to an embodiment.
4 is a view for explaining an image control system according to another embodiment.
FIG. 5 is a diagram for explaining a process of generating moving images according to an embodiment.
FIG. 6 is a view for explaining an embodiment for controlling a moving picture being reproduced.
7 is a view for explaining an embodiment for displaying guide information.
8 is a view for explaining another embodiment for displaying guide information.
9 is a view for explaining an embodiment for controlling peripheral external devices.
10 is a diagram for explaining an embodiment of a moving picture that interacts with the operation of an external apparatus.
11 is a view for explaining another embodiment of a moving picture that interacts with the operation of an external apparatus.
12 is a timing diagram illustrating a process of generating a moving image according to an embodiment.
13 is a timing chart for explaining a process of controlling a moving picture according to an embodiment.
FIG. 14 is a flowchart of a terminal device control method according to an embodiment.

Best Mode for Carrying Out the Invention

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. The configuration and operation of the present invention shown in the drawings and described herein by way of example are described as at least one embodiment, and the technical idea of the present specification and its core configuration and operation are not limited thereby.

As used herein, terms used in the present specification are taken to be those of ordinary skill in the art and are not intended to limit the scope of the present invention to those of ordinary skill in the art. In addition, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description part of the corresponding specification. Accordingly, it is intended that the terminology used herein should be defined not only by the nomenclature of the term, but also by the meaning of the term and its scope throughout this specification.

1 is a block diagram of a terminal device according to an embodiment.

Referring to FIG. 1, a terminal device 100 may include a communication unit 110, a control unit 120, and a display unit 130.

The communication unit 110 can transmit and receive signals through the network. The communication unit 110 can exchange data with an external device or a server. The external device includes a communication module and a sensor module and can be connected to a network. An external device can sense information using a sensor module. The information detected by the external device may include direct information of the external device and information around the external device.

For example, the direct information of the external device may include operation information of the external device, temperature information of the external device, component information of the substance contained in the external device, or capacity information of the substance contained in the external device. The information around the external device may include sound information around the external device and the like. Herein, the information of the external device or the information related to the external device may include all the information that the external device can detect. The external device may further include a storage module. Thus, the external device can store the sensed information in the storage module. In one embodiment, the external device may include an Internet Of Things (IoT) device.

The display unit 130 can display a moving image captured by an external device. The moving image captured and included in the external device may include information of the photographed external device. In one embodiment, when the terminal device 100 photographs a TV to generate a moving image, the terminal device 100 can receive information of the TV. The terminal device 100 can insert the information of the TV received in the frame including the TV in the moving image. A detailed description of the video generation will be given later. The terminal device 100 generating the moving picture may be the same device as the terminal device 100 reproducing the moving picture or may be another device. That is, the terminal device 100 can capture an external device, generate a moving image, and reproduce the generated moving image. Alternatively, one terminal device 100 may capture an external device to generate a moving image, and may transmit the generated moving image to the other terminal device 100. The terminal device 100 receiving the moving image can reproduce the transmitted moving image. The display unit 130 may be implemented with a touch sensor to receive touch input from the user.

The control unit 120 may sense information during playback of a moving image. That is, the control unit 120 may detect the inserted operation information of the photographed external device included in the moving image during the reproduction of the moving image. In one embodiment, a moving image may be a set of frames composed of images. An image of the photographed external apparatus may be included in a plurality of frames in the moving image. That is, a moving image of a photographed external apparatus may be composed of a plurality of frames including an image of a photographed external apparatus which changes little by little. The moving image producing apparatus (or the terminal apparatus) can insert information of an external apparatus photographed in one of a plurality of frames when the moving image is generated. For example, the information of the inserted external device may include unique information, location information, authority information, type information, size information, operation information, component information, or temperature information. The information of the inserted external device may be one or a plurality of information. The unique information may mean identification information capable of identifying an external device. The authority information may be information indicating a range in which information of the external apparatus can be shared. For example, the credential can have a value of 'ownership only', 'shared as specified', or 'shared as a whole'. The type information may be information indicating the type of the external apparatus. The motion picture production apparatus can insert information into the first frame in which motion of the photographed external apparatus starts. Alternatively, the moving picture production apparatus may insert the information into the last frame in which the motion of the photographed external apparatus is terminated.

The control unit 120 may detect the information of the inserted external device when reproducing the frame in which the information of the photographed external device is inserted. When the controller 120 detects the information of the inserted external device, the control unit 120 can control the playback of the moving image. For example, the control unit 120 can pause the playback of the moving image or display the guide information. Alternatively, the control unit 120 may repeatedly reproduce a plurality of frames including an external device photographed slowly or photographed.

Meanwhile, the control unit 120 may wait for reception of information of the first external device related to the photographed external device. The first external device associated with the photographed external device may be an external device that is not the same external device as the photographed external device but has a certain relevance. For example, if the photographed external device is a cup of 200 ml capacity, the first external device may be a cup of 500 ml capacity. That is, the first external device may include the kind information of the same attribute as the photographed external device.

Upon receiving the information of the first external apparatus, the control unit 120 can compare the information of the external apparatus inserted in the moving picture. The control unit 120 can control the reproduction of the moving picture when the information of the received first external apparatus is associated with the information of the inserted external apparatus. For example, the control unit 120 can normally reproduce the moving image. Alternatively, the control unit 120 may output a notification informing that the information of the inserted external device corresponds to the information of the received first external device.

2 is a block diagram of a terminal device according to another embodiment.

2, the terminal includes a communication unit 110, a control unit 120, a display unit 130, a camera unit 140, an input unit 150, a sensor unit 160, an audio unit 170, A power supply unit 180, and a power unit 190.

The communication unit 110 can communicate with an external device using various protocols and transmit and receive data. The external device may be a mobile terminal or a fixed terminal. For example, the mobile terminal may be a cellular phone, a smart phone, a tablet PC, a smart pad, a notebook, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP) And a fixed terminal may be a desktop, a DVD (Digital Video Disc or Digital Versatile Disc) player, a TV, or the like. Further, the external device may include an IoT device in addition to a general electronic device. The IoT device may be any object including a sensor module and a communication module. That is, all objects capable of sensing information using a sensor module and transmitting information using a communication module may be IoT devices. For example, if a sensor module and a communication module are included, the IoT device may be a restaurant spice container, a cutting board, a plate, a frying pan, a cup, a vehicle, a vase, an umbrella, a frame, a cosmetic, a pencil case, a notebook, It is a matter of course that the mobile terminal and the fixed terminal described above can also be IoT devices.

The various protocols may be wired or wireless communication protocols. In the case of wired communication, the communication unit 110 may include various input / output interfaces (not shown) that enable wired data transmission communication with an external device. For example, the interface may include an interface that considers data transmission according to Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Digital Visual Interface (DVI), IEEE 1394, have. In the case of wireless communication, the communication unit () may include an RF electric network considering wireless connection to an external communication network such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network). The wireless communication network to be connected by the communication unit may be a Global System for Mobile Communications (GSM), an Enhanced Data GSM Environment (EDGE), a Code Division Multiple Access (CDMA), a Wideband Code Division Multiple Access (W- (Radio Frequency Identification), infrared (IrDA) communication, and the like, as well as mobile communication methods such as wireless access, Wibro, High Speed Packet Access (HSPA), High Speed Downlink Packet Access (HSDPA) (UWB), ZigBee, and WLAN (Wi-Fi), may be supported by the wireless LAN.

Herein, the wired / wireless interface scheme is an embodiment for facilitating understanding of the present specification, and the interface scheme for transmitting / receiving information can be easily changed by a person skilled in the art, so that the interface scheme in this specification is limited to the above- .

The control unit 120 may sense information of the inserted external device during reproduction of the moving image. The control unit 120 receives the information of the first external device related to the external device in the moving image and controls the reproduction of the moving image when the information of the received first external device corresponds to the information of the inserted external device have. Also, the control unit 120 may execute the content stored in the storage unit 180 or the content received through the data communication. In addition, the control unit 120 can execute various applications and process data in the terminal device. In addition, the control unit 120 may control the respective components of the terminal device, and may control data transmission / reception between the respective components. The control unit 120 includes a processor, an application-specific integrated circuit (ASIC), another chipset, a logic circuit, a register, a communication modem, a data processing device, and the like, which are known in the related art for executing various control logic can do. Further, when the above-described control logic is implemented in software, the control unit 120 may be implemented as a set of program modules. At this time, the program module is stored in the storage unit 180 and can be executed by the processor.

The display unit 130 may output an image on a display screen. The display unit 130 may output an image based on a content or a control command executed by the control unit 120. [ In this specification, the display unit 130 may output a moving image including information on the photographed external device including the photographed external device. Meanwhile, the display unit 130 may be implemented in the form of a display unit or a beam projector of a head mounted display.

The camera unit 140 captures an image of the surrounding environment of the terminal and converts the captured image into an electrical signal. To this end, the camera unit 140 may include an image sensor capable of converting an optical signal into an electrical signal. An image photographed by the camera unit 140 and converted into an electrical signal may be output to the control unit 120 after being stored in the storage unit 180 or may be output to the control unit 120 without being stored. The image captured by the camera unit 140 may be a still image or a moving image. The camera unit 140 may be used as a motion sensor or a video sensor. The camera unit 140 may include a plurality of cameras as needed. In this specification, the camera unit 140 can photograph an external device connected to the network.

The input unit 150 can receive a user command from the outside. The input unit 150 may be implemented in various ways. For example, the input unit 150 may be implemented by a keyboard, a keypad, a mouse, a touchpad, a button, a soft key, or the like. In a broad sense, the input unit 150 may include a microphone, a touch screen, and the like. The microphone can receive the user's voice, and the touch screen can receive the user's touch gesture. In some cases, the microphone may be included in the audio unit 170, and the touch screen may be included in the display unit 130.

The sensor unit 160 may sense the surrounding environment of the terminal device using at least one sensor mounted on the terminal device and may transmit the detected ambient environment to the control unit 120. In addition, the sensor unit 160 may detect a user input and transmit the detected user input to the controller 120. At this time, the sensor unit 160 may include at least one sensing means. For example, the sensing means may be a gravity sensor, a geomagnetic sensor, a motion sensor, a gyro sensor, an acceleration sensor, an infrared sensor, an inclination sensor, a brightness sensor, an altitude sensor, A sensor, a banding sensor, an audio sensor, a video sensor, a global positioning system (GPS) sensor, a touch sensor, and a fingerprint sensor. The sensor unit 160 collectively refers to the various sensing means described above, and can sense various inputs of the user and the environment of the user, and can transmit the sensing result so that the controller 120 can perform the operation accordingly. The sensing means may be included in the terminal device as a separate element or integrated into at least one or more elements.

Meanwhile, the terminal device may include the sensor unit 160 on the display unit 130 together. Accordingly, the terminal device can sense various user inputs performed on the display unit 130 through the sensor unit 160. [ For example, when the sensor unit 160 includes a touch sensor, the terminal device can receive a variety of touch inputs of the user performed on the display unit 130. Also, when the sensor unit 160 includes a fingerprint sensor, the terminal device can collect fingerprint information of the user from the user input performed on the display unit 130. [ As described above, the terminal device of the present invention may include a screen sensor in which the display unit 130 and the sensor unit 160 have a mutual layer structure.

The audio unit 170 may include audio output means such as a speaker and audio input means such as a microphone. The output means can output the audio signal of the content being executed in the terminal device. The content may be provided from the storage unit 180 or may be provided from the external device through the communication unit 110. The output means may include at least one of an air conduction speaker and a bone conduction speaker. The air conduction speaker is an earphone or the like. The air conduction speaker generates sound waves by vibrating the air according to the audio signal. In other words, the vibration of the sound transmitted to the air is transmitted to the eardrum in the ear, and the vibration of the eardrum is transmitted to the screw-shaped wow through the three bones in the eardrum. The wah is filled with a liquid called lymph fluid. The vibration of the liquid is converted into an electric signal, which is transmitted to the auditory nerve, and the human brain recognizes the sound. The bone conduction speaker can be placed in various positions that can provide the user with an audio signal that is converted to vibration in frequency form. If a bone conduction speaker is used, the audio signal drives the bone conduction speaker so that the bone conduction sound wave is conducted to the user ' s skull so that the frequency type vibration is transmitted to the user. The bone conduction speaker allows the user to hear the audio signal on the eardrum without difficulty. Meanwhile, the audio unit 170 may be used as an audio sensor.

The storage unit 180 may store various digital data such as video, audio, photographs, and applications. The application may be a program for the operation of the control unit 120. [ In addition, the video or the photograph may be data acquired through the camera unit 140. The storage unit 180 may be implemented as a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), or a programmable read only memory . In addition, the storage unit 180 may operate in association with a web storage on the internet to perform a storage function. The storage unit 180 may further include an external storage medium detachable from the terminal apparatus. The external storage medium may be a slot type such as SD (Secure Digital) or CF (Compact Flash) memory, a memory stick type, or a USB (Universal Serial Bus) type. That is, the external storage medium can be detachably attached to the terminal device, and can provide the terminal device with contents such as audio, photograph, moving picture, application, and the like. For example, the storage unit 180 may be a RAM, a ROM, a cache memory, a hard disk drive, a solid state drive (SSD), a CD, a DVD, a Blu-ray disk, a floppy disk (FDD) , A magnetic disk, a memory card, a flash memory, a USB memory, and the like.

The power unit 190 is a power source connected to a battery or an external power source, and can supply power to the terminal device. The battery includes a secondary battery as well as a primary battery. In the case of a secondary battery, a rechargeable lithium ion battery, a lithium polymer battery, a nickel cadmium battery, a nickel hydride battery, a nickel zinc battery, or the like may be used.

Meanwhile, a terminal device may mean various types of devices capable of processing digital data and performing operations corresponding thereto. As the performance of such a terminal device is improved, various contents can be executed through the terminal device. For example, the terminal device may include a TV, a laptop computer, a desktop computer, a monitor, a smart phone, a tablet PC, a navigator, a PMP, a payment terminal, a security terminal, a kiosk, Particularly, a portable device which is widely used recently is also utilized as a comprehensive multimedia contents reproducing apparatus.

The block diagram of the terminal device has been described so far. The image control system and specific embodiments will be described below.

3 is a view for explaining an image control system according to an embodiment.

Referring to FIG. 3, the image control system may include a terminal device 100 and external devices 200a, 200b, 200c, 200d, and 200e. There can be only one external device, and there may be a plurality of external devices. The external device 200 can be connected to a network and exchange information with the terminal device 100.

First, each external device 200 may include a sensor module and a communication module. The external device 200 can sense information using the sensor module. For example, a frying pan can sense information such as the time used, the heating temperature, and the like, and the glass can sense information such as the time used, the composition of the material used, and the amount of material used. The pollen can detect information such as temperature, humidity and the like. The external device 200 can generate the information group by tagging the detected information together with the unique information. That is, the external device 200 can generate the tagging information as information together with the unique information, the identification information, the authority information, the owner information, the location information, and the like. The type of the sensing information may vary depending on the external device 200. The external device 200 may further include a storage module to store the sensed information.

The terminal device 100 can take an image including an external device. The terminal device 100 can generate moving images using a series of images. A moving picture may be a set of frames including an image. The terminal device 100 can sense the external device 200 included in the photographed image. In one embodiment, the terminal device 100 can recognize the appearance information such as the outline and color of the external device in the photographed image. The terminal device 100 can communicate with an external device in the vicinity to detect which device is included in the photographed image. For example, the terminal device 100 can transmit the external information of the external device included in the image to the external device 200 and inquire whether the external device 200 matches the external information. The external device 200 can transmit a response signal to the terminal device 100 indicating that the external information received from the terminal device 100 matches the information of the external device 200 itself. The terminal device 100 can recognize the external device in the image as a response signal received from the external device 200. [ The external device 200 may transmit the tagging information together with the response signal indicating matching to the terminal device 100 together.

As another example, the terminal device 100 can request tagging information from the external device 200 in the vicinity. The external device 200 can transmit the tagging information at the request of the terminal device 100. [ The terminal device 100 can compare the received tagging information with the recognized external appearance information of the external device to confirm whether the tagging information matches or not. The terminal device 100 can detect the external device in the image based on the tagging information of the matching external device. The terminal device 100 can perform communication with the external device 200 within a certain distance. The distance to the external device 200 can be recognized based on the positional information of the external device 200 or the intensity of the signal received from the external device 200 or the like. However, a certain distance may be a concept of an approximate distance arbitrarily set rather than a concept of a clear distance which is generally recognized. As shown in FIG. 3, the fifth external device 200e, which is distant from the terminal device 100 by a predetermined distance, may not perform communication.

In one embodiment, the terminal device 100 can recognize the operation of the external device 200 when capturing an image including the external device. The terminal device 100 can recognize the motion of the external device 200 by using the image processing method. When the movement of the external device 200 is recognized, the terminal device 100 can request the operation information of the external device 200. [ The external device 200 can transmit the operation information to the terminal device 100. [ Alternatively, when the external device 200 moves, the external device 200 can transmit its operation information to the terminal device 100 regardless of the request of the terminal device 100. [ Alternatively, when the image capturing is started, the terminal device 100 requests the external device 200 for the operation information, and when receiving the request, the external device 200 transmits its operation information to the terminal device 100 .

The terminal device 100 can insert the operation information received from the external device 100 into the portion including the captured image of the motion of the external device. That is, when the terminal device 100 captures a moving image including the external device 200, the external device 200 may be photographed with a plurality of images and composed of a plurality of frames. The terminal apparatus 100 can insert operation information of an external apparatus received in one of a plurality of frames including an external apparatus. For example, the external device 200 can insert the operation information of the received external device into the frame in which the external device 200 starts to move, or insert it into the frame in which the movement ends. The terminal device 100 can generate a moving image in which the operation information of the received external device is inserted.

On the other hand, the terminal device 100 can insert other information in addition to the operation information of the external device. For example, the terminal device 100 may insert unique information, identification information, authority information, owner information, location information, type information, size information, component information, or temperature information.

The terminal device can reproduce the generated moving image. In some cases, the generated moving image can be transmitted to other terminal devices. That is, the terminal device that captures the generated moving image and the terminal device that reproduces the moving image may be the same terminal device or different terminal devices.

When the terminal device 100 reproduces a moving image, it can detect the information of the inserted external device. Then, the terminal device 100 can detect the external device 200 in the vicinity. The external device 200 included in the moving picture and the external device 200 around the terminal device 100 when the moving picture is reproduced may be the same external device or another external device. An external device in the vicinity of the terminal device when the moving image is reproduced to distinguish it from the photographed external device 200 will be referred to as a first external device 200. [ The process of the terminal device 100 sensing the first external device 200 is the same as the process of detecting the external device for photographing. The terminal device 100 can perform communication with the first external device 200 within a predetermined distance. The terminal device 100 can determine whether the first external device 200 exists by receiving the information of the first external device.

The terminal device 100 can control the reproduction of the moving picture in the frame in which the information of the photographed external apparatus is inserted during the reproduction of the moving picture. For example, the terminal device 100 may stop reproduction of a moving image, and may repeatedly reproduce a plurality of frames in which information of a photographed external apparatus is inserted. Alternatively, the terminal device 100 may display guide information in a moving image or may give a notification effect to the corresponding frame. Alternatively, the terminal device 100 may play back the moving image slowly.

The terminal device 100 can receive the operation information from the first external device associated with the photographed external device. For example, if the photographed external device is glass, the cup or beaker may be the first external device involved. However, the knife or spoon may not be the first external device involved. That is, a first external device associated with one external device may refer to an external device that can be regarded as a similar group in terms of use. A similar group can be determined using category information or identification information, and a similar group of criteria can be set by the user. The terminal device 100 can control the playback of the moving image again if the received operation information of the first external device corresponds to the operation information of the inserted external device. For example, the terminal device 100 can normally reproduce a moving image.

As shown in FIG. 3, since there may be a plurality of external devices that are photographed and included in the moving image, a plurality of related first external devices may exist. The terminal device 100 may perform the same process each time a frame including information of an external device included in a moving image is reproduced. Meanwhile, the external device 200 may transmit the sensing information to the server, and the terminal device 100 may connect to the server and receive necessary sensing information.

4 is a view for explaining an image control system according to another embodiment.

Referring to FIG. 4, the image control system may include a terminal device 100, an external device 200, and a server 300. Although one external device is shown in Fig. 4, one or more external devices 200 may exist. The server 300 may be connected to the terminal device 100 and the external device 200 via a network to exchange information.

As described with reference to FIG. 3, the external device 200 can sense information using the sensor module. The external device 200 can transmit the sensed information to the server 300 using the communication module. For example, the external device 200 can transmit identification information, category information, operation information, and the like to the server 300. The server 300 may store information received from the external device 200. [ When receiving information from the external device 200, the server 300 may determine whether the external device 200 has transmitted the information. If the external device 200 has transmitted the information, it can store the received information in addition to the existing information. Accordingly, the server 300 can generate the history information associated with one external device 200. [

The terminal device 100 can take an image including the external device 200. [ The terminal device 100 can sense the external device 200 included in the photographed image. In one embodiment, the terminal device 100 can recognize the outline and color of the external device 200 in the photographed image. The terminal device 100 can transmit the external information of the recognized external device 200 to the server 300. [ The server 300 can retrieve the information of the stored external device 200 that matches the external information of the external device 200 received from the terminal device 100. [ The server 300 can transmit information of the matched external device to the terminal device 100. [ The terminal device 100 can detect the photographed external device based on the information of the external device received from the server 300. [

Alternatively, the terminal device 100 may request the server 300 for information about the external device in the vicinity. The server 300 may transmit information about the external device 200 in the vicinity of the terminal device 100 in response to a request from the terminal device 100. [ The server 300 may transmit all the information of the external device 200 or may transmit only information necessary to identify the external device 200. [ For example, the information required to identify the external device 200 may be at least one of unique information, identification information, position information, color information, type information, and type information. The unique information or identification information may mean information unique to each external device 200 or information that can be distinguished from other external devices. The type information may mean information about the type of external device. The terminal device 100 can retrieve information that matches the information of the recognized external device from the information of the received external device. The terminal device 100 can detect the external device 200 from the searched information. On the other hand, the terminal device 100 can transmit its own information to the server 300. The server 300 may store the information of the received terminal device 100.

As described above, the external device 200 can transmit the sensed information to the server 300. [ The terminal device 100 can receive the operation information of the external device and insert it into the moving image. In one embodiment, the terminal device 100 can recognize the operation of the external device 200 when capturing an image including the external device. When the movement of the external device 200 is recognized, the terminal device 100 can request the operation of the external device 200 from the server 300. [ The server 300 may transmit the operation information of the external device 200 to the terminal device 100. [ Alternatively, the server 300 may transmit the received operation information of the external device to the terminal device 100 even if there is no request from the terminal device 100. Alternatively, when image capturing is started, the terminal device 100 requests the server 300 for operation information of the external device, and the server 300 that has received the request transmits the operation information of the received external device to the terminal device 100 .

The terminal device 100 can insert the operation information received from the external device 100 into the portion including the captured image of the motion of the external device. That is, when the terminal device 100 captures a moving image including the external device 200, the external device 200 may be photographed with a plurality of images and composed of a plurality of frames. The terminal apparatus 100 can insert operation information of an external apparatus received in one of a plurality of frames including an external apparatus. On the other hand, the terminal device 100 can insert other information in addition to the operation information of the external device.

The terminal device can reproduce the generated moving image. In some cases, the generated moving image can be transmitted to other terminal devices. That is, the terminal device that captures the generated moving image and the terminal device that reproduces the moving image may be the same terminal device or different terminal devices.

When the terminal device 100 reproduces a moving image, it can detect the information of the inserted external device. Then, the terminal device 100 can detect the first external device 200 in the vicinity. The external device 200 captured in the moving image and the first external device 200 in the vicinity may be the same external device or another external device. The terminal device 100 can perform communication with the first external device 200 within a predetermined distance. The terminal device 100 can determine whether the first external device 200 exists by receiving the information of the first external device.

When the terminal device 100 recognizes the frame in which information of the photographed external device is inserted during reproduction of the moving image, the terminal device 100 can receive the operation information from the first external device 200 related to the photographed external device. The terminal apparatus 100 can control the reproduction of the moving picture when the received operation information of the first external apparatus corresponds to the operation information of the inserted external apparatus.

In FIG. 4, information exchange between the terminal device 100 and the external device 200 can be performed through the server 300 as an intermediary. The operation of the terminal device 100 or the external device 200 may be the same as that described with reference to FIG. 3, except that information exchange is performed through the server 300. [

FIG. 5 is a diagram for explaining a process of generating moving images according to an embodiment.

Referring to Fig. 5 (a), there are a plurality of external apparatuses 201a, 201b, 201c and 201d. For example, a plurality of external devices may be glass 201a, saucepan 201b, frying pan 201c, and chopping 201d. The user can shoot an image including a plurality of external devices 201a, 201b, 201c, and 201d using the terminal device 100. [ As described above, the terminal device 100 can detect the external device from the image displayed. The terminal apparatus 100 can receive information of the external apparatus. For example, the information of the received external device may include unique information, location information, authority information, type information, size information, and the like.

The terminal device 100 can detect the movement of the external device. When the motion of the external device is detected, the terminal device 100 can receive the operation information of the external device. For example, when the glass 201a is used, the terminal device 100 can recognize the movement of the glass 201a and request the operation information of the glass 201a. As described above, the terminal device 100 May perform direct communication with an external device, or may perform communication with a server. Alternatively, as described above, the terminal device 100 requests information of the external device in the photographed image. Accordingly, the external device that has transmitted the information to the terminal device 100 can also recognize the presence of the terminal device 100. Accordingly, the external device can detect the movement of the external device and transmit the operation information to the terminal device 100. [ That is, the glass 201a can sense its movement and transmit its operation information to the terminal device 100. [ Alternatively, the terminal device 100 may request transmission of information to the detected external device when photographing is started. The external device can transmit operation information to the terminal device 100 when it detects its own motion. On the other hand, the external device can transmit further sensing information in addition to the operation information. For example, the other sensing information may include motion information, component information, temperature information, and the like. If the server transmits information to the terminal device 100, the server can transmit the operation information of the external device.

Referring to FIG. 5 (b), a moving picture with information of an external device is shown. A moving picture may be a set of frames including an image. Video refers to content that allows users to feel motion by continuously displaying a series of frames at a constant rate. For example, if the terminal device 100 photographs the operation of pouring the water of the glass 201a into the frying pan, it may be implemented as a plurality of frames as shown in Fig. 5 (b). The first frame 11 is an image in which the user holds the glass 210a, the second frame 12 and the third frame 13 are images in which the user starts swelling water, The image that has completed swelling, the fifth frame may include an image that the user prepares for the next action.

The terminal apparatus 100 may insert the information of the received external apparatus into a frame including an operation image of the external apparatus. For example, the terminal apparatus 100 may insert information of the received external apparatus into a frame including an image in which the operation of the external apparatus is completed. 5, since the fourth frame 14 includes the completed image of the water swelling operation, the terminal device 100 can insert the information of the external device into the fourth frame 14. [ (I.e., the terminal device 10) can insert operation information with the type information, the capacity information, and the component information in the fourth frame 14. [ The operation information may be information such as 'swelling', and may be expressed as 'once'. The insertion information exemplified in Fig. 5 is one embodiment, and the information shown in Fig. 5 (b) may be excluded or other information may be added. In addition, the information to be inserted may vary depending on the type of the external apparatus.

In some cases, the terminal apparatus 100 may insert information of an external apparatus into a frame including an image in which the operation of the external apparatus starts. That is, the terminal device 100 may insert the information into the first frame 11.

The terminal device 100 can generate a moving image in which information of the external device is inserted. The generated moving image may be transmitted to the other terminal apparatuses 100. Then, the terminal device 100 having the generated moving image can reproduce the moving image.

FIG. 6 is a view for explaining an embodiment for controlling a moving picture being reproduced.

Referring to Fig. 6 (a), a plurality of external devices 202a, 202b, and 202c may exist. The plurality of external devices 202a, 202b, and 202c may be the same device as the photographed external device, or may be another device. The terminal device 100 can reproduce the moving picture. When the terminal device 100 detects the information of the inserted external device during the reproduction of the moving image, the reproduction of the moving image can be controlled. For example, the terminal device 100 can pause the playback of the moving image or display the guide information. Alternatively, the terminal device 100 may repeatedly reproduce a plurality of frames including an external device that is slow-reproduced or photographed.

Referring to FIG. 6 (b), an embodiment of a terminal device 100 for controlling moving images is shown. The terminal device 100 can control the moving picture when the motion information of the inserted glass is detected while reproducing the moving picture. For example, the terminal device 100 may suspend the reproduction of the moving image. The terminal device 100 can stop the reproduction of the moving picture until it receives the operation information from the associated first external device.

As shown in Fig. 6 (b), when the moving image is stopped at the portion where the glass is used, the user can use the saucepan 202b and use the cup 202a. First, when the user uses the saucepot 202b, he can transmit the operation information to the terminal device 100 when the saucepot 202b senses the movement. The terminal device 100 can receive the operation information of the sauce container 202b and compare it with the operation information of the glass inserted in the moving image. Since the glass (external device) and the saucepot 202b (first external device) in the moving image are different devices, the operation information is also different. Therefore, the operation information of the glass and the operation information of the seasoning bucket 202b do not correspond. The terminal device 100 can continue to control the moving picture.

When the user uses the cup 202a, when the cup 202a senses the movement, the operation information can be transmitted to the terminal device 100. [ The terminal device 100 receives the operation information of the cup 202a and can compare the information with the operation information of the glass inserted into the moving image. Since the glass (external device) and the cup 202a (first external device) in the moving picture are the same kind of devices, the terminal device 100 can compare the operation information. When the operation information of the glass and the operation information of the cup 202a correspond to each other, the terminal device 100 can control the moving image again. For example, the terminal device 100 can reproduce a paused moving picture from a point in time at which it is stopped. Alternatively, the control unit 120 may output a notification informing that the information of the inserted external device corresponds to the information of the received first external device.

In one embodiment, the terminal device 100 may detect an external device included in a moving image and search for a first external device associated with the external device included in the moving image. For example, a first external device associated with an external device included in a moving picture may mean a device having the same type information. As described above, the glass and cup 202a may be devices that are related to each other, and the glass and spice bottle 202b may be unrelated devices. In one embodiment, the terminal device 100 can search for the presence of the first external device within a predetermined range.

When the first external device related to the external device included in the moving picture exists, the terminal device 100 can control the moving picture by detecting the operation information of the inserted external device. For example, the terminal device 100 may suspend the reproduction of the moving image. The terminal device 100 can receive the operation information from the associated first external device. For example, the terminal device 100 may request operation information from the first external device, or may transmit the operation information to the terminal device 100 when the first external device senses the movement of the first external device. In some cases, signals may be transmitted and received via the server. The terminal apparatus 100 can control the reproduction of the moving picture again if the operation information of the received first external apparatus corresponds to the operation information of the inserted external apparatus. For example, the terminal device 100 can reproduce a paused moving picture from a point in time when the paused moving picture is stopped. As a result, if there is no first external device associated with the external device included in the moving image, the terminal device 100 may continue to display the moving image even if the terminal device 100 detects the operation information of the inserted external device Or reproduced.

7 is a view for explaining an embodiment for displaying guide information.

Referring to FIG. 7, a terminal device 100 displaying guide information on a display screen is shown. The terminal device 100 can control the moving picture when it detects the operation information of the inserted external device. For example, the terminal device 100 can pause the playback of the moving image or display the guide information. Alternatively, the terminal device 100 can pause playback of the moving image and display the guide information.

For example, the displayed guide information may include type information of the external device, capacity information, and operation information. As shown in FIG. 7, the terminal device 100 can display information such as '200 ml' and '1 cup'. The guide information shown in FIG. 7 is one embodiment, and the terminal device 100 can display various information. The terminal device 100 may further display additional information of the external device.

As described above, the terminal apparatus 100 can receive the operation information of the first external apparatus related to the external apparatus and correlate the operation information with the operation information of the inserted external apparatus. If the received operation information of the first external device corresponds to the operation information of the inserted external device, the terminal device can control the moving image. In FIG. 7, the terminal device 100 can control the moving picture when the operation information of the received first external device is 1 cup. However, the user does not always use the same device. For example, devices of similar type but different sizes and shapes may be used.

8 is a view for explaining another embodiment for displaying guide information.

Referring to FIG. 8, a terminal device 100 displaying guide information on a display screen is shown. As described above, the first external device may not be the same as the external device included in the moving image. That is, the first external device may have the same type information as the external device included in the moving image but may have a different size. For example, an external device included in a moving image is used once with a capacity of 200 ml, but a related first external device can have a capacity of 4000 ml. The terminal device 100 can receive various pieces of information of the first external device when searching for the first external device. For example, the terminal device 100 can further receive capacity information and the like. Then, when the operation information of the inserted external device is detected, the terminal device 100 can pause the moving image and display the guide information.

The terminal device 100 may further display additional information of the external device. That is, the terminal device 100 can display the guide information considering the first external device together with the information of the inserted external device. As shown in FIG. 8, when the information of the inserted external device is 200 ml and 1 cup, and the information of the first external device is 400 ml, information of 400 ml and 0.5 cup can be displayed. Therefore, when the information of the inserted external device is 200 ml, 1 cup, 400 ml of the first external device corresponds to information of the inserted external device when 0.5 cup is used. That is, the terminal device 100 can determine the correspondence of information by considering the size information in addition to the type information and the operation information.

9 is a view for explaining an embodiment for controlling peripheral external devices.

Referring to FIG. 9 (a), the terminal device 100 reproduces a moving image. A plurality of external devices 202a, 202b, and 202c may be present. The terminal device 100 can detect the information of the inserted external device when reproducing the moving image. The terminal device 100 can detect the external device included in the moving image and search for the first external device related to the external device included in the moving image. For example, the external device included in the moving picture is glass, and the first external device that is searched may be the cup 202a. The terminal device 100 can detect the operation information of the inserted external device while reproducing the moving image. The terminal device 100 can control the moving picture when it detects the operation information of the inserted external device. For example, the terminal device 100 may suspend the reproduction of the moving image.

The terminal device 100 may detect the inserted operation information of the external device during the reproduction of the moving image and may transmit the notification signal to the first external device when the first external device exists. As shown in FIG. 9 (a), the terminal device 100 can transmit a vibration signal to the cup 202a. The cup 202a may receive a signal from the terminal device 100 and vibrate. As another example, when the terminal device 100 transmits a notification signal, the cup 202a can perform a notification operation as much as possible. For example, the cup 202a may blink an LED or output a sound. The terminal device 100 may transmit a notification signal to the first external devices 202a, 202b, and 202c, and the first external devices 202a, 202b, and 202c may perform corresponding operations. Accordingly, the terminal device 100 can help the user to perform necessary operations.

Referring to FIG. 9 (b), an embodiment of a terminal device 100 for controlling moving pictures is shown. The terminal device 100 can control the moving picture when the motion information of the inserted glass is detected while reproducing the moving picture. For example, the terminal device 100 may suspend the reproduction of the moving image. The terminal device 100 can stop the reproduction of the moving picture until it receives the operation information from the associated first external device.

10 is a diagram for explaining an embodiment of a moving picture that interacts with the operation of an external apparatus.

Referring to Fig. 10 (a), a cup 202a which is the first external device is used. For example, the terminal device 100 can control the moving picture when the motion information of the inserted glass is detected while reproducing the moving picture. In one embodiment, the terminal device 100 can temporarily stop the reproduction of the moving image. The terminal device 100 can stop the reproduction of the moving picture until it receives the operation information from the associated first external device. When the moving picture is stopped at the portion where the glass is used, the user can use the cup 202a.

The cup 202a may include a sensor module and a communication module as one device. When the cup 202a senses the movement, the operation information can be transmitted to the terminal device 100. [ The terminal device 100 receives the operation information of the cup 202a and can compare the information with the operation information of the glass inserted into the moving image. Since the glass (external device) and the cup 202a (first external device) in the moving picture are the same kind of devices, the terminal device 100 can compare the operation information. When the operation information of the glass and the operation information of the cup 202a correspond to each other, the terminal device 100 can control the moving image again.

Referring to FIG. 10 (b), a moving picture that is controlled when the information of the inserted external device corresponds to the information of the received first external device is shown. The terminal device 100 can reproduce the temporarily stopped moving image from the stopped point.

On the other hand, the portion of the moving picture where the information of the external apparatus is inserted may further include an attribute value. The attribute value can be set to a value of a forced attribute or an arbitrary attribute value. If the portion where the information of the external device is inserted is set as the forced attribute value, the terminal device 100 can reproduce the moving image from the point of time when the moving image is temporarily stopped until the information of the received first external device corresponds to the information of the inserted external device . That is, when the inserted portion of the external device is set as the forced attribute value and the inserted portion is temporarily stopped, when the received operation information of the first external device corresponds to the inserted operation information of the external device, The moving part of the moving picture can be moved to the next part of the moving part.

However, if the inserted part of the external device is set to an arbitrary attribute value, the terminal device 100 can start the moving image from the point of time when the moving image is temporarily stopped even if the information of the received first external device does not correspond to the information of the inserted external device Can be reproduced. The terminal device 100 can temporarily stop the reproduction of the moving picture and then reproduce it again. Alternatively, the terminal device 100 can maintain the reproduction state without temporarily stopping the moving picture even when reproducing the portion in which the information of the external device is inserted. At this time, the terminal device 100 may reproduce the moving image at a slow speed or may only display the notification.

11 is a view for explaining another embodiment of a moving picture that interacts with the operation of an external apparatus.

Referring to Fig. 11 (a), a saucepan 202b which is the first external device is used. For example, the terminal device 100 can control the moving picture when the motion information of the inserted glass is detected while reproducing the moving picture. In one embodiment, the terminal device 100 can temporarily stop the reproduction of the moving image. The terminal device 100 can stop the reproduction of the moving picture until it receives the operation information from the associated first external device. When the moving picture is stopped at the part where the glass is used, the user can use the saucepan 202b.

The spice bottle 202b may include a sensor module and a communication module as one device. When the saucepot 202b senses the movement, the operation information can be transmitted to the terminal device 100. [ The terminal device 100 can receive the operation information of the sauce container 202b and compare it with the operation information of the glass inserted in the moving image. Since the glass (external device) and the saucepot 202b (first external device) in the moving picture are different devices, the terminal device 100 does not correspond to the operation information. As described above, the portion of the moving picture where the information of the external apparatus is inserted can be set as the forced attribute value or the arbitrary attribute value. The terminal device 100 can control the reproduction of the moving picture according to the set property value.

Referring to FIG. 11 (b), a moving picture that is controlled when the information of the inserted external device corresponds to the information of the received first external device is shown. If the portion of the moving image where the glass is inserted is set as the forced attribute value, the operation information of the glass and the operation information of the seasoning container 202b do not correspond to each other, so that the terminal device 100 can maintain the moving image in a stopped state. That is, when the selection signal of the first external device is received and the currently reproduced portion of the moving image is set as the forced attribute value, the terminal device 100 can maintain the portion currently reproduced.

If the portion of the moving image where the glass is inserted is set to an arbitrary attribute value, the operation information of the glass and the operation information of the seasoning container 202b do not correspond, so that the terminal device 100 can move the moving portion of the moving image. The terminal device 100 can move the reproduction part to a part corresponding to the operation information of the seasoning container 202b in the moving picture. That is, when the selection signal of the first external device is received and the currently reproduced portion of the moving image is set to an arbitrary attribute value, the terminal device 100 receives the operation information of the photographed external device related to the first external device The moving part of the moving image can be moved.

12 is a timing diagram illustrating a process of generating a moving image according to an embodiment.

Referring to FIG. 12, the terminal device 100 can detect an external device by photographing an external device connected to the network (S1210). The terminal device 100 can recognize the presence of an external device in the photographed image through the image processing process.

The external device 200-1 connected to the network can sense the sensing information (S1220). The external device 200-1 may include identification information, unique information, appearance information, authority information, and the like, and may detect motion of the external device 200-1 to obtain motion information and the like.

The terminal device 100 can request information of the external device from the external device 200-1 (S1230). The terminal device 100 can transmit information of the recognized external device to the external devices in the vicinity. For example, the information of the recognized external device may be external information.

The external device 200-1 can transmit the information of the external device to the terminal device 100 (S1240). The external device 200-1 can transmit its own information to the terminal device 100 when the information received from the terminal device 100 matches its own information.

The terminal device 100 can insert the information of the received external device and generate a moving image (S1250). The information of the received external device may include operation information. The terminal apparatus 100 can insert the information of the received external apparatus into a frame where the motion of the photographed external apparatus starts and insert the information into the frame in which the motion of the photographed external apparatus is terminated. Alternatively, the terminal device 100 may insert a bookmark in a frame in which the motion of the photographed external device starts, and insert information of the received external device in a frame in which the motion of the photographed external device is terminated.

13 is a timing chart for explaining a process of controlling a moving picture according to an embodiment.

Referring to FIG. 13, the terminal device 100 may display a moving image in which information of an external device is inserted (S1310). The terminal device 100 can detect the information of the inserted external device (S1320). Then, the terminal device 100 can recognize the surrounding external device when the moving image is reproduced. The terminal device 100 can control the moving picture in the portion where the information of the external apparatus is inserted during the reproduction of the moving picture. For example, the terminal device 100 may suspend the reproduction of the moving image. The portion where the information of the external apparatus is inserted may include an attribute value. The attribute value may be a forced attribute value or an arbitrary attribute value.

The terminal device 100 may request information of the first external device related to the external device (S1330). The first external device 200-1 can transmit information of the first external device (S1340). The information of the first external device may include operation information. When the information of the first external apparatus received corresponds to the information of the inserted external apparatus, the terminal apparatus 100 can control reproduction of the moving picture. For example, the terminal device 100 may suspend the reproduction of the moving image.

For example, if the portion of the external device including the embedded information includes the forced attribute value and the received information of the first external device corresponds to the information of the inserted external device, the terminal device 100 can reproduce the video normally have. If the portion of the external device including the embedded information includes the forced attribute value and the received information of the first external device does not correspond to the information of the inserted external device, the terminal device 100 can maintain the video still state .

If the portion where the information of the external apparatus is inserted includes an arbitrary attribute value and the information of the received first external apparatus does not correspond to the information of the inserted external apparatus, the terminal apparatus 100 transmits the information of the received first external apparatus The moving part of the moving image can be moved to the moving part of the external device included in the moving image related to the moving image.

FIG. 14 is a flowchart of a terminal device control method according to an embodiment.

Referring to FIG. 14, the terminal device may display a moving image in which an external device connected to the network is photographed and embedded with operation information of an external device detected by the external device (S1410). The terminal device can detect the operation information of the inserted external device during the reproduction of the moving image. In one embodiment, the terminal device may suspend the reproduction of the moving picture in the frame in which the motion information of the moving picture external device is inserted. The terminal device can receive the operation information of the first external device associated with the external device (S1420). When the operation information of the received first external apparatus corresponds to the operation information of the inserted external apparatus, the terminal apparatus can control the reproduction of the moving image (S1430).

In one embodiment, if there is no first external device associated with an external device within a predetermined range of the terminal device, the terminal device can maintain playback of the video regardless of insertion of operation information of the external device.

The portion where the operation information of the external device of the moving picture is inserted can be set as the forced attribute value or the arbitrary attribute value. The portion where the motion information of the external device of the moving image is inserted is set as the forced attribute value and the terminal device can maintain the stop of the moving image if the operation information of the received first external device does not correspond to the operation information of the inserted external device .

When the operation information of the external device of the moving picture is inserted into the arbitrary attribute value and the operation information of the second external device other than the first external device is received, The playback point of the moving image can be moved to the portion where the operation information of the external device is inserted.

The terminal device may transmit a notification signal to the first external device when the terminal device reproduces the frame in which the motion information of the external device of the moving image is inserted.

The terminal device and the control method according to the present invention are not limited to the configuration and the method of the embodiments described above but may be variously modified in whole or in part by selectively combining the embodiments.

Meanwhile, the control method of the terminal device of the present invention can be implemented as software on a recording medium readable by a processor included in the terminal device. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It should be understood that various modifications may be made by those skilled in the art, and such modifications are not to be understood individually from the technical idea or viewpoint of the specification.

DETAILED DESCRIPTION OF THE INVENTION

-

INDUSTRIAL APPLICABILITY The present invention has industrial applicability that can be used in a terminal device and has repeatability.

Claims (20)

A display unit for displaying a moving image captured by an external device;
A communication unit for transmitting and receiving signals through a network; And
And a controller for detecting information during reproduction of the moving picture,
The external device includes:
A sensor unit for sensing information, connected to a network to transmit the sensed information,
The moving picture includes:
Wherein the information includes motion information of an external device detected and transmitted by the external device and inserted by the moving image production device,
Wherein,
Wherein the operation information of the inserted external apparatus is detected during the reproduction of the moving picture, the operation information of the first external apparatus related to the external apparatus is received, And controls reproduction of the moving picture when it corresponds to the operation information. The method according to claim 1,
Wherein the portion where the operation information of the external device is inserted is set as a forced attribute value or an arbitrary attribute value. 3. The method of claim 2,
Wherein,
When the operation information of the external device is inserted, and when the operation information of the received external device is set to the forced attribute value and the portion is reproduced, when the operation information of the received external device corresponds to the operation information of the inserted external device, And moves the reproduction portion of the moving picture to the next portion of the reproduced portion. 3. The method of claim 2,
Wherein,
When the selection signal of the second external device is received and the currently reproduced portion of the moving image is set to the arbitrary attribute value, And the terminal is moved. 3. The method of claim 2,
Wherein,
Wherein when the selection signal of the second external device is received and the currently reproduced portion of the moving image is set as the forced attribute value, the terminal device maintains the currently reproduced portion. The method according to claim 1,
Wherein the moving image includes a plurality of photographed external apparatuses and a plurality of portions into which operation information of the plurality of external apparatuses is inserted. The method according to claim 1,
Wherein,
And suspends reproduction of the moving picture when the first external apparatus associated with the external apparatus included in the moving picture exists. 8. The method of claim 7,
Wherein,
And reproduces the paused moving picture from the stopped point of time when the received operation information of the first external apparatus corresponds to the operation information of the inserted external apparatus. 8. The method of claim 7,
Wherein,
Wherein the controller determines whether the first external apparatus is present by receiving information of the first external apparatus within a predetermined distance when starting playback of the moving picture. The method according to claim 1,
Wherein,
Wherein the moving picture is continuously reproduced when there is no first external device associated with the external device included in the moving picture. The method according to claim 1,
The first external device includes:
And type information of the same attribute as the captured external device. The method according to claim 1,
Wherein the portion including the operation image of the external apparatus includes a plurality of frames,
Wherein the operation information of the external device is inserted into the last frame of the plurality of frames. The method according to claim 1,
And a camera unit for photographing an external device connected to the network,
Wherein,
Wherein the controller receives the operation information of the external apparatus when the operation of the external apparatus is detected and inserts the operation information of the received external apparatus into a portion including the operation image of the external apparatus to generate a moving image, Terminal device. The method according to claim 1,
Wherein,
Wherein the controller controls the playback of the moving picture by further considering size information of the external apparatus and size information of the first external apparatus. The method according to claim 1,
The moving picture includes:
Wherein additional information of the external apparatus is further inserted in a portion where operation information of the external apparatus is inserted. 16. The method of claim 15,
The additional information of the external device may include:
Wherein the terminal information includes at least one of unique information, location information, authority information, type information, size information, component information, and temperature information. 16. The method of claim 15,
Wherein,
And controls the display unit to display at least one of additional information and operation information of the inserted external apparatus. The method according to claim 1,
Wherein,
And controls the communication unit to transmit the notification signal to the first external apparatus when the first external apparatus associated with the external apparatus is detected while the operation information of the inserted external apparatus is detected during the reproduction of the moving picture . A method of controlling a moving picture display apparatus, the method comprising: displaying a moving picture in which an external device connected to a network is photographed and included and operation information of an external device sensed by the sensor unit of the external device is inserted; A terminal device for requesting operation information of a first external device associated with an external device; And
And a first external device for transmitting operation information at the request of the terminal device,
The terminal apparatus comprises:
And controls reproduction of the moving picture when the operation information of the first external apparatus is associated with the operation information of the inserted external apparatus. Displaying a moving image in which an external device connected to the network is photographed and included and operation information of an external device sensed by the external device is inserted;
Detecting operation information of the inserted external device during playback of the moving image and receiving operation information of a first external device associated with the external device; And
And controlling reproduction of the moving picture when the received operation information of the first external apparatus corresponds to operation information of the inserted external apparatus.

Images