KR102309451B1 - Wearable Device and Control Method of Displaying on the Device Thereof - Google Patents

Abstract

Disclosed is a method for controlling a display in smart glasses. Specifically, the method for controlling the display may include performing pairing with a set-top box; recognizing an external display that reproduces the first type image transmitted from the set-top box; and receiving a second type image from the set-top box, wherein the second type image is an extended image of the first type image.

Description

Wearable Device and Control Method of Displaying on the Device Thereof

The present invention relates to a wearable device. In particular, the present invention relates to a method of controlling to display information desired by a user on the wearable device.

In accordance with the recent trend of weight reduction and miniaturization of digital devices, more various wearable devices are being developed. The wearable device is used as a generic term for all digital devices that are worn on a user's head, arm, eye, etc. to provide various information to the user. For example, the wearable device is a 'smart glass' type worn on the head (eg, head-mounted display (HMD, Head Mount Display), G-Glass, etc.), worn on the arm It is being developed in a variety of ways and types, such as a 'smart watch' type, which is a 'smart watch' type, and a 'Samrt Lens' type that is inserted into the eye like a contact lens.

In addition, most wearable devices are provided with a display means on one side to provide various information to the user as described above. The display means is combined with, for example, augmented reality technology (AR, Argumented Reality), N-screen technology, and the like to conveniently provide information desired by a user. In this regard, an object of the present invention is to provide more meaningful information to a user using the various types of wearable devices.

An object of the present invention is to provide information so that a user can receive more vivid information by using a display means of a wearable device.

An object of the present invention is to recognize a motion and a gesture of a wearable device wearer and provide information suitable therefor.

An object of the present invention is to provide more interesting content through pairing of a display means of a wearable device with another display means.

Another object of the present invention is to provide a wearable device that satisfies at least one of the above objects.

A method of controlling a display in smart glasses according to an aspect of the present invention includes: performing pairing with a set-top box; recognizing an external display that reproduces the first type image transmitted from the set-top box; and receiving a second type image from the set-top box, wherein the second type image is an extended image of the first type image.

Here, the method for controlling a display in smart glasses, which is an aspect of the present invention, may further include performing calibration in consideration of at least one of a distance from the external display, an angle, and location information.

The step of recognizing the external display may include determining whether a preset portion of the external display is within a preset angle of view of the smart glasses.

In addition, when at least one of the preset parts of the external display is located outside the preset angle of view of the smart glasses while the second type image is being reproduced, the second type image is turned off.

Meanwhile, when the user of the smart glasses moves, a distance between the smart glasses and the external display may be measured, and the size of the second type image may be adjusted based on the measured distance and a result of the calibration.

Alternatively, when the user of the smart glasses moves, an angle between the external displays may be measured, and the angle or depth of the second type image may be adjusted based on the measured angle and a result of the calibration.

Here, the first type image and the second type image are images captured using a dual camera, respectively.

Furthermore, the first type image and the second type image may be images captured using a dual camera, respectively.

Another aspect of the present invention is a method for controlling a display in a set-top box coupled to an external display, comprising: receiving a first type image and a second type image from a broadcast management server; performing pairing with smart glasses; displaying the first type image on the external display; and transmitting the second type image to the smart glasses, wherein the second type image is an expanded image of the first type image.

Here, the step of transmitting the second type image to the smart glasses may be performed based on information about calibration with the smart glasses.

The information on the calibration may be generated based on a distance between the smart glasses and the set-top box.

The transmitting of the second type image to the smart glasses may be performed when the smart glasses recognize an external display coupled to the set-top box.

In addition, the recognition of the external display may be determined based on whether a preset portion of the external display is within a preset angle of view of the smart glasses.

When the user of the smart glasses moves, the size of the second type image may be adjusted based on the measured distance and the calibration result by measuring a distance between the smart glasses and the external display.

When the user of the smart glasses moves, the size of the second type image may be adjusted based on the measured distance and the calibration result by measuring a distance between the smart glasses and the external display.

The first type image and the second type image are images captured by using a dual camera, respectively. Also, the second type image may be an image captured using a depth camera.

Smart glasses for performing a method for controlling a display, which is another aspect of the present invention, includes a processor that performs pairing with a set-top box and recognizes an external display that reproduces a first type image transmitted from the set-top box, , a transceiver module for receiving a second type image from the set-top box, wherein the second type image is an extended image of the first type image.

A set-top box coupled to an external display for performing a display control method according to another aspect of the present invention includes a transmission/reception module for receiving a first type image and a second type image from a broadcast management server; and a processor configured to perform pairing with smart glasses and to display the first type image on the external display, wherein the processor controls the transceiver module to transmit the second type image to the smart glasses; The second type image is an extended image of the first type image.

On the other hand, in another aspect of the present invention, a program for performing a method of controlling a display in smart glasses is recorded in a computer readable recording medium, the program comprising: performing pairing with a set-top box; recognizing an external display that reproduces the first type image transmitted from the set-top box; and receiving a second type image from the set-top box, wherein the second type image is an extended image of the first type image.

In another aspect of the present invention, a program for performing a method of controlling a display in a set-top box coupled to an external display, which is another aspect of the present invention, is recorded in a computer readable recording medium, wherein the program is provided with a first type image and a second type from a broadcast management server. receiving a type image; performing pairing with smart glasses; displaying the first type image on the external display; and transmitting the second type image to the smart glasses, wherein the second type image is an extended image of the first type image.

According to the present invention, the user can be provided with more vivid information by using the display means of the wearable device.

In addition, according to the present invention, the display means of the wearable device can provide more interesting content through pairing with another display means.

In addition, by operating independently of the display means of another wearable device, content specialized for each user may be provided.

1 is a view for explaining the background technology for understanding the present invention.
2 is a diagram for explaining a system to which an embodiment of the present invention is applied.
3 is a diagram illustrating an internal configuration of a broadcast receiving apparatus applicable to the present invention.
4 is a reference diagram illustrating the appearance of smart glass as an example of a wearable device applicable to the present invention.
5 is a diagram illustrating an internal configuration of a wearable device as an example of a wearable device applicable to the present invention.
6 is a diagram for explaining a concept applied to the present invention.
7 is a view for explaining an external display recognition operation of the wearable device according to the present invention.
8 is a flowchart applied to the present invention.
9 is a detailed flowchart applied to the present invention.
10 is a flowchart illustrating an interaction between an external device and a wearable device according to an embodiment applied to the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "module" and "part" for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have a meaning or role distinct from each other by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as "comprises" or "have" are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In addition, the terms used in the present specification have been selected as currently widely used general terms as possible while considering functions in the present invention, which may vary depending on the intention, custom, or emergence of new technology of those of ordinary skill in the art. Also, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the relevant invention. Therefore, it is intended to clarify that the terms used in the present specification should be interpreted based on the actual meaning and the overall content of the present specification, rather than the names of simple terms. In particular, it is intended to clarify that the term 'wearable device' used in the present invention is used to refer to all types of devices that can be attached or detached to a user's body rather than simply referring to a specific product. Therefore, in the present invention, in order to facilitate understanding of the present invention, even if there is a part of the 'wearable device' that is mainly described with respect to a specific type (eg, smart glasses type), the specific operation or function of the corresponding type is excluded. It should be understood that all inventive concepts are equally applicable to other types of wearable devices. Hereinafter, the present invention will be described by taking the smart glasses type among the 'wearable devices' as an example, but it will be obvious that the contents of the present invention are equally applicable to the smart lens type.

1A to 1B are diagrams for explaining background technology for understanding the present invention. Peripheral vision is due to the function of rods distributed throughout the retina except for the macula. Since many rods are arranged as a single bipolar cell, the function in cows is excellent, but the decomposition ability such as two-point discrimination ability is not good. That is, as shown in FIG. 1A , the peripheral vision sees objects as blurred, but the power to capture motion is strong. Accordingly, it is an object of the present invention to provide peripheral vision contents to smart glasses to increase the presence of an image.

FIG. 1B is a case in which the peripheral vision image is displayed on smart glasses and a plurality of viewers simultaneously view the peripheral vision image with respect to an image reproduced on the main display device or an external device in a general home or public place. For example, while viewing the main image (or main image), it is possible to provide the extended content in which snow falls or the extended content in which rain falls as a peripheral vision image related to the main image. Hereinafter, in the description of the present invention, the image provided by the smart glasses is referred to as a second type image, and the image reproduced by the external device is referred to as a first type image. As described above, the second type image may include a peripheral vision image or an expanded image, which is an image processed so as to feel a sense of vitality using peripheral vision as described above. Here, the peripheral vision image or the extended image is an image having additional information with respect to the main image. For example, the peripheral vision image or the extended image may be augmented reality. Augmented reality is a technology that superimposes virtual objects on the real world that users see with their eyes. It is also called mixed reality (MR) because the real world is combined with the virtual world with additional information in real time and shown as a single image.

That is, the second type image (eg, augmented reality) is a concept that complements the first type image (eg, the real world). This is to make the distinction between the real environment and the additional screen (eg, virtual screen) ambiguous by overlapping the 3D virtual image or the additional image on the actual image viewed by the user. On the other hand, “extended image” and “peripheral view image” are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. Hereinafter, specific methods for carrying out the present invention will be described.

2 is a diagram for explaining a system in which the present invention is performed.

A system in which the present invention is performed may include a wearable device 100 , a broadcast receiving apparatus 200 , and a broadcast management server 300 .

Specifically, the wearable device 100 for performing an embodiment of the present invention may receive and reproduce a peripheral vision image (or extended content) in relation to the main display device 250 that reproduces the main image. The wearable device 100 may communicate with the broadcast reception device 200 using wired/wireless communication and receive the peripheral vision image from the broadcast reception device 200 .

The main display device 250 for performing an embodiment of the present invention may receive and reproduce a main image (or a basic image) under the control of the broadcast reception device 200 in relation to the wearable device 100 . . The main display device 150 may be configured integrally with the broadcast reception device 200 or may be configured separately.

The broadcast reception device 200 for performing an embodiment of the present invention may receive the main image and/or the peripheral view image from the broadcast management server 300 . In addition, each received image may be reproduced using the main display apparatus 250 or transmitted to the wearable device 100 . Also, the broadcast reception device 200 may include the main display device 250 or the main display device 250 may be configured separately.

The broadcast management server 300 may refer to a server that generates and transmits a broadcast signal and/or broadcast-related information, or a server that receives and transmits a previously generated broadcast signal and/or broadcast-related information to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal. Here, the broadcast management server 300 applied to the present invention generates an image to be transmitted to the dual camera. The two types of synchronized video are processed by the broadcasting station. That is, the operation of excluding the general image part from the extended image is performed. Also, the two types of images are synchronized and transmitted to the broadcast receiving device 200 .

That is, the broadcast management server 300 may use augmented reality (AR) to capture an extended image to be provided to the surrounding city. Here, a depth camera may be used as a camera for capturing an extended image among the dual cameras. A depth camera is a camera in which a pixel value of a captured image is proportional to the distance between points of an object corresponding to the pixel in the camera. If the depth camera is used, it is also possible to track and analyze only the shape regardless of the color of the first camera.

The broadcast reception device 200 includes a wireless communication unit 210 , an input unit 220 , a sensing unit 240 , an output unit 250 , an interface unit 260 , a memory 270 , a control unit 280 and a power supply unit ( 290) and the like. The components shown in FIG. 2 are not essential for implementing the broadcast reception device, so the broadcast reception device described in this specification may have more or fewer components than those listed above.

More specifically, among the above components, the wireless communication unit 210 may be configured between the broadcast reception device 200 and the wireless communication system, between the broadcast reception device 200 and another broadcast reception device 100 , or the broadcast reception device 200 . ) and one or more modules that enable wireless communication between the external server. Also, the wireless communication unit 210 may include one or more modules for connecting the broadcast reception device 200 to one or more networks.

The wireless communication unit 210 may include at least one of a broadcast reception module 211 , a mobile communication module 212 , a wireless Internet module 213 , a short-range communication module 214 , and a location information module 215 . .

The input unit 220 includes a camera 221 or an image input unit for inputting an image signal, a microphone 222 or an audio input unit for inputting an audio signal, and a user input unit 223 for receiving information from a user, for example, , a touch key, a push key, etc.). The voice data or image data collected by the input unit 220 may be analyzed and processed as a user's control command.

The sensing unit 240 may include one or more sensors for sensing at least one of information in the broadcast reception device, surrounding environment information surrounding the broadcast reception device, and user information. For example, the sensing unit 240 may include a proximity sensor 241, an illumination sensor 242, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. Sensor (G-sensor), gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint sensor (finger scan sensor), ultrasonic sensor , optical sensors (eg, cameras (see 221)), microphones (see 222), battery gauges, environmental sensors (eg, barometers, hygrometers, thermometers, radiation detection sensors, It may include at least one of a thermal sensor, a gas sensor, etc.) and a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the broadcast reception device disclosed in the present specification may combine and utilize information sensed by at least two or more of these sensors.

The output unit 250 is for generating an output related to visual, auditory or tactile sense, and includes at least one of a display unit 251 , a sound output unit 252 , a haptip module 253 , and an optical output unit 254 . can do. The display unit 151 may implement a touch screen by forming a layer structure with the touch sensor or being formed integrally with the touch sensor. Such a touch screen may function as a user input unit 223 that provides an input interface between the broadcast reception device 200 and a user, and may provide an output interface between the broadcast reception device 200 and a user.

The interface unit 260 serves as a passage with various types of external devices connected to the broadcast reception device 200 . This interface unit 160, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, for connecting a device equipped with an identification module It may include at least one of a port, an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port. In response to the connection of the external device to the interface unit 160 , the broadcast reception device 200 may perform appropriate control related to the connected external device.

Also, the memory 270 stores data supporting various functions of the broadcast reception device 200 . The memory 270 may store a plurality of application programs (or applications) driven in the broadcast reception device 200 , data for operation of the broadcast reception device 200 , and commands. At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some of these application programs may exist on the broadcast reception device 200 from the time of shipment for a basic function of the broadcast reception device 200 . Meanwhile, the application program may be stored in the memory 270 , installed on the broadcast reception device 200 , and driven to perform an operation (or function) of the broadcast reception device by the controller 280 .

In addition to the operation related to the application program, the controller 280 generally controls the overall operation of the broadcast reception device 200 . The controller 280 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 270 .

In addition, the controller 280 may control at least some of the components discussed with reference to FIG. 1A in order to drive an application program stored in the memory 270 . Furthermore, in order to drive the application program, the controller 280 may operate at least two or more of the components included in the broadcast reception device 200 in combination with each other.

The power supply unit 290 receives external power and internal power under the control of the control unit 280 to supply power to each component included in the broadcast reception device 200 . The power supply unit 290 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the respective components may operate in cooperation with each other in order to implement an operation, control, or control method of a broadcast reception device according to various embodiments to be described below. Also, the operation, control, or control method of the broadcast reception device may be implemented on the broadcast reception device by driving at least one application program stored in the memory 270 .

Hereinafter, before looking at various embodiments implemented through the broadcast receiving device 200 as described above, the above-listed components will be described in more detail with reference to FIG. 1A .

First, referring to the wireless communication unit 210 , the broadcast reception module 211 of the wireless communication unit 210 receives a broadcast signal and/or broadcast-related information from an external broadcast management server 300 through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more of the broadcast reception modules may be provided to the broadcast reception device 200 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server 300 may mean a server that generates and transmits a broadcast signal and/or broadcast-related information or a server that receives a previously generated broadcast signal and/or broadcast-related information and transmits the received broadcast signal and/or broadcast-related information to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal.

The broadcast signal may be encoded according to at least one of technical standards (or a broadcast method, for example, ISO, IEC, DVB, ATSC, etc.) for transmitting and receiving digital broadcast signals, and the broadcast reception module 211 is configured to The digital broadcast signal may be received using a method suitable for technical standards set by technical standards.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may be provided through a mobile communication network. In this case, it may be received by the mobile communication module 212 .

The broadcast-related information may exist in various forms, for example, an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H). A broadcast signal and/or broadcast related information received through the broadcast reception module 211 may be stored in the memory 260 .

The mobile communication module 212 is a technical standard or communication method for mobile communication (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced, etc.) transmits/receives a radio signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to (Long Term Evolution-Advanced).

The wireless signal may include various types of data according to transmission/reception of a voice call signal, a video call signal, or a text/multimedia message.

The wireless Internet module 213 refers to a module for wireless Internet access, and may be built-in or external to the broadcast reception device 200 . The wireless Internet module 213 is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

As wireless Internet technologies, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), etc., and the wireless Internet module ( 113) transmits and receives data according to at least one wireless Internet technology within a range including Internet technologies not listed above.

From the point of view that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, the wireless Internet module 213 for performing wireless Internet access through the mobile communication network. ) may be understood as a type of the mobile communication module 212 .

Short-range communication module 214 is for short-range communication, Bluetooth (Bluetooth), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association; IrDA), UWB (Ultra Wideband), ZigBee, NFC ( Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology may be used to support short-distance communication. The short-range communication module 214, between the broadcast receiving device 200 and the wireless communication system, between the broadcast receiving device 200 and another broadcast receiving device 200, or through a wireless local area network (Wireless Area Networks), or broadcast It is possible to support wireless communication between the receiving device 200 and a network in which another broadcast receiving device 100 or an external server is located. The local area network may be a local area network (Wireless Personal Area Networks).

Here, another broadcast reception device 100 is a wearable device capable of exchanging (or interworking) data with the broadcast reception device 200 according to the present invention, for example, a smart watch, It may be a smart glass (smart glass) or a head mounted display (HMD). The short-range communication module 214 may detect (or recognize) a wearable device capable of communicating with the broadcast reception device 200 in the vicinity of the broadcast reception device 200 . Furthermore, when the detected wearable device is a device authenticated to communicate with the broadcast reception apparatus 200 according to the present invention, the controller 280 transmits at least a portion of data processed by the broadcast reception apparatus 200 to the short-range communication. It can be transmitted to the wearable device through the module 214 . Accordingly, the user of the wearable device may use data processed by the broadcast reception apparatus 200 through the wearable device. For example, according to this, when a call is received by the broadcast receiving apparatus 200, the user performs a phone call through the wearable device, or when a message is received by the broadcast receiving apparatus 200, the user through the wearable device It is possible to confirm the received message.

The location information module 215 is a module for acquiring the location (or current location) of the broadcast reception device, and a representative example thereof includes a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the broadcast reception device utilizes the GPS module, it may acquire the location of the broadcast reception device by using a signal transmitted from a GPS satellite. As another example, if the broadcast receiving device utilizes the Wi-Fi module, it can acquire the location of the broadcast receiving device based on the Wi-Fi module and the information of the wireless AP (Wireless Access Point) that transmits or receives the wireless signal. have. If necessary, the location information module 215 may perform any function of the other modules of the wireless communication unit 210 in order to obtain data on the location of the broadcast receiving device as a substitute or additionally. The location information module 215 is a module used to obtain the location (or current location) of the broadcast reception device, and is not limited to a module that directly calculates or obtains the location of the broadcast reception device.

Next, the input unit 220 is for input of image information (or signal), audio information (or signal), data, or information input from a user. For input of image information, the broadcast receiving device 200 is One or a plurality of cameras 121 may be provided. The camera 221 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 251 or stored in the memory 270 . On the other hand, the plurality of cameras 221 provided in the broadcast reception device 200 may be arranged to form a matrix structure, and through the cameras 221 forming the matrix structure as described above, the broadcast reception device 200 may be provided at various angles or A plurality of image information having a focus may be input. In addition, the plurality of cameras 221 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 222 processes an external sound signal as electrical voice data. The processed voice data may be variously utilized according to a function (or a running application program) being performed by the broadcast receiving device 200 . Meanwhile, various noise removal algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 222 .

The user input unit 223 is for receiving information from a user, and when information is input through the user input unit 223 , the control unit 280 may control the operation of the broadcast reception device 200 to correspond to the input information. have. The user input unit 223 is a mechanical input means (or a mechanical key, for example, a button located on the front, rear or side of the broadcast receiving device 200 , a dome switch, and a jog wheel). , jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, a soft key, or a visual key displayed on the touch screen through software processing, or a part other than the touch screen. On the other hand, the virtual key or the visual key may be displayed on the touch screen while having various shapes, for example, a graphic, text ), an icon, a video, or a combination thereof.

Meanwhile, the sensing unit 240 senses at least one of information in the broadcast receiving device, surrounding environment information surrounding the broadcast receiving device, and user information, and generates a sensing signal corresponding thereto. The controller 280 may control the driving or operation of the broadcast reception device 200 or perform data processing, functions, or operations related to an application program installed in the broadcast reception device 200 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 240 will be described in more detail.

First, the proximity sensor 241 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. As the proximity sensor 241 , the proximity sensor 241 may be disposed in an inner region of the broadcast receiving device covered by the touch screen described above or in the vicinity of the touch screen.

Examples of the proximity sensor 241 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like. In the case where the touch screen is capacitive, the proximity sensor 241 may be configured to detect the proximity of an object having conductivity as a change in an electric field according to the proximity of the object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, the act of approaching an object on the touch screen without contacting it so that the object is recognized that it is located on the touch screen is called “proximity touch”, and the touch The act of actually touching an object on the screen is called "contact touch". The position where the object is touched in proximity on the touch screen means a position where the object is perpendicular to the touch screen when the object is touched in proximity. The proximity sensor 241 may detect a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.) have. On the other hand, the controller 280 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern detected through the proximity sensor 241 as described above, and further, provides visual information corresponding to the processed data. It can be printed on the touch screen. Furthermore, the controller 280 may control the broadcast receiving device 200 to process different operations or data (or information) according to whether a touch to the same point on the touch screen is a proximity touch or a contact touch. have.

The touch sensor receives a touch (or touch input) applied to the touch screen (or the display unit 251) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. detect

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific part of the touch screen or a change in capacitance occurring in a specific part of the touch screen into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of a touch, an electrostatic capacitance at the time of a touch, etc. in which a touch object applying a touch on the touch screen is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen or a stylus pen, a pointer, or the like.

As such, when there is a touch input to the touch sensor, a signal(s) corresponding thereto is sent to the touch controller. The touch controller processes the signal(s) and then transmits the corresponding data to the controller 280 . Accordingly, the controller 280 can know which area of the display unit 251 has been touched, and the like. Here, the touch controller may be a component separate from the controller 280 , or may be the controller 280 itself.

Meanwhile, the controller 280 may perform different controls or may perform the same control according to the type of the touch object that touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of the touch object may be determined according to the current operating state of the broadcast reception device 200 or the running application program.

On the other hand, the above-described touch sensor and proximity sensor are independently or in combination, a short (or tap) touch on the touch screen (short touch), long touch (long touch), multi touch (multi touch), drag touch (drag touch) ), flick touch, pinch-in touch, pinch-out touch, swype touch, hovering touch, etc. It can sense touch.

The ultrasonic sensor may recognize location information of a sensing target by using ultrasonic waves. Meanwhile, the control unit 280 may calculate the position of the wave source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source may be calculated using the property that light is much faster than ultrasonic waves, that is, the time for light to reach the optical sensor is much faster than the time for ultrasonic waves to reach the ultrasonic sensor. More specifically, the position of the wave source may be calculated using a time difference between the time that the ultrasonic wave arrives using light as the reference signal.

On the other hand, the camera 221 as seen in the configuration of the input unit 220 includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 221 and the laser sensor may be combined with each other to detect a touch of a sensing target for a 3D stereoscopic image. The photo sensor may be stacked on the display device, and the photo sensor is configured to scan the movement of the sensing target close to the touch screen. More specifically, the photo sensor mounts photo diodes and transistors (TRs) in rows/columns and scans the contents placed on the photo sensor using electrical signals that change according to the amount of light applied to the photo diodes. That is, the photo sensor calculates the coordinates of the sensing target according to the amount of change in light, and through this, location information of the sensing target can be obtained.

The display unit 251 displays (outputs) information processed by the broadcast reception device 200 . For example, the display unit 251 may display execution screen information of an application program driven in the broadcast reception device 200, or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information. have.

Also, the display unit 251 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

A three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glasses-free method), or a projection method (holographic method) may be applied to the stereoscopic display unit.

In general, a 3D stereoscopic image consists of a left image (image for left eye) and a right image (image for right eye). Depending on the method in which the left and right images are merged into a 3D stereoscopic image, a top-down method in which the left and right images are arranged up and down in one frame, and the left and right images in one frame L-to-R (left-to-right, side by side) method, which arranges the pieces of the left and right images in a tile form, the checker board method, which arranges the left and right images in columns Alternatively, it is divided into an interlaced method in which a row is alternately arranged, and a time sequential (frame by frame) method in which a left image and a right image are displayed alternately by time.

In addition, the 3D thumbnail image may be generated as a single image by generating a left image thumbnail and a right image thumbnail from the left image and the right image of the original image frame, respectively, and combining them. In general, a thumbnail means a reduced image or a reduced still image. The generated left image thumbnail and right image thumbnail may be displayed with a left and right distance difference on the screen by a depth corresponding to the parallax between the left image and the right image, thereby representing a three-dimensional sense of space.

The left image and the right image necessary for realizing the 3D stereoscopic image may be displayed on the stereoscopic display unit by the stereoscopic processing unit. The stereoscopic processing unit receives a 3D image (an image of a reference view and an image of an extended view) and sets a left image and a right image therefrom, or receives a 2D image and converts it into a left image and a right image.

The sound output unit 252 may output audio data received from the wireless communication unit 210 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 also outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the broadcast reception device 200 . The sound output unit 252 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 253 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 253 may be vibration. The intensity and pattern of vibration generated by the haptic module 253 may be controlled by a user's selection or setting of the controller. For example, the haptic module 253 may synthesize and output different vibrations or output them sequentially.

In addition to vibration, the haptic module 253 is configured to respond to stimuli such as a pin arrangement that moves vertically with respect to the contact skin surface, a jet or suction force of air through a nozzle or an inlet, a touch on the skin surface, contact of an electrode, an electrostatic force, etc. Various tactile effects can be generated, such as an effect caused by heat absorption and an effect by reproducing a feeling of cold and heat using an element capable of absorbing heat or generating heat.

The haptic module 253 may not only deliver a tactile effect through direct contact, but may also be implemented so that the user can feel the tactile effect through a muscle sense such as a finger or arm. Two or more haptic modules 253 may be provided according to the configuration of the broadcast receiving device 200 .

The light output unit 254 outputs a signal for notifying the occurrence of an event by using the light of the light source of the broadcast receiving device 200 . Examples of the event generated in the broadcast receiving device 200 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the optical output unit 254 is implemented as the broadcast receiving device emits light of a single color or a plurality of colors toward the front or rear. The signal output may be terminated when the broadcast receiving device detects the user's event confirmation.

The interface unit 260 serves as a passage with all external devices connected to the broadcast reception device 200 . The interface unit 260 receives data from an external device, receives power and transmits it to each component inside the broadcast reception device 200 , or transmits data inside the broadcast reception device 200 to an external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port for connecting a device equipped with an identification module (port), an audio I/O (Input/Output) port, a video I/O (Input/Output) port, an earphone port, etc. may be included in the interface unit 160 .

On the other hand, the identification module is a chip storing various information for authenticating the use authority of the broadcast reception device 200, a user identification module (UIM), a subscriber identity module (subscriber identity module; SIM), a general-purpose user It may include a universal subscriber identity module (USIM) and the like. A device equipped with an identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device may be connected to the terminal 200 through the interface unit 260 .

Also, when the broadcast reception device 200 is connected to an external cradle, the interface unit 260 serves as a passage through which power from the cradle is supplied to the broadcast reception device 200, or by a user in the cradle. It may be a path through which input various command signals are transmitted to the broadcast reception device 200 . The various command signals or the power input from the cradle may be operated as signals for recognizing that the broadcast reception device 200 is correctly mounted in the cradle.

The memory 270 may store a program for the operation of the controller 280 , and may temporarily store input/output data (eg, a phone book, a message, a still image, a moving picture, etc.). The memory 270 may store data related to vibration and sound of various patterns output when a touch input on the touch screen is input.

The memory 270 is a flash memory type, a hard disk type, a solid state disk type (SSD), a silicon disk drive type (SDD), and a multimedia card micro type. ), card-type memory (such as SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read (EEPROM) -only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The broadcast reception device 200 may be operated in relation to a web storage that performs a storage function of the memory 270 on the Internet.

Meanwhile, as described above, the controller 280 controls the operation related to the application program and the general operation of the broadcast reception device 200 . For example, when the state of the broadcast receiving device satisfies a set condition, the controller 280 may execute or release a lock state that restricts input of a user's control command to applications.

In addition, the control unit 280 performs control and processing related to voice call, data communication, video call, etc., or performs pattern recognition processing capable of recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively. can Furthermore, the controller 280 may control any one or a plurality of the components described above in order to implement various embodiments described below on the broadcast reception device 200 according to the present invention.

The power supply unit 290 receives external power and internal power under the control of the control unit 280 to supply power required for operation of each component. The power supply unit 290 includes a battery, and the battery may be a built-in battery configured to be rechargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 290 may include a connection port, and the connection port may be configured as an example of the interface 260 to which an external charger that supplies power for charging the battery is electrically connected.

As another example, the power supply unit 290 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply 290 uses one or more of an inductive coupling method based on a magnetic induction phenomenon or a resonance coupling method based on an electromagnetic resonance phenomenon from an external wireless power transmitter. power can be transmitted.

Meanwhile, various embodiments below may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

4 conceptually illustrates the appearance of smart glass as an example of a wearable device applicable to the present invention. As described above, in the present invention, the contents of the present invention will be described using the smart glasses 100 as an example among 'wearable devices', but the contents of the present invention are equally applicable to other wearable devices.

4 is a model of smart glasses that are widely known at present, and is characterized in that it has a form similar to general glasses in appearance, but includes a display unit 101 operating with an optical effect and an electrical signal on one side. Since the configuration and operation principle of the display unit 101 are the same as those of the conventional method, a detailed description thereof will be omitted. If a 'smart watch' or a 'smart lens' is applied as an example of the wearable device of the present invention, it will be obvious that the driving principle of the display means is different. After all, the present invention will be equally applicable to all wearable devices equipped with any type of display means regardless of the type of display means.

5 is a diagram illustrating an internal configuration of a wearable device to which the present invention is applied. For convenience of description, the wearable device will be described using the smart glasses 100 as an example. Referring to FIG. 5 , the smart glasses 100 of the present invention include a display unit 101 , a storage unit 102 , a communication unit 103 , a user control information input unit 104 , a camera 105 , and a sensor unit 106 . , and a motion determination unit 107 and a control unit 110 .

The display means 101 is a device for displaying information, and can be implemented in various ways. In particular, depending on the type of wearable device, LCD, LED, or optical method may be applied. In addition, it is possible to provide the above-described user interaction menu information through the display means 101 .

The storage unit 102 may not only store a software program necessary for driving the smart glasses 100 , but also temporarily store specific information when necessary. In relation to this, according to some embodiments of the present invention, channel information being displayed on an external device is stored in the storage unit 102 or a preference image set by a user is stored in advance. The storage unit 102 may also store various digital data such as video, audio, photo, moving picture, and application. In addition, the terminal specific identifier of the smart glasses 100 may be stored.

The communication unit 103 may connect to a network by wire or wirelessly and transmit/receive data to/from various external devices. For example, according to some embodiments of the present invention, the communication unit 103 is wirelessly connected to the above-described external broadcast reception device 200 to transmit/receive necessary information. In addition, the smart glasses 100 may perform pairing and communication connection with a nearby digital device using the communication unit 103 , and may transmit/receive data to/from the connected digital device. Meanwhile, the communication unit 103 may include a plurality of antennas. The smart glasses 100 may directly receive a broadcast signal from the outside by using the plurality of antennas. Also, the smart glasses 100 may detect a position state of a digital device communicatively connected to the smart glasses 100 by using the plurality of antennas. That is, it is possible to detect the positional relationship between the smart glasses 100 and the connected digital device by using the time difference, phase difference, etc. of signals transmitted/received through the plurality of antennas provided in the smart glasses 100 , respectively. do.

The user control information input unit 104 means a means for receiving a user control command for controlling the use of the smart glasses 100 . The user control command may include a user configuration command. For example, in relation to the embodiment of the present invention, the wearer 10 of the smart glasses 100 may preset a preference image through the user control information input unit 104 . Also, the wearer 10 of the smart glasses 100 may initially set an image display method through the user control information input unit 104 .

The camera 105 is utilized for object recognition and photo taking in front of the smart glasses 100 . The sensor unit 106 performs a function of detecting a movement of an object around the smart glasses 100 . The sensor unit 106 may be configured with a plurality of sensing means, for example, a gravity sensor, a geomagnetic sensor, a motion sensor, a gyro sensor, an acceleration sensor, an infrared sensor, inclination (inclination). ) sensor, brightness sensor, altitude sensor, olfactory sensor, temperature sensor, depth sensor, pressure sensor, banding sensor, audio sensor, video sensor, GPS (Global Positioning System) sensor, including at least one sensing means such as a touch sensor, etc. configurable. However, the present invention is not limited to the above-described sensing means. For example, the sensor unit 106 detects a motion or gesture of the wearer 10 of the smart glasses 100 for editing or processing an image to be displayed according to an embodiment of the present invention and provides the detected result. Suffice.

The motion determining unit 107 receives the sensing detection result of the sensor unit 106 and determines that there has been a meaningful movement when the detected result is greater than or equal to a specific size compared with a preset reference value. Alternatively, even when the detected result is recognized as a specific gesture (eg, picking up an object, etc.), it is determined that there was a meaningful movement. The determination result of the motion determination unit 107 is provided to the control unit 110 to be used as a basis for final determination.

The controller 110 is a main controller of the smart glasses 100, and controls each of the components 101 to 107 of the smart glasses 100 as described above, as well as information and data between each component. It controls transmission and reception. In this regard, the operation process according to some embodiments of the present invention may be implemented based on the function of the control unit 110 . For example, in determining the channel change subject, the control unit 110 considers the sensing result detected from the sensor unit 106 and the motion determination result provided from the motion determination unit 107 to determine the expanded image decide whether to change.

Meanwhile, it should be understood that the internal configuration block diagram of the smart glasses 100 shown in FIG. 5 is illustrated as an example for convenience of description. Accordingly, each of the components shown in FIG. 5 may be configured to be integrated into one block or configured to be configured by separating only a plurality of necessary blocks in an actual design. For example, the motion determination unit 107 is only shown as a separate component for convenience of explanation, but it is obvious that it can be configured in one processor together with the control unit 110 when the actual product is implemented. will do

6 is a diagram for explaining a concept applied to the present invention. In this specification, a case in which the present invention is applied to the smart glasses 100 among the wearable devices will be described in detail. Referring to FIG. 6 , it is assumed that the user A 10 wears the smart glasses 100 and the viewer B 20 does not wear the smart glasses 100 . That is, in FIG. 6 , it is assumed that the user A 10 and the viewer B 20 wearing the smart glasses 100 watch the main image through the external display device 250 in a general home or public place. Also, it is assumed that the user A 10 wearing the smart glasses 100 views the extended image through the pairing of the smart glasses 100 and the broadcast reception device 200 . Hereinafter, as described above, the main image output to an external display device connected to the broadcast reception device is referred to as a first type image. In addition, the extended image or the peripheral vision image output to the smart glasses 100 is referred to as a second type image. Meanwhile, it is assumed that the broadcast reception device 200 of the present invention is a set-top box 200 . In addition, a display device connected to the output unit 250 of the broadcast receiving device 200 will be referred to as an external display device. The external display device may be applied not only to a mobile terminal such as a digital broadcasting terminal, a smart phone, a laptop computer, etc., but also to a fixed terminal such as a digital TV or a desktop computer.

In the above situation, the viewer B 20 has been described as an example of not wearing the smart glasses 100 , but the embodiment of the present invention can be established regardless of whether the viewer B 20 wears the smart glasses 100 . do. Assuming that the viewer B 20 wears the smart glasses 100 , the first type image viewed by the viewer B 20 and the first type image viewed by the user A 10 are the same image, but the user The second type image viewed by A ( 10 ) and viewer B ( 20 ) may be the same image or may be independent images. That is, if the image viewed by the user A 10 is an extended image in which it is raining, the image that the viewer B 20 watches through the smart glasses may be an extended image in which it is snowing. Hereinafter, the characteristic operation and detailed description of the present invention will be described in more detail with reference to FIGS. 7 to 10 .

First, pairing between the smart glasses 100 and the set-top box 200 will be described. Pairing represents a connection for data transmission/reception between the smart glasses 100 and the set-top box 200 . When pairing is performed, the smart glasses 100 and the set-top box 200 perform communication connection, so that data transmission/reception is possible in both directions. Pairing with the set-top box 200 may be performed using the communication unit 170 or the network interface included in the smart glasses 100 . In addition, pairing may be performed through Bluetooth, Near Field Communication (NFC), or the like. For example, pairing may be performed by a user input through the smart glasses 100 or the set-top box 200 . Here, the user input may include a touch input, a voice input, and the like. For example, the smart glasses 100 may provide a separate button or user interface for communication connection with the set-top box 200 . In addition, the user may perform a communication connection between the smart glasses 100 and the set-top box 200 through a user input using a button or a user interface. A communication connection between the smart glasses 100 and the set-top box 200 may be performed by recognizing the wearing operation of the smart glasses 100 .

When a communication connection is performed, the smart glasses 100 may transmit/receive data to/from the set-top box 200 in an open state. Meanwhile, the smart glasses 100 may selectively transmit/receive data among the set-top boxes 200 by performing a communication connection through pairing.

In addition, the smart glasses 100 may perform data communication through pairing with the set-top box 200 to output the same image output from the external display device on the smart glasses 100 . Alternatively, an image related to an image output from the set-top box 200 (eg, a second type image) may be output.

Hereinafter, a procedure for driving the content received from the smart glasses 200 will be described with reference to FIG. 7 . The smart glasses 200 may determine whether to display the second type image (or content) by recognizing whether the external display device is inside the display area. Here, the display area may mean an area capable of displaying the second type image. Alternatively, it may be a preset area corresponding to the field of view of the user wearing the smart glasses 100 . The inside of the display area may be referred to as an angle of view area.

In order to recognize whether the external display device is inside the display area, it is checked whether all corners (7a, 7b, 7c, 7d of FIG. 7 ) of the external display device exist within the field of view of the smart glasses 200 . judge When all corners of the external display device are recognized within the field of view of the smart glasses 200, it is determined that the external display device is within the display area of the smart glasses 100, and the set-top box (eg, set-top box) receives and transmits the second type video from 200 . The second type image may be transmitted after pairing with the smart glasses 200 or may be transmitted as a response to the image transmission request of the smart glasses 200 . Meanwhile, the image transmission request of the smart glasses 200 may be transmitted when the smart glasses 200 recognize the external display device within the angle of view area for a preset time.

On the other hand, when the preset number of corners is out of the field of view of the smart glasses 200, it is determined that the external display device is no longer present in the display area of the smart glasses 100, and the smart glasses 200 ) does not display the second type image anymore.

Hereinafter, a method of controlling an image output to the smart glasses based on the location or distance of the smart glasses will be described.

After pairing of the smart glasses 100 and the set-top box 200 is completed, calibration is performed between the smart glasses 200 or the external display device to reproduce the second type image. This is to prepare a scale for controlling and reproducing the second type image by performing calibration with respect to the distance, movement, and direction of the smart glasses 100 or the user. To this end, for example, after pairing between the set-top box 200 and the smart glasses 100 is completed, distance information between the current user (or the smart glasses) and the external display device is acquired. In addition, direction information is obtained by instructing the user to move the head left and right. Information on the user's movement is calculated using the distance information and direction information and a sensor unit (eg, a gyro sensor) of the smart glasses 100 . The smart glasses 100 reproduce the second type image based on the movement information. Alternatively, the set-top box 200 may correct an image based on the calibration result and transmit the corrected image.

That is, the smart glasses 100 may receive an image whose resolution and size are adjusted based on the distance or location information between the user A (or the smart glasses) and the external display device. Alternatively, the smart glasses 100 may reproduce an image whose resolution, size, etc. are adjusted based on the distance or location information between the received image and the external display device.

When there is a movement of the user A, after measuring the distance between the external display device and the smart glasses 100, the image output through the output unit of the smart glasses 100 can be enlarged or reduced in proportion to the distance. can Through this, the smart glasses 100 may provide an image specialized for user movement.

8 is a conceptual diagram illustrating a flowchart applied to the present invention. In relation to this, the flowchart of FIG. 5 relates to a display control method of the smart glasses 100 worn by the aforementioned user A 10 .

First, the user A 10 wears the smart glasses 100 while viewing the first type image through the external display device. (S801) The smart glasses 100 may be turned on by receiving the input of the user A 10 or may be turned on by the wearing action of the user A.

When the smart glasses 100 are turned on, the smart glasses 100 recognize the set-top box 200 or the external display device to perform pairing, and a user setting for the set-top box 200 is performed. do. (s803) In this case, the smart glasses 100 may transmit the identifier of the smart glasses 100 to the set-top box 200 . The set-top box 200 receiving the identifier may transmit the second type image to the smart glasses 100 using data communication. Here, the second type image is specific to the smart glasses 100 and may be transmitted including the identifier.

The smart glasses 100 that have completed the pairing perform calibration on the external display device. (s805) Here, the calibration information may be shared with the set-top box 200 after the calibration operation is performed. Alternatively, when an image transmission request to the set-top box 200 is requested, it may be transmitted together to the set-top box 200 .

When the user selects the second type image after performing the calibration, the set-top box 100 still transmits the first type image to the external display device. (s807)

When the user selects the second type image, the set-top box 200 transmits the second type image to the smart glasses 100 . (s809) Here, the smart glasses 100 display to the user whether the second type image is being reproduced on the display, and the user can determine whether the second type image is reproduced from the user through the input unit of the smart glasses 100 . Receive selection information. Upon receiving the selection information, the smart glasses 100 may request transmission of the second type image to the set-top box 200 . The set-top box 200 receiving the transmission request of the second type image may transmit the second type image to the smart glasses 100 based on previously calibrated information.

Here, the second type image may be specific to the smart glasses 100 and may be transmitted including the identifier. Correspondingly, the smart glasses 100 may display the second type image when it is authenticated that the image is for the smart glasses 100 by including the identifier included in the second type image. . In this case, the image output of the smart glasses is possible through communication between the set-top box 200 and the smart glasses 100 .

Also, the smart glasses 100 may display whether the first type image is reproduced on the display. If the user requests to play the first type image instead of the second type image, the smart glasses 100 may request the set-top box 200 to transmit the first type image. In this case, the image output of the smart glasses is possible through communication between the set-top box 200 and the smart glasses 100 .

9 is a detailed flowchart applied to the present invention, particularly illustrating the second type image control method in steps S805 and S809. That is, the operation of controlling the second type image based on the calibration information is performed by detecting and analyzing a motion and a gesture of the smart glasses wearer 10 ( S901 ). That is, the step S805 is performed by the sensor unit 106 and/or the camera 105 among the internal components of the smart glasses 100 of FIG. 5, which will be described later. motion, etc.) is detected, and the motion determination unit 107 analyzes the detected motion and/or gesture to determine that a specific motion exists. (S903)

Specifically, the sensor unit 106 detects all movements of the smart glasses wearer 10 . For example, the smart glasses wearer 10 tilts the upper body forward, or a specific object (for example, a remote control) using either the left arm or the right arm. ) is detected. Thereafter, the motion determination unit 107 determines that the detected motion and/or gesture of the smart glasses wearer 10 has a movement greater than a preset specific size, or that a specific motion (eg, grabbing an object) gesture is the same as the motion. If it is determined, it is determined that there is a movement of the smart glasses wearer.

After the step S901, for example, following the meaningful movement of the smart glasses wearer 10 through step S805, it is determined whether the external display device exists in the display area (S905).

If it is determined that the user's movement is present in step S903 and it is determined that the external display device does not exist in the display area, the smart glasses 100 display the second type image. stop (S909)

If it is determined that the user's movement exists in step S903 and it is determined that the external display device exists in the display area, the smart glasses 100 take the user A's movement into consideration. The second type image is corrected and displayed based on the calibration information. (S907) If the distance of the user approaches the external display device, the enlarged second type image is output in consideration of the close distance. do. If the distance of the user increases from the external display device, a reduced second type image is output in consideration of the approaching distance. If the angle between the user and the external display device is changed, the changed second type image is output in consideration of the angle.

An embodiment according to the present invention will be described with reference to FIG. 10 .

The hatch top box 200 receives the processed first type image and the second type image from the broadcast management server 300 . It is assumed that the set-top box 200 outputs the first type image through the output unit 250 connected to the set-top box 200 .

It is assumed that calibration for setting information between the smart glasses 100 and the set-top box 200 is performed in a state in which communication is set up between the set-top box 200 and the smart glasses 100 and data transmission between them is possible.

The smart glasses 100 receive an input from the user A and transmit a request message for transmitting the second type image to the set-top box 100 . (S1003) Here, the request message may include the identifier of the smart glasses 100 .

The set-top box 100 transmits the second type image to the smart glasses 100 . (S1005) In this case, the second type image may include an identifier for identifying the smart glasses 100 .

The smart glasses 100 may output the received second type image based on calibration information. If a preset motion from the user A of the smart glasses 100 is detected through the sensor unit, the smart glasses 100 calculate information about the motion of the user A and the smart glasses 100, and based on the calibration information, Correction may be performed on the second type image. The smart glasses 100 transmit the corrected second type image to the smart glasses 100 through an output unit.

If the field of view of the smart glasses 100 is out of the display area due to the movement of the user A, the smart glasses 100 no longer display the second type image.

However, due to the user A's movement, if the field of view of the smart glasses 100 is again included in the display area and the preset time is exceeded, the smart glasses 100 again display the second type image. can be printed out.

On the other hand, when the user B wears the smart glasses at the same time, the above procedure may be applied to the user B. In addition, the user B may independently select an image other than the image of the user A, and in this case, the user B may view a different second type image with respect to the same first type image as the user A. That is, the second type image may be independent of the user.

According to the present invention, an image can be provided using peripheral vision using a wearable device, particularly smart glasses. In addition, by using smart glasses, you can enjoy content individually without affecting people around you. In addition, it is possible to enjoy the content appropriately for the situation by grasping the user's movement. That is, it is possible to provide an image specialized for user movement.

In the mobile terminal described above, the configuration and method of the embodiments described above cannot be limitedly applied, but all or part of each embodiment can be selectively combined so that various modifications can be made. may be

The present invention has the effect of increasing the usability of the wearable device by increasing user convenience of the wearable device. In addition, it will be apparent to those skilled in the art that the scope of the present invention described in the claims is equally applicable to all types of wearable devices as well as similar digital devices.

Claims (28)

A method for controlling a display in smart glasses, comprising:
performing pairing with a set-top box;
recognizing an external display that reproduces the first type image transmitted from the set-top box; and
Receiving a second type image from the set-top box,
The second type image is a peripheral vision image of the first type image,
Recognizing the external display comprises:
It is determined whether a preset portion of the external display is within a preset angle of view of the smart glasses,
When at least one of the corners of the external display is located outside the preset angle of view of the smart glasses while the second type image is being reproduced, the second type image is turned off,
How to control the display. The method of claim 1,
Further comprising the step of performing a calibration (Calibration) in consideration of at least one of the distance to the external display, angle, and location information,
How to control the display. delete delete 3. The method of claim 2,
When the user of the smart glasses moves,
measuring a distance between the smart glasses and the external display, and adjusting the size of the second type image based on the measured distance and a result of the calibration,
How to control the display. 3. The method of claim 2,
When the user of the smart glasses moves,
measuring an angle between the external displays, and adjusting an angle or depth of the second type image based on the measured angle and a result of the calibration;
How to control the display. The method of claim 1,
The first type image and the second type image are
Each video was shot using a dual camera,
How to control the display. 8. The method of claim 7,
The second type image is
An image captured using a depth camera,
How to control the display. A method for controlling a display in a set-top box coupled to an external display, the method comprising:
receiving a first type video and a second type video from a broadcast management server;
performing pairing with smart glasses;
displaying the first type image on the external display; and
transmitting the second type image to the smart glasses,
The second type image is a peripheral vision image of the first type image,
When at least one of the corners of the external display is located outside the preset field of view of the smart glasses during the reproduction of the second type image, the second type image is turned off,
How to control the display. 10. The method of claim 9,
The step of transmitting the second type image to the smart glasses comprises:
performed based on information about calibration with the smart glasses,
How to control the display. 11. The method of claim 10,
Information about the calibration,
Generated based on the distance between the smart glasses and the set-top box,
How to control the display. 10. The method of claim 9,
Transmitting the second type image to the smart glasses comprises:
Transmitted when the smart glasses recognize an external display coupled to the set-top box,
How to control the display. 10. The method of claim 9,
Recognition of the external display,
determining whether a preset portion of the external display is within a preset angle of view of the smart glasses,
How to control the display. 11. The method of claim 10,
When the user of the smart glasses moves,
measuring a distance between the smart glasses and the external display to adjust the size of the second type image based on the measured distance and a result of the calibration;
How to control the display. 11. The method of claim 10,
When the user of the smart glasses moves,
When the user of the smart glasses moves,
measuring the angle between the external displays in consideration of the calibration result, and adjusting the angle or depth of the second type image based on the measured angle and the calibration result,
How to control the display. 10. The method of claim 9,
The first type image and the second type image are
Each video was shot using a dual camera,
How to control the display. 10. The method of claim 9,
The second type image is
An image captured using a depth camera,
How to control the display. In the smart glasses for performing a method of controlling a display,
a processor for recognizing an external display that performs pairing with a set-top box and reproduces a first type image transmitted from the set-top box; and
and a transceiver module for receiving a second type image from the set-top box,
The second type image is a peripheral vision image of the first type image,
When at least one of the corners of the external display deviates from the preset angle of view of the smart glasses while the second type image is being reproduced,
The second type image is characterized in that turned-off,
smart glasses. 19. The method of claim 18,
The processor is
Calibration is performed in consideration of at least one of distance, angle, and location information with respect to the external display,
smart glasses. 19. The method of claim 18,
The processor recognizes a preset range of the external display and performs recognition of the external display based on whether the range is within a preset angle of view of the smart glasses,
smart glasses. delete 20. The method of claim 19,
When the user of the smart glasses moves,
measuring a distance between the smart glasses and the external display, and adjusting the size of the second type image based on the measured distance and a result of the calibration,
smart glasses. 20. The method of claim 19,
When the user of the smart glasses moves,
measuring an angle between the external displays, and adjusting an angle or depth of the second type image based on the measured angle and a result of the calibration;
smart glasses. A set-top box coupled to an external display for performing a display control method, the set-top box comprising:
a transceiver module for receiving the first type video and the second type video from the broadcast management server; and
a processor configured to perform pairing with smart glasses and to display the first type image on the external display, wherein the processor controls the transceiver module to transmit the second type image to the smart glasses;
The second type image is a peripheral vision image of the first type image,
When at least one of the corners of the external display deviates from the preset angle of view of the smart glasses while the second type image is being reproduced,
The second type image is characterized in that turned-off,
set-top box. 25. The method of claim 24,
The processor is
Transmitting the second type image to the smart glasses based on information about calibration with the smart glasses,
set-top box. 26. The method of claim 25,
Information about the calibration,
Generated based on the distance between the smart glasses and the set-top box,
set-top box. A computer readable recording medium having recorded thereon a program for performing a method for controlling a display in smart glasses, comprising:
The program is
performing pairing with a set-top box;
recognizing an external display that reproduces the first type image transmitted from the set-top box; and
Receiving a second type image from the set-top box,
The second type image is a peripheral vision image of the first type image,
When at least one of the corners of the external display deviates from a preset angle of view of the smart glasses during reproduction of the second type image, the second type image is turned off,
computer readable recording medium. In a computer-readable recording medium recorded with a program for performing a method of controlling a display in a set-top box coupled to an external display,
The program is
receiving a first type video and a second type video from a broadcast management server;
performing pairing with smart glasses;
displaying the first type image on the external display; and
transmitting the second type image to the smart glasses,
The second type image is a peripheral vision image of the first type image,
When at least one of the corners of the external display deviates from the preset angle of view of the smart glasses while the second type image is being reproduced,
The second type image is characterized in that turned-off,
computer readable recording medium.

Images