KR101801146B1 - Image display apparatus, and method for operating the same - Google Patents

Abstract

The present invention relates to an image display apparatus and an operation method thereof. A method of operating an image display apparatus according to an embodiment of the present invention includes acquiring distance information between a remote control apparatus and an image display apparatus, receiving motion information from a remote control apparatus, Calculating a coordinate of a pointer indicating a movement of the remote control device based on the calculated coordinates; and displaying the pointer in accordance with the calculated coordinates. This makes it possible to improve the usability of the user.

Description

[0001] The present invention relates to an image display apparatus and a method of operating the same,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image display apparatus and an operation method thereof, and more particularly, to an image display apparatus and an operation method thereof that can improve the usability of a user.

A video display device is a device having a function of displaying an image that a user can view. The user can view the broadcast through the video display device. A video display device displays a broadcast selected by a user among broadcast signals transmitted from a broadcast station on a display. Currently, broadcasting is changing from analog broadcasting to digital broadcasting around the world.

Digital broadcasting refers to broadcasting in which digital video and audio signals are transmitted. Compared to analog broadcasting, digital broadcasting is strong against external noise and has a small data loss, is advantageous for error correction, has a high resolution, and provides a clear screen. Also, unlike analog broadcasting, digital broadcasting is capable of bidirectional service.

It is an object of the present invention to provide a video display device and an operation method thereof that can improve the usability of the user.

Another object of the present invention is to provide an image display apparatus and an operation method thereof capable of calculating coordinates of a pointer indicating the motion of a remote control apparatus based on distance information between the remote control apparatus and the image display apparatus .

According to another aspect of the present invention, there is provided a method of operating an image display device, including: obtaining distance information between a remote control device and a video display device; receiving motion information from the remote control device; Calculating coordinates of a pointer indicating movement of the remote control device based on the motion information and the distance information, and displaying the pointer in accordance with the calculated coordinates.

According to another aspect of the present invention, there is provided a method of operating a remote control device, including: acquiring distance information between a remote control device and a video display device; acquiring color-difference video information for the remote control device; Calculating coordinates of a pointer indicating movement of the remote control device based on the motion information, the distance information, and the chrominance image information; And displaying the pointer accordingly.

According to another aspect of the present invention, there is provided an image display apparatus including a display, an interface unit for receiving motion information from the remote control apparatus, and an image capturing unit for capturing a depth image including the remote control apparatus. Acquires distance information between the remote control device and the image display device based on the depth image acquired by the image capturing section and calculates coordinates of a pointer indicating movement of the remote control device based on the motion information and the distance information And controls the pointer to be displayed on the display in accordance with the calculated coordinates.

According to the embodiment of the present invention, the coordinates of the pointer indicating the movement of the remote control device are calculated on the basis of the distance information between the remote control device and the video display device, thereby making it possible to improve the usability of the user in displaying the pointer .

In particular, as the distance information value becomes larger, the pointer movement amount can be set to be smaller by setting the scaling factor smaller, and accuracy can be increased when the pointer is displayed.

On the other hand, by increasing the distance information value and setting the pointer size to be larger, the user can intuitively recognize the distance between the remote control device and the image display device.

On the other hand, the scaling factor and the like can be adjusted through the setting menu for the remote control device, so that the user's convenience of use can be increased.

On the other hand, when the distance information value is variable, a desired image can be easily enlarged or reduced by zooming in or zooming out the image displayed in the image display device.

1 is a view illustrating an appearance of an image display apparatus according to an embodiment of the present invention.
2 is an internal block diagram of an image display apparatus according to an embodiment of the present invention.
3 is an internal block diagram of the control unit of FIG.
FIG. 4 is a diagram showing a control method of the remote control apparatus of FIG. 2. FIG.
5 is an internal block diagram of the remote control device of Fig.
FIG. 6 is a flowchart illustrating an operation method of a remote control apparatus according to an embodiment of the present invention.
Figures 7-9 are diagrams that are referenced to illustrate various examples of the method of operation of Figure 6.
10 is a flowchart illustrating an operation method of a remote control apparatus according to another embodiment of the present invention.
Figures 11A-D are diagrams that are referenced to illustrate various examples of the method of operation of Figure 10.
13 is a flowchart illustrating an operation method of a remote control apparatus according to another embodiment of the present invention.
14 is a diagram referred to explain various examples of the operating method of FIG.

Hereinafter, the present invention will be described in detail with reference to the drawings.

The suffix "module" and " part "for components used in the following description are given merely for convenience of description, and do not give special significance or role in themselves. Accordingly, the terms "module" and "part" may be used interchangeably.

1 is a view illustrating an appearance of an image display apparatus according to an embodiment of the present invention.

Referring to FIG. 1, an image capturing unit 125 may be disposed on an image display apparatus 100, particularly, a display 180. The image photographing unit 125 may be, for example, a camera, a camcorder, or the like.

In FIG. 1, the image capturing unit 125 includes a plurality of cameras 125a and 125b.

In addition, the image capturing unit 125 may be installed outside the image display apparatus 100, or may be installed in an embedded form.

On the other hand, the image capturing unit 125 is a unit separate from the image display apparatus 100, and can be connected to an image display apparatus as an external apparatus. Hereinafter, it is described as a component in the image display apparatus 100 for convenience of explanation.

On the other hand, when the user uses the remote control apparatus 200 while watching an image displayed in the image display apparatus 100, the image capturing section 125 can photograph an image including the remote control apparatus 200 . Hereinafter, the operation of the image display apparatus 100, such as a photographed image, will be described in detail with reference to the internal block diagram of FIG.

2 is an internal block diagram of an image display apparatus according to an embodiment of the present invention.

2, an image display apparatus 100 according to an exemplary embodiment of the present invention includes a broadcast receiving unit 105, an image capturing unit 125, an external device interface unit 130, a storage unit 140, An interface unit 150, a sensor unit (not shown), a controller 170, a display 180, an audio output unit 185, and a remote controller 200.

The broadcast receiving unit 105 may include a tuner 110, a demodulator 120, and a network interface unit 130. Of course, the tuner 110 and the demodulator 120 may be provided, but the network interface 130 may not be included. Alternatively, the tuner 110 and the demodulator 120 may be included, It is also possible to design the demodulation unit 120 not to include it.

The tuner 110 selects a channel selected by the user or an RF broadcast signal corresponding to all previously stored channels among RF (Radio Frequency) broadcast signals received through the antenna. Also, the selected RF broadcast signal is converted into an intermediate frequency signal, a baseband image, or a voice signal.

For example, if the selected RF broadcast signal is a digital broadcast signal, it is converted into a digital IF signal (DIF). If the selected RF broadcast signal is an analog broadcast signal, it is converted into an analog baseband image or voice signal (CVBS / SIF). That is, the tuner 110 can process a digital broadcast signal or an analog broadcast signal. The analog baseband video or audio signal (CVBS / SIF) output from the tuner 110 can be directly input to the controller 170.

Also, the tuner 110 can receive RF carrier signals of a single carrier according to an Advanced Television System Committee (ATSC) scheme or RF carriers of a plurality of carriers according to a DVB (Digital Video Broadcasting) scheme.

In the present invention, the tuner 110 sequentially selects RF broadcast signals of all broadcast channels stored through a channel memory function among RF broadcast signals received through an antenna, and outputs the selected RF broadcast signals as an intermediate frequency signal, a baseband image, or a voice signal Can be converted.

On the other hand, the tuner 110 can include a plurality of tuners for receiving broadcast signals of a plurality of channels. Alternatively, a single tuner that simultaneously receives broadcast signals of a plurality of channels is also possible.

The demodulator 120 receives the digital IF signal DIF converted by the tuner 110 and performs a demodulation operation.

The demodulation unit 120 may perform demodulation and channel decoding, and then output a stream signal TS. At this time, the stream signal may be a signal in which a video signal, a voice signal, or a data signal is multiplexed.

The stream signal output from the demodulation unit 120 may be input to the controller 170. The control unit 170 performs demultiplexing, video / audio signal processing, and the like, and then outputs an image to the display 180 and outputs audio to the audio output unit 185.

The image photographing unit 125 can photograph a user or a remote control device. For example, when the user holds the remote control device, it can photograph the user's hand, especially the remote control device.

Meanwhile, the image photographing unit 125 may include different types of photographing units, for example, a depth camera and a color camera.

The Depth Camera is composed of an infrared light source and an image sensor. It emits infrared rays from an infrared light source and calculates the distance information between the subject and the depth camera from the phase difference between the emitted infrared ray and the infrared ray reflected from the subject There is Time Of Flight (TOF). In addition, it is also possible to emit an infrared pattern (a large number of infrared points) from an infrared light source, photograph the reflection of the pattern on an object with an image sensor provided with a filter, and determine a distance between the subject and the depth camera There is a way to obtain information (Structured Light).

Meanwhile, the RGB camera may include three image sensors (CMOS) that acquire information on each color of R (Red), G (Green), and B (Blue) Compared to Depth Camera, it can acquire relatively high resolution image.

The image capturing unit 125 according to an exemplary embodiment of the present invention acquires a depth image including an image of a remote control device through a Depth Camera and transmits a depth image including a color difference Obtain the image.

On the other hand, when the image capturing unit 125 includes one camera, it is preferable to provide a depth camera. That is, the depth image including the image of the remote control device can be acquired through the Depth Camera.

On the other hand, the image capturing unit 125 may include a left-eye camera and a right-eye camera. That is, the left eye camera acquires the left eye image, and the right eye camera acquires the right eye image.

On the other hand, the photographed image is transmitted to the control unit 170. That is, the depth image, the color difference image, the depth image, or the left eye image and the right eye image captured by the image capturing unit 125 are transmitted to the controller 170.

The external device interface unit 130 can transmit or receive data with the connected external device 190. [ To this end, the external device interface unit 130 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The external device interface unit 130 can be connected to an external device such as a DVD (Digital Versatile Disk), a Blu ray, a game device, a camera, a camcorder, a computer , And may perform an input / output operation with an external device.

The A / V input / output unit can receive video and audio signals from an external device. Meanwhile, the wireless communication unit can perform short-range wireless communication with other electronic devices.

The network interface unit 135 provides an interface for connecting the video display device 100 to a wired / wireless network including the Internet network. For example, the network interface unit 135 can receive, via the network, content or data provided by the Internet or a content provider or a network operator.

Meanwhile, the network interface unit 135 can perform data communication with neighboring electronic devices in the vicinity of the video display device 100. At this time, the network interface unit 135 may receive device information of other electronic devices, channel information that can be remotely controlled by other electronic devices, frequency information, code information, and the like.

The storage unit 140 may store a program for each signal processing and control in the control unit 170 or may store the processed video, audio, or data signals.

In addition, the storage unit 140 may perform a function for temporarily storing video, audio, or data signals input to the external device interface unit 130. [ In addition, the storage unit 140 may store information on a predetermined broadcast channel through a channel memory function such as a channel map.

Although the storage unit 140 of FIG. 2 is provided separately from the control unit 170, the scope of the present invention is not limited thereto. The storage unit 140 may be included in the controller 170.

The user input interface unit 150 transmits a signal input by the user to the control unit 170 or a signal from the control unit 170 to the user.

(Not shown), such as a power key, a channel key, a volume key, and a set value, from the remote control apparatus 200, (Not shown) that senses a user's gesture to the control unit 170 or transmits a signal from the control unit 170 to the control unit 170 It is possible to transmit it to the sensor unit (not shown).

The control unit 170 demultiplexes the input stream or processes the demultiplexed signals through the tuner 110 or the demodulation unit 120 or the external device interface unit 130 to generate a signal for video or audio output Can be generated and output.

The video signal processed by the controller 170 may be input to the display 180 and displayed as an image corresponding to the video signal. Also, the image signal processed by the controller 170 may be input to the external output device through the external device interface unit 130.

The audio signal processed by the control unit 170 may be output to the audio output unit 185 as an audio signal. The audio signal processed by the controller 170 may be input to the external output device through the external device interface unit 130. [

Although not shown in FIG. 2, the controller 170 may include a demultiplexer, an image processor, and the like. This will be described later with reference to FIG.

In addition, the control unit 170 can control the overall operation in the video display device 100. [ For example, the controller 170 controls the tuner 110 to control the tuner 110 to select an RF broadcast corresponding to a channel selected by the user or a previously stored channel.

In addition, the controller 170 may control the image display apparatus 100 according to a user command or an internal program input through the user input interface unit 150.

Meanwhile, the control unit 170 may control the display 180 to display an image. At this time, the image displayed on the display 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

Meanwhile, the controller 170 may generate a 3D object for a predetermined object among the images displayed on the display 180, and display the 3D object. For example, the object may be at least one of a connected web screen (newspaper, magazine, etc.), EPG (Electronic Program Guide), various menus, widgets, icons, still images, moving images, and text.

Meanwhile, according to the embodiment of the present invention, the control unit 170 can recognize the position of the user or the remote control device based on the image photographed by the image photographing unit 125. [ For example, the distance (z-axis coordinate) between the remote control apparatus 200 and the image display apparatus 100 can be grasped. In addition, the x-axis coordinate and the y-axis coordinate in the display 180 corresponding to the position of the remote control device 200 can be grasped.

Specifically, the control unit 170 can obtain distance information between the remote control device and the image display device from the depth image photographed by the image photographing unit 125. [

Alternatively, the controller 170 may obtain distance information between the remote control device and the image display device from the left eye image and the right eye image captured by the image sensing section 125. [

On the other hand, the control unit 170 can calculate the coordinates of the pointer indicating the motion of the remote control device, based on the motion information from the remote control device and the obtained distance information. Then, a pointer can be displayed on the display 180 according to the calculated coordinates.

The controller 170 may control the pointer coordinate change amount through the motion information and the scaling factor so as to vary the scaling factor according to the distance information. At this time, the controller 170 can set the scaling factor to be smaller as the distance information value becomes larger. On the other hand, the controller 170 can control so that the larger the distance information value, the larger the pointer size is.

On the other hand, the control unit 170 controls to enter the setting menu for the remote control apparatus, and controls the scaling factor in the menu to vary according to the input.

On the other hand, when the acquired distance information is variable, the control unit 170 can control to zoom in or zoom out an image displayed on the display 180 according to the variable distance information.

Although not shown in the drawing, a channel browsing processing unit for generating a channel signal or a thumbnail image corresponding to an external input signal may be further provided. The channel browsing processing unit receives the stream signal TS output from the demodulation unit 120 or the stream signal output from the external device interface unit 130 and extracts an image from an input stream signal to generate a thumbnail image . The generated thumbnail image may be stream-decoded together with a decoded image and input to the controller 170. The control unit 170 may display a thumbnail list having a plurality of thumbnail images on the display 180 using the input thumbnail image.

At this time, the thumbnail list may be displayed in a simple view mode displayed on a partial area in a state where a predetermined image is displayed on the display 180, or in a full viewing mode displayed in most areas of the display 180. The thumbnail images in the thumbnail list can be sequentially updated.

The display 180 converts a video signal, a data signal, an OSD signal, a control signal processed by the control unit 170, a video signal, a data signal, a control signal, and the like received from the external device interface unit 130, .

The display 180 may be a PDP, an LCD, an OLED, a flexible display, or the like, and may also be capable of a 3D display.

 Meanwhile, the display 180 may be configured as a touch screen and used as an input device in addition to the output device.

The audio output unit 185 receives the signal processed by the control unit 170 and outputs it as a voice.

The remote control apparatus 200 transmits the user input to the user input interface unit 150. [ To this end, the remote control device 200 may be connected to a communication network such as Bluetooth, RF (Radio Frequency) communication, IR (infrared) communication, UWB (Ultra Wideband), ZigBee, Can be used.

Also, the remote control apparatus 200 can receive the video, audio, or data signal output from the user input interface unit 150. Further, the remote control device 200 can display or output audio on the remote control device 200 based on the received video, audio, or data signal.

In the embodiment of the present invention, the remote control device 200 mainly describes a space remote control capable of displaying a pointer corresponding to the movement of the user.

Meanwhile, the video display device 100 may be a digital broadcast receiver capable of receiving a fixed or mobile digital broadcast.

Meanwhile, a block diagram of the image display apparatus 100 shown in FIG. 2 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the image display apparatus 100 actually implemented. That is, two or more constituent elements may be combined into one constituent element, or one constituent element may be constituted by two or more constituent elements, if necessary. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and apparatuses do not limit the scope of the present invention.

2, the image display apparatus 100 does not include the tuner 110 and the demodulation unit 120 shown in FIG. 2, and the network interface unit 130 or the external device interface unit 135 , It is possible to receive and reproduce the image contents.

3 is an internal block diagram of the control unit of FIG.

The control unit 170 includes a demultiplexing unit 310, an image processing unit 320, a processor 330, an OSD generating unit 340, a mixer 345, A frame rate conversion unit 350, and a formatter 360. [0031] A voice processing unit (not shown), and a data processing unit (not shown).

The demultiplexer 310 demultiplexes the input stream. For example, when an MPEG-2 TS is input, it can be demultiplexed into video, audio, and data signals, respectively. The stream signal input to the demultiplexer 310 may be a stream signal output from the tuner 110 or the demodulator 120 or the external device interface 130.

The image processing unit 320 may perform image processing of the demultiplexed image signal. For this, the image processing unit 320 may include a video decoder 325 and a scaler 335.

The video decoder 325 decodes the demultiplexed video signal and the scaler 335 performs scaling so that the resolution of the decoded video signal can be output from the display 180.

The video decoder 325 can include a decoder of various standards.

The processor 330 may control the overall operation in the image display apparatus 100 or in the control unit 170. [ For example, the processor 330 may control the tuner 110 to select a channel selected by the user or an RF broadcast corresponding to a previously stored channel.

In addition, the processor 330 may control the image display apparatus 100 by a user command or an internal program input through the user input interface unit 150. [

In addition, the processor 330 may perform data transfer control with the network interface unit 135 or the external device interface unit 130.

The processor 330 may control operations of the demultiplexing unit 310, the image processing unit 320, the OSD generating unit 340, and the like in the controller 170.

The OSD generation unit 340 generates an OSD signal according to a user input or by itself. For example, based on a user input signal, a signal for displaying various information in a graphic or text form on the screen of the display 180 can be generated. The generated OSD signal may include various data such as a user interface screen of the video display device 100, various menu screens, a widget, and an icon. In addition, the generated OSD signal may include a 2D object or a 3D object.

The OSD generating unit 340 can generate a pointer that can be displayed on the display based on the pointing signal input from the remote control device 200. [ In particular, such a pointer may be generated by a pointing signal processing unit, and the OSD generating unit 240 may include such a pointing signal processing unit (not shown). Of course, a pointing signal processing unit (not shown) may be provided separately from the OSD generating unit 240.

The mixer 345 may mix the OSD signal generated by the OSD generator 340 and the decoded video signal processed by the image processor 320. At this time, the OSD signal and the decoded video signal may include at least one of a 2D signal and a 3D signal. The mixed video signal is supplied to a frame rate converter 350.

A frame rate converter (FRC) 350 can convert the frame rate of an input image. On the other hand, the frame rate converter 350 can output the frame rate without conversion.

The formatter 360 receives the mixed signal from the mixer 345, that is, the OSD signal and the decoded video signal, and changes the format of the signal to be suitable for the display 180 and outputs the signal. For example, the R, G, and B data signals may be output as low voltage differential signaling (LVDS) signals or mini-LVDS signals.

On the other hand, the formatter 360 can separate the 2D video signal and the 3D video signal for 3D video display. It is also possible to change the format of the 3D video signal or convert the 2D video signal to the 3D video signal.

On the other hand, the audio processing unit (not shown) in the control unit 170 can perform the audio processing of the demultiplexed audio signal. To this end, the voice processing unit (not shown) may include various decoders.

In addition, the audio processing unit (not shown) in the control unit 170 can process a base, a treble, a volume control, and the like.

The data processing unit (not shown) in the control unit 170 can perform data processing of the demultiplexed data signal. For example, if the demultiplexed data signal is a coded data signal, it can be decoded. The encoded data signal may be EPG (Electronic Program Guide) information including broadcast information such as a start time and an end time of a broadcast program broadcasted on each channel.

3, the signals from the OSD generating unit 340 and the image processing unit 320 are mixed in the mixer 345 and then 3D processed in the formatter 360. However, the present invention is not limited to this, May be located behind the formatter.

Meanwhile, the block diagram of the controller 170 shown in FIG. 3 is a block diagram for an embodiment of the present invention. Each component of the block diagram can be integrated, added, or omitted according to the specifications of the control unit 170 actually implemented.

In particular, the frame rate converter 350 and the formatter 360 are not provided in the controller 170, but may be separately provided.

4 is a diagram showing a control method of the remote control device of FIG.

4A, it is exemplified that a pointer 205 corresponding to the remote control device 200 is displayed on the display 180. In the example of FIG.

The user can move or rotate the remote control device 200 up and down, left and right (FIG. 4B), and back and forth (FIG. 4C). The pointer 205 displayed on the display 180 of the video display device corresponds to the movement of the remote control device 200. [ In this remote control device 200, as shown in the figure, since the pointer 205 is moved according to the movement in the 3D space, it can be called a space remote controller.

4B illustrates that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display 180 of the image display device also shifts to the left corresponding thereto.

Information on the motion of the remote control device 200 sensed through the sensor of the remote control device 200 is transmitted to the image display device. The image display apparatus can calculate the coordinates of the pointer 205 from the information on the motion of the remote control apparatus 200. [ The image display apparatus can display the pointer 205 so as to correspond to the calculated coordinates.

4C illustrates a case where the user moves the remote control device 200 away from the display 180 while pressing a specific button in the remote control device 200. [ Thereby, the selected area in the display 180 corresponding to the pointer 205 can be zoomed in and displayed. Conversely, when the user moves the remote control device 200 close to the display 180, the selection area within the display 180 corresponding to the pointer 205 may be zoomed out and zoomed out. On the other hand, when the remote control device 200 moves away from the display 180, the selection area is zoomed out, and when the remote control device 200 approaches the display 180, the selection area may be zoomed in.

On the other hand, when the specific button in the remote control device 200 is pressed, it is possible to exclude recognizing the up, down, left, and right movement. That is, when the remote control apparatus 200 moves away from or approaches the display 180, it is not recognized that the up, down, left, and right movements are recognized, and only the forward and backward movements are recognized. Only the pointer 205 is moved in accordance with the upward, downward, leftward, and rightward movement of the remote control device 200 in a state where the specific button in the remote control device 200 is not pressed.

On the other hand, the moving speed and moving direction of the pointer 205 may correspond to the moving speed and moving direction of the remote control device 200.

5 is an internal block diagram of the remote control device of FIG.

The remote control device 200 includes a wireless communication unit 220, a user input unit 230, a sensor unit 240, an output unit 250, a power supply unit 260, a storage unit 270, And may include a control unit 280.

The wireless communication unit 220 transmits / receives signals to / from any one of the video display devices according to the embodiments of the present invention described above. Of the video display devices according to the embodiments of the present invention, one video display device 100 will be described as an example.

In this embodiment, the remote control apparatus 200 may include an RF module 221 capable of transmitting and receiving signals with the image display apparatus 100 according to the RF communication standard. Also, the remote control apparatus 200 may include an IR module 223 capable of transmitting and receiving signals with the image display apparatus 100 according to the IR communication standard. The remote control device 200 may also include an NFC module 225 for short-range magnetic field communication with an electronic device.

The remote control device 200 can transmit a signal including information on the motion of the remote control device 200 to the image display device 100 through the RF module 221.

Also, the remote control device 200 can receive the signal transmitted by the video display device 100 through the RF module 221. [ Also, the remote control device 200 can transmit commands to the video display device 100 via the IR module 223, such as power on / off, channel change, volume change, and the like, as needed.

The user input unit 230 may include a keypad, a button, a touch pad, or a touch screen. The user can input a command related to the image display apparatus 100 to the remote control apparatus 200 by operating the user input unit 230. [ When the user input unit 230 includes a hard key button, the user can input a command related to the image display device 100 to the remote control device 200 through the push operation of the hard key button. When the user input unit 230 includes a touch screen, the user may touch a soft key of the touch screen to input a command related to the image display apparatus 100 to the remote control apparatus 200. [ In addition, the user input unit 230 may include various types of input means such as a scroll key, a jog key, and the like that can be operated by the user, and the present invention does not limit the scope of the present invention.

The sensor unit 240 may include a gyro sensor 241 or an acceleration sensor 243. The gyro sensor 241 can sense information on the motion of the remote control device 200. [

For example, the gyro sensor 241 can sense information on the operation of the remote control device 200 based on the x, y, and z axes. The acceleration sensor 243 can sense information on the moving speed of the remote control device 200 and the like.

The output unit 250 may output an image or voice signal corresponding to the operation of the user input unit 230 or corresponding to the signal transmitted from the image display apparatus 100. The user can recognize whether the user input unit 230 is operated or whether the image display apparatus 100 is controlled through the output unit 250.

For example, the output unit 250 includes an LED module 251 that is turned on when the user input unit 230 is operated or a signal is transmitted and received between the image display device 100 and the image display device 100 through the wireless communication unit 225, 253 for outputting sound, an acoustic output module 255 for outputting sound, or a display module 257 for outputting an image.

The power supply unit 260 supplies power to the remote control device 200. The power supply unit 260 can reduce the power consumption by stopping the power supply when the remote controller 200 is not moving for a predetermined time. The power supply unit 260 may resume power supply when a predetermined key provided in the remote control device 200 is operated.

The storage unit 270 may store various types of programs, application data, and the like necessary for the control or operation of the remote control apparatus 200. [ When the remote control device 200 wirelessly transmits and receives a signal through the image display device 100 and the RF module 221, the remote control device 200 and the image display device 100 transmit signals through a predetermined frequency band Send and receive. The control unit 280 of the remote control device 200 stores information on the frequency band and the like capable of wirelessly transmitting and receiving signals with the video display device 100 paired with the remote control device 200 in the storage unit 270 Can be referenced.

The control unit 280 controls various matters related to the control of the remote control device 200. [ The control unit 280 transmits a signal corresponding to a predetermined key operation of the user input unit 230 or a signal corresponding to the motion of the remote control device 200 sensed by the sensor unit 240, (100).

FIG. 6 is a flowchart showing an operation method of a remote control apparatus according to an embodiment of the present invention, and FIGS. 7 to 9 are drawings referred to explain various examples of the operation method of FIG.

Referring to the drawing, first, the distance information between the remote control device and the image display device is obtained (S610).

The control unit 170 of the image display apparatus receives the image photographed by the image photographing unit 125. [

For example, the photographed image may be a depth image photographed by a depth camera. This depth image may include an image corresponding to the remote control device. Accordingly, the control unit 170 can extract an image corresponding to the remote control apparatus from among the depth images including the image corresponding to the remote control apparatus. Then, the depth of the image corresponding to the remote control device can be extracted. Thereby, the control unit 170 can acquire distance information, that is, depth information, from the depth image.

7 illustrates a depth image 710 taken by a depth camera of the image capturing unit 125. As shown in FIG. The depth image 710 may include a remote control device image 715 held by the user and the user.

On the other hand, the depth image 710 has no chrominance component and only a luminance component is displayed. As the depth becomes larger, the dark image is displayed. The smaller the depth, the brighter the image can be displayed. In the figure, it can be seen that the brightness of the remote control device image 715 is the brightest.

As another example, the photographed image may be a left eye image and a right eye image photographed from a left eye camera and a right eye camera. Such a left eye image and a right eye image may each include an image corresponding to a remote control device. Accordingly, the control unit 170 detects the disparity between the images corresponding to the respective remote control devices in the left eye image and the right eye image, and extracts the depth of the image corresponding to the remote control device in accordance with the corresponding time difference . Thereby, the control unit 170 can acquire distance information, that is, depth information, from the depth image. On the other hand, the larger the parallax, the larger the depth information value.

Next, the motion information is received from the remote control device (S620).

The user input interface unit 150 of the video display device 100 receives the motion information from the remote control device 200. [ Then, the control unit 170 transmits it to the control unit 170.

The motion information may be x-axis coordinate information on the space, y-axis coordinate information, and z-axis coordinate information. Alternatively, it may be spatial, rotational angle information, and rotational speed information. Or information including all of them.

The controller 170 of the video display device 100 receives such motion information.

Next, on the basis of the motion information and the distance information, the coordinates of the pointer indicating the motion of the remote control apparatus are calculated (S630). Then, the pointer is displayed according to the calculated coordinates (S640).

The pointer coordinate calculation can be calculated by the following equations (1) and (2).

Here, P _{present - axis} is the current coordinate value, P _{prvious - axis} is the previous coordinate value, and ΔP _{present - axis} is the current coordinate change amount.

Here, f _sc denotes a scaling factor value, and? P _presen denotes a current pointer movement change amount.

That is, the pointer coordinate calculation is, through the movement variation information (ΔP _presen) with a scaling factor (f _sc), a pointer-coordinate change _amount-but determining (P _{present axis),} to vary the scaling factor (f _sc) in accordance with the distance information .

The motion change amount information ΔP _presen is received from the remote control device 200 and transmitted to the control unit 170 via the user input interface unit 150 of the image display device 100.

On the other hand, the scaling factor f _sc may vary depending on the distance information. For example, the larger the distance information value, the smaller the scaling factor can be set. This is to prevent the phenomenon that the pointer movement amount increases even if the image display apparatus is moved at the same angle as the distance information value becomes larger. Accordingly, even if the distance between the video display device 100 and the remote controller 200 is increased, the movement amount of the pointer can be appropriately adjusted.

On the other hand, the control unit 170 controls the pointer 180 to be displayed on the display 180 in accordance with the calculated pointer coordinates. Such a pointer may be generated in the OSD generation unit 340 in the control unit 170. [

On the other hand, the controller 170 can be set such that the larger the distance information value, the larger the pointer size. Accordingly, the user can intuitively grasp that the distance between the video display device 100 and the remote control device 200 is distant from the size of the pointer. Further, since the pointer size increases as the distance increases, the pointer can be easily grasped.

5, coordinate values are calculated by the coordinate value calculation unit 215. Alternatively, the coordinate value calculation unit 215 may calculate the coordinate values.

8A shows a case where the pointer 205 corresponding to the motion of the remote control device 200 is displayed at the first point A in a state in which the broadcast image 710 is displayed on the display 180 of the video display device 100 .

At this time, the control unit 170 of the video display device receives the motion information of the remote control device and controls to display a pointer based on the motion information.

8B shows a case where the pointer 205 is moved from the first point A to the second point B when the remote control device 200 rotates at the first distance d1 at the first rotation angle? .

At this time, the control unit 170 of the video display device controls to display the pointer on the basis of the motion information and the distance information d1 of the remote control device.

8C shows a case where the pointer 205 is moved from the first point A to the third point C when the remote control device 200 rotates at the second distance d2 at the first rotation angle? .

At this time, when the controller 170 of the video display device ignores the second distance information and is based only on the motion information of the remote control device, that is, when maintaining the same scaling factor as in Fig. 8C, The pointer 205 can be displayed at the point C.

8B and 8C are compared with each other, it can be seen that the distance difference d1 and d2 between the remote control device 200 and the video display device 100, despite the same rotation angle [ Is significant.

In the embodiment of the present invention, as described above, the amount of pointer movement change is determined based on the distance information between the remote control device 200 and the video display device 100. [ That is, the larger the distance information value is, the smaller the pointer movement variation is set. To this end, as described above, the scaling factor value for determining the amount of pointer movement change can be set to be smaller as the distance information value increases.

8D shows a case where the pointer 205 is moved from the first point A to the fourth point D when the remote control device 200 rotates at the first distance d2 from the first rotation angle? .

FIG. 8D illustrates that as the distance between the remote control device 200 and the image display device 100 increases, the scaling factor value becomes smaller as compared with FIG. 8C or FIG. 8B, .

That is, the fourth point D is located between the second point B and the third point C. Accordingly, even if the distance between the video display device 100 and the remote controller 200 is increased, the movement amount of the pointer can be appropriately adjusted.

It is also possible to set the pointer size to be larger as the distance between the video display device 100 and the remote control device 200 increases, in addition to the above-described pointer movement amount adjustment. This makes it easy to grasp the pointer.

On the other hand, the above-described scaling factor can be adjusted in the setting menu.

9 illustrates that the setting menu of the remote control device is displayed on the display 180 of the image display device. In the figure, the setting menu 910 of the remote control device includes a pointer shape item, a pointer color item, a pointer movement amount item (scaling factor item), and the like.

The setting menu 910 may be generated in the OSD generating unit 340 in the control unit 170.

On the other hand, the setting menu 910 can be displayed according to a predetermined input, and it is possible to set the pointer shape, the pointer color, and the pointer movement amount to be large or small according to the selection input of the user.

The set value may be stored in the storage unit 140, and the control unit 170 may apply the scaling factor as described above according to the setting. In particular, the scaling factor can be reduced in proportion to the distance information.

FIG. 10 is a flowchart showing an operation method of a remote control apparatus according to another embodiment of the present invention, and FIGS. 11A to 12D are views referred to explain various examples of the operation method of FIG.

Referring to the drawing, first, the distance information between the remote control device and the image display device is obtained (S1010). Step 1010 (S1010) is similar to step 610 (S610) of FIG. 6, and a description thereof will be omitted.

11A illustrates a depth image 1110 taken by a depth camera of the image capturing unit 125. FIG. The depth image 1110 may include a remote control device image 1115 held by the user and the user.

On the other hand, the depth image 1110 has no chrominance component, and only the luminance component is displayed. As the depth becomes larger, the dark image is displayed. The smaller the depth, the brighter the image can be displayed. In the figure, it can be seen that the brightness of the remote control apparatus image 1115 is the brightest.

Next, chrominance image information for the remote control apparatus is obtained (S1015).

The control unit 170 of the image display apparatus receives the image photographed by the image photographing unit 125. [

For example, the photographed image may be a color image photographed by an RGB camera. Such a color difference image may include an image corresponding to the remote control device. Accordingly, the control unit 170 can extract an image corresponding to the remote control apparatus from the color difference image including the image corresponding to the remote control apparatus.

On the other hand, FIG. 11B illustrates a chrominance image 1120 captured by the RGB camera of the image capturing unit 125. FIG. The chrominance image 1120 may include a remote control device image 1125 held by the user and the user.

11B is a chrominance image, although chrominance component representation can not be expressed in the drawing.

Next, motion information is received from the remote control apparatus (S1020). Operation 1020 (S1020) is similar to operation 620 (S620) of FIG. 6, and thus description thereof will be omitted.

Next, based on the motion information, the chrominance image information, and the distance information, the coordinates of the pointer indicating the motion of the remote control apparatus are calculated (S1130). Then, the pointer is displayed according to the calculated coordinates (S1140).

The pointer coordinate calculation can be calculated by the following equation (3).

Here, P _{present - axis} is calculated based on the current coordinate value, P _{prvious - axis} is the previous coordinate value, and err _present is calculated based on the current coordinate change amount estimated based on the image capturing unit 125 and the motion information of the remote control apparatus Which is the difference of the current pointer coordinate change. In addition, B means a scaling factor. Specifically, it means a blending factor.

That is, the pointer coordinate calculation includes a step of estimating a pointer coordinate change amount indicating movement of the remote control device 200 based on the distance information and the position change information of the remote control device 200 in the color difference image, Extracting a pointer coordinate change amount based on the motion information of the pointer coordinate system; and calculating the final coordinate change amount B * err _present based on the estimated coordinate change amount and the extracted coordinate change amount.

The step of estimating the pointer coordinate change is based on the distance information from the depth image photographed by the image photographing section 125 and the x- and y-coordinate information of the image display device from the color difference image photographed by the image photographing section 125 , And estimates the pointer coordinate change amount.

The step of extracting the pointer coordinate change amount extracts the pointer coordinate change amount based on the motion information received from the remote control device 200, for example, rotation information, specifically, rotation angle variation amount.

The step of calculating the final coordinate change amount B * err _present may calculate the final coordinate change amount through the difference (err _present ) between the estimated coordinate change amount and the extracted coordinate change amount and the scaling factor (B).

At this time, the scaling factor B can vary its value according to the distance information. That is, the larger the distance information value, the smaller the scaling factor B can be set.

For example, the larger the distance information value, the larger the err _present value between the estimated coordinate change amount and the extracted coordinate change amount. Thus, by setting the scaling factor B small, it becomes possible to adjust the final coordinate change amount B * err _present to be small.

5, coordinate values are calculated by the coordinate value calculation unit 215. Alternatively, the coordinate value calculation unit 215 may calculate the coordinate values.

12A shows a case where the pointer 205 corresponding to the motion of the remote control device 200 is displayed at the first point A in a state in which the broadcast image 1210 is displayed on the display 180 of the image display device 100 .

At this time, the control unit 170 of the video display device receives the motion information of the remote control device and controls to display a pointer based on the motion information.

12B shows a case where the pointer 205 is moved from the first point A to the second point B when the remote control device 200 rotates at the first distance d1 at the first rotation angle? .

At this time, the control unit 170 of the video display device controls to display the pointer on the basis of the motion information and the distance information d1 of the remote control device.

12C shows a case where the pointer 205 is moved from the first point A to the third point C when the remote control device 200 rotates at the second distance d2 at the first rotation angle? .

At this time, when the control unit 170 of the video display device ignores the second distance information and is based solely on motion information of the remote control device, the pointer 205 may be displayed at the third point C have.

12B and 12C are compared, it can be seen that, due to the difference in distance d1 and d2 between the remote control device 200 and the image display device 100 despite the same rotation angle [ Is significant.

12D shows a case where the pointer 205 is moved from the first point A to the fourth point D when the remote control device 200 rotates at the first distance d2 from the first rotation angle? .

The pointer coordinates calculated by the above-described estimation of the pointer coordinate change amount may be the fifth point D1 and the pointer coordinates calculated by extracting the pointer coordinate change amount may be the sixth point D2.

In order to compensate for such a difference err _present , the scaling factor B value is used as shown in Equation (3), and in particular, as the distance between the remote control device 200 and the image display device 100 increases, The value of the factor B is set to be small.

Accordingly, FIG. 12D shows a result that the value of the scaling factor B becomes smaller as the distance between the remote control device 200 and the image display device 100 becomes larger as compared with FIG. 12C or FIG. 12B, The amount of change is reduced.

That is, the fourth point D is located between the fifth point D1 and the sixth point D2. Accordingly, even if the distance between the video display device 100 and the remote controller 200 is increased, the movement amount of the pointer can be appropriately adjusted.

It is also possible to set the pointer size to be larger as the distance between the video display device 100 and the remote control device 200 increases, in addition to the above-described pointer movement amount adjustment. This makes it easy to grasp the pointer.

FIG. 13 is a flowchart illustrating an operation method of a remote control apparatus according to another embodiment of the present invention, and FIG. 14 is a diagram referred to explain various examples of the operation method of FIG.

Referring to the drawing, first, an image is displayed on the display 180 of the image display apparatus (S1305). The displayed image may be a broadcast image, a predetermined menu screen of the image display device, a stored content playback image, an externally connected content playback image, or a web screen of a connected web site.

Next, distance information between the remote control device and the image display device is obtained (S1310). Step 1310 (S1310) is similar to operation 610 (S610), and thus description thereof will be omitted.

Next, motion information is received from the remote control device (S1320). The remote control device 200 can perform the z-axis movement, which is close to or farther from the image display device, in addition to the rotation and movement of the remote control device in the x- and y-axes.

The control unit 170 receives the motion information from the remote control device. On the other hand, the depth camera of the image capturing unit 125 can continuously acquire the depth image. Accordingly, the distance information of the image display device can be continuously acquired through the depth image.

Next, it is determined whether or not the acquired distance information is variable (S1330). If so, the display image is zoomed in or zoomed out according to the variable distance information (S1340).

The control unit 170 can continuously receive the depth image photographed by the depth camera. Then, the controller 170 can continuously acquire the distance information between the video display device and the remote control device through the depth image.

The control unit 170 can control the displayed image to be zoomed in and displayed when the distance information value increases, that is, when the remote control device 200 moves away from the image display device 100. [ In addition, the controller 170 can control to zoom out and display the displayed image when the distance information value decreases, that is, when the remote control device 200 approaches the image display device 100. [

On the other hand, the reverse is also possible. That is, when the distance information value increases, the control unit 170 controls the displayed image to be zoomed out and displayed, and when the distance information value decreases, the displayed image is zoomed and displayed.

14 illustrates that a zoom-in operation is performed in accordance with the movement of the remote control device in a state that the Internet screen is displayed on the display 180 of the image display device 100. FIG.

Fig. 14 (a) illustrates that an internet screen including a text area 1410 and an image area 1420 is displayed on the display 180. Fig. At this time, the pointer indicating the movement of the remote control device 200 is displayed in the image area 1420.

At this time, when the remote control apparatus 200 moves away from the image display apparatus 100, that is, when the distance information value increases, at least a part of the area of the Internet screen is zoomed in and displayed as shown in FIG. 14B.

In the drawing, only the image area 1420 in which the pointer 205 is located in the text area 1410 and the image area 1420 is zoomed in to show an enlarged image area 1430, It is also possible to display them all zoomed in. Thereby, the user can easily perform the zoom-in operation.

 On the other hand, the control unit 170 can also acquire the distance information between the video display device and the remote control device from the motion information from the remote control device. That is, the remote control device 200 can transmit, to the image display device, z-axis movement information that is closer to or farther from the image display device than rotation and movement of the remote control device in the x-axis and y-axis. Thereby, the control unit 170 can obtain the distance information between the video display device and the remote control device.

Such a zoom-in or zoom-out display is applicable to the flowchart of FIG. 6 or the flowchart of FIG. 10 described above. That is, when the obtained distance information is variable, the displayed image can be zoomed in or zoomed out.

The image display apparatus and the operation method thereof according to the present invention are not limited to the configuration and method of the embodiments described above but the embodiments can be applied to all or some of the embodiments May be selectively combined.

Meanwhile, the operation method of the image display apparatus of the present invention can be implemented as a code that can be read by a processor on a recording medium readable by a processor included in the image display apparatus. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

Claims (20)

Obtaining a depth image and a color difference image including a remote control device;
Obtaining distance information between the remote control device and the image display device based on the depth image;
Receiving motion information from the remote control device;
Extracting an image corresponding to the remote control device from among the color difference images;
Calculating coordinates of a pointer indicating a motion of the remote control device based on the motion information, the distance information, and the chrominance image information corresponding to the extracted image; And
And displaying the pointer according to the calculated coordinates,
Wherein the coordinate calculating step comprises:
Estimating a pointer coordinate change amount indicating movement of the remote control device based on the distance information and the position change information of the remote control device in the color difference image;
Extracting a pointer coordinate change amount based on motion information of the remote control device; And
And calculating a final coordinate change amount based on the estimated coordinate change amount and the extracted coordinate change amount. The method according to claim 1,
Wherein the pointer coordinate calculation step comprises:
Determining a pointer coordinate change amount through the motion information and the scaling factor,
Wherein the scaling factor is varied according to the distance information. 3. The method of claim 2,
Wherein the scaling factor is set to be smaller as the distance information value becomes larger. The method according to claim 1,
The pointer display step may include:
Wherein the pointer is displayed such that the size of the pointer increases as the distance information value increases. 3. The method of claim 2,
Entering a setting menu for the remote control device;
And changing a scaling factor in the menu according to an input. The method according to claim 1,
Wherein the coordinate calculating step comprises:
Wherein the coordinates of the pointer are calculated so that as the distance between the remote control device and the image display device increases, the amount of movement of the pointer by the motion information decreases. The method according to claim 1,
The distance information acquiring step includes:
Obtaining a depth image including the remote control device; And
And extracting a depth of the depth image for the remote control device. The method according to claim 1,
Further comprising zooming in or zooming out the display image according to the variable distance information when the obtained distance information is variable. delete delete delete delete delete delete display;
An interface for receiving motion information from a remote control device;
An image capturing unit for capturing a depth image and a color difference image including the remote controller; And
Acquiring distance information between the remote control device and the image display device based on the depth image, extracting an image corresponding to the remote control device based on the color difference image, And a control unit for calculating coordinates of a pointer indicating movement of the remote control device based on the color difference image information corresponding to the extracted image and controlling the pointer to be displayed on the display in accordance with the calculated coordinates ,
Wherein,
Estimating a pointer coordinate change amount indicating movement of the remote control device based on the distance information and the position change information of the remote control device in the color difference image,
Extracts a pointer coordinate change amount based on the motion information of the remote control device,
And calculates a final coordinate change amount based on the estimated coordinate change amount and the extracted coordinate change amount. 16. The method of claim 15,
Wherein,
Determining a pointer coordinate change amount through the motion information and the scaling factor,
And the scaling factor is varied according to the distance information. 17. The method of claim 16,
Wherein,
Wherein the scaling factor is set to be smaller as the distance information value becomes larger. 16. The method of claim 15,
Wherein,
Controls to enter a setting menu for the remote control device, and controls the scaling factor in the menu to vary according to the input. 16. The method of claim 15,
Wherein,
And controls the zooming-in or zooming-out of the display image according to the variable distance information when the obtained distance information is variable. delete

Images