KR101102996B1 - Remote control device for diplay - Google Patents

Abstract

The present invention relates to a remote control device for a display. The present invention relates to a remote control device for remotely controlling a display device including a screen, wherein a pad for sensing an object is provided and is spaced apart from the pad. A sensing module for detecting a position of an object, detecting whether an object contacts the pad, sensing a position where the object is in contact with the pad, and sensing a pressure at which the object presses the pad, the remote control device A microcontroller for controlling the operation of the sensing module, generating an operation signal for a graphical user interface (GUI) using a signal output from the sensing module, and transmitting the operation signal to an external display device. It includes a wireless transmitter for. According to the present invention, there is an effect of overcoming the lack of directness which is a problem of the conventional TV remote control apparatus and enabling intuitive and efficient operation.

Display, remote control, remote control, directivity, LED, photo transistor, thumb.

Description

Remote control device for display {Remote control device for diplay}

The present invention relates to a remote control device for a display, and more particularly, to a remote control device that can remotely control an object in a display device such as a TV, a monitor, a home game machine.

A graphical user interface, or GUI, refers to a work environment in which a user exchanges information with a computer through graphics. In the past, the user interface was performed by a command through a keyboard, and the text was displayed on the screen. However, in the graphical user interface, a mouse or the like is used to select a menu from the screen to instruct the work. The GUI is in contrast to the DOS command interface. The elements of the GUI include Windows, scrollbars, icon images, and buttons. Since the late 1980s, IBM PCs and workstations have been using the GUI, and most computers use it. Microsoft's Windows and Apple Macintosh GUIs are examples.

Recently, a remote controller including a direction key, an analog joystick, a mini trackball, a touch pad, and the like is mainly used to manipulate GUI objects such as menus, buttons, and icons on a screen in a TV viewing environment. Currently, most domestic IPTVs use a GUI that selects and executes a command by moving the focus between the GUI objects on the screen without using a cursor. In order to operate the GUI, a direction key is mainly used. On the other hand, as the types of IPTV services are diversified, there is a lack of a method of moving the focus by using a direction key. Therefore, a need for a more efficient GUI arises. In this case, an analog joystick, a mini trackball, a touch pad, and the like are being included in the remote controller for the manipulation of the cursor.

Another method for manipulating GUI objects such as menus, buttons, and icons on the screen in a TV viewing environment is to manipulate the objects as if the laser pointer is aimed at and selects the objects on the screen. A representative example of commercialization is a controller of a Nintendo Wii game machine. In this case, the image sensor embedded in the controller detects the position of the infrared light source located on the TV, and the controller measures the angle of inclination with respect to the direction of the light source, and uses this value to determine the position of the cursor on the screen. Another example is the implementation of a similar function using the directivity of the LED, US Patent 6727887 (Wireless pointing device for remote cursor control).

The GUI operation method used in the TV viewing environment described above can be roughly divided into two types. The first is to select and execute the focus between buttons or menu items by using the arrow keys, and the second is to move the cursor to the pointing device and manipulate the GUI object through the cursor as in the PC GUI. The first method is difficult to cope with TV functions that are expected to be more diverse in the future, and the second method is more efficient than the first method, but lacks directness in that GUI objects must be manipulated through a cursor. In this case, directness refers to the intuitiveness of a user interface that can give a feeling of directly manipulating GUI objects as if they were physical objects. The most direct user interface is the touchscreen interface. However, the problem with the touch screen interface is that it is difficult to use the touch screen in a TV viewing environment. That is, in general, in a TV viewing environment, since the user is separated from the screen, it is impossible to touch an object on the screen by hand. Accordingly, there is a need for a pointing device capable of overcoming the lack of directivity of a pointing device for a conventional TV and providing directness such as a touch screen.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object thereof is to provide a remote control apparatus that can directly manipulate an object on a screen of a display device such as a touch screen.

In order to achieve the above object, the present invention provides a remote control apparatus for remotely controlling a display device including a screen, wherein a pad for sensing an object is provided and a position of an object spaced apart from the pad is sensed. And a sensing module for detecting whether the object is in contact with the pad, a position at which the object is in contact with the pad, and sensing pressure at which the object presses the pad, and overall control of the remote control device. And a microcontroller for controlling an operation of the sensing module, generating an operation signal for a graphical user interface (GUI) using a signal output from the sensing module, and a wireless transmitter for transmitting the operation signal to an external display device. do.

The sensing module may include a touch sensing unit for detecting whether an object contacts the pad, a light emitting unit for emitting light, and a light receiving sensor for sensing light reflected from the object from among the light emitted from the light emitting unit A part and an object may include a pressure sensing unit for sensing the pressure to press the pad.

The touch sensor circuit may further include a touch sensor circuit unit configured to detect whether or not an object contacts the pad by using the signal from the touch detector and transmit the detected contact to the microcontroller. The touch sensing unit may be a transparent electrode.

The light emitting unit may be a light emitting diode (LED) matrix.

The light receiving sensor unit may be a photo transistor array.

The pressure sensing unit may be a force sensor.

The pad may have a form ratio having the same aspect ratio as the aspect ratio of the screen. In this case, each area of the pad and each area of the screen may correspond one-to-one.

The microcontroller may generate an operation signal for displaying a position of an object sensed on the pad in a corresponding area on the screen by using the signal output from the sensing module.

The microcontroller may generate an operation signal for displaying the object in an area on the screen corresponding to the position of the detected object when the object is separated within a detectable distance from the pad.

As the distance between the pad and the object increases, the microcontroller may adjust the transparency so that the corresponding object on the screen is dimly displayed.

The microcontroller displays a corresponding object and a shadow of the corresponding object on the screen when the object is separated within a detectable distance from the pad, and displays the distance between the object and the shadow from the pad and the object. It can be displayed to be proportional to the distance to and.

The microcontroller may display a corresponding object on the screen and a distance between the pad and the object when the object is separated from the pad within a detectable distance.

The microcontroller may generate an operation signal for adding an outline to a corresponding object on the screen when an object contacts the pad.

The microcontroller may generate an operation signal for changing a color of the corresponding object on the screen to another color when an object is contacted with the pad and a pressure is applied.

The microcontroller may control each LED constituting the LED matrix to be sequentially turned on one at a time.

The microcontroller may read an output voltage of the photo transistor array to generate an image of an object spaced apart from the pad. At this time,

The microcontroller may signal-process the generated object image to calculate a planar position of the object and a distance from the pad.

The object is a thumb, and the microcontroller can normalize the generated thumb image and estimate the height z of the thumb from the pad plane from the normalized thumb image value.

The object is a thumb, the microcontroller normalizes the generated thumb image, converts the normalized thumb image value to a level required for image processing, obtains the center of gravity and ellipticity of the converted value, and the weight The coordinates of the thumb image may be corrected using the center and the flatness, and the corrected value may be filtered through an adaptive low pass filter to calculate the planar position of the object.

The rate of change of the thumb height z may be measured to determine the parameters of the adaptive low pass filter. The change rate of the thumb height z may be measured by filtering the thumb height z through a low pass filter and differentially filtering the value of the thumb height z.

The wireless transmitter may transmit a signal using infrared communication, Bluetooth, or Zigbee.

The apparatus may further include an input unit configured to receive an input signal for operating the display device from a user.

According to the present invention, there is an effect of overcoming the lack of directness which is a problem of the conventional TV remote control apparatus and enabling intuitive and efficient operation.

In addition, the present invention is expected to be a more preferred pointing device when watching TV, which should be a time of rest, because the fatigue is less compared to the operating device to operate as a laser pointer.

In addition, the one-to-one correspondence between the thumb sensor and the TV screen can correspond to the main functions for the TV operation to the four sides of the thumb sensor square, thereby enabling an effective interface design utilizing the user's location memory capability. For example, if the channel menu is placed along the bottom side of the screen and the volume control is placed along the right side, the channel menu is activated by touching the bottom side of the sensor provided in the remote control and the bottom side of the sensor You can select a channel as you move along, or touch the right side of the sensor to activate the volume control and adjust the volume as you move along the right side of the sensor.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used for the same reference numerals even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a view showing a remote control device and a screen according to an embodiment of the present invention.

Referring to FIG. 1, a remote control device 100 and a display device 200 are shown. In one embodiment of the present invention, the display device 200 may be a TV. In the remote control apparatus 100, a sensing module 110 for recognizing an object is provided. In the present invention, the user may interface the GUI object on the display device 200 through the sensing module 110 of the remote control device 100. 1 illustrates an embodiment of manipulating a GUI object on the display device 200 with the thumb 300. In one embodiment of the present invention, the pad is preferably in the form of having the same aspect ratio as the aspect ratio of the screen of the display device 200.

1 illustrates a one-to-one correspondence between the sensing module 110 and the display device 200, which are the core concepts of the present invention. In detail, the coordinates of one point on the sensing module 110 are coordinated according to a coordinate system in which the upper left corner of the sensing module 110 is the origin and the right direction is the x-axis, and the lower direction is the y-axis. It is expressed as y). The coordinates of one point on the screen are expressed as (X, Y) according to a coordinate system in which the upper left corner of the display device 200 is the origin, the right direction is the X-axis, and the lower direction is the Y axis. The directions of the z-axis and the Z-axis in each coordinate system are defined as directions coming out of the sensing module 110 and directions coming out of the display device 200, respectively. The one-to-one correspondence between the sensing module 110 and the display device 200 is expressed by the following equation using variables of two coordinate systems.

[Equation 1]

은 센싱모듈(110)의 오른쪽 아래 꼭지점의 좌표를 나타내고,

은 TV 화면의 오른쪽 아래 꼭지점의 좌표를 나타낸다. 즉, 사각형 센싱모 듈(110)의 네 꼭지점은 화면의 해당되는 네 꼭지점으로 대응되고 센싱모듈(110) 내부의 점은 선형적으로 비례하여 디스플레이 기기(200)의 내부 점으로 대응된다. here

Represents the coordinates of the lower right corner of the sensing module 110,

Represents the coordinates of the lower right corner of the TV screen. That is, the four vertices of the rectangular sensing module 110 correspond to the corresponding four vertices of the screen, and the points inside the sensing module 110 correspond to the internal points of the display device 200 in a linear proportion.

As shown in FIG. 1, the thumb object A is displayed at a position on the display device 200 corresponding to the position of the thumb 300 on the sensing module 110 to provide an effective visual feedback on the current position of the thumb. do. The thumb 300 is displayed when the thumb 300 is in contact with the sensing module 110 as well as when the thumb 300 is floating on the sensing module 110 without being touched to provide feedback about the position of the thumb 300. Of course, in this case, the thumb is different from the visual characteristics of the object to express the degree away from the sensing module (110). For example, the greater the gap between the thumb 300 and the sensing module 110, the greater the transparency of the thumb object (A) and is not visible if it exceeds a certain limit. Alternatively, while displaying the shadow of the thumb object A semitransparently under the thumb object, the distance between the thumb object A and the shadow may be expressed in proportion to the distance between the thumb 300 and the sensing module 110.

2 is a block diagram showing the internal configuration of the remote control device 100 and the display device 200 according to an embodiment of the present invention. 2 shows an overall system configuration to which the remote control apparatus 100 including the sensing module 110 and the display apparatus 200 are connected. In the embodiment of Figure 2 the remote control device 100 is a remote control, the display device 200 is shown an embodiment of the TV.

2, the remote control apparatus 100 includes a sensing module 110, a touch sensor circuit 120, a microcontroller 130, an input unit 140, and a wireless transmitter 150.

The sensing module 110 is provided with a pad for sensing an object, and detects a position of an object spaced on the pad, detects whether or not the object is in contact with the pad, and detects a position where the object is in contact with the pad. In addition, the object senses the pressure to press the pad.

The sensing module 110 may include a touch sensing unit for detecting whether an object is touched on a pad, a light emitting unit for emitting light, and a light receiving sensor unit for detecting light returned from an object from among light emitted from the light emitting unit. And a pressure sensing unit for sensing a pressure in which the object presses the pad. In the embodiment of FIG. 2, the touch sensing unit is a transparent electrode 111, the light emitting unit is a light emitting diode (LED) matrix 113, and the light receiving sensor 115 is a photo transistor array 115. An embodiment in which the pressure sensing unit is a force sensor 117 is shown.

The touch sensor circuit 120 detects whether an object contacts the pad by using a signal from the transparent electrode 111 and transmits the detected contact to the microcontroller 130.

The microcontroller 130 performs overall control of the remote control apparatus 100, controls the operation of the sensing module 110, and uses a signal output from the sensing module 110 to operate a GUI (Graphical User Interface) operation signal. Create That is, the microcontroller 130 generates an operation signal for displaying the position of the object sensed on the pad in a corresponding area on the screen of the display device 200 using the signal output from the sensing module 110.

In the present invention, the microcontroller 130 may generate an operation signal for displaying the object in the area on the screen corresponding to the position of the detected object when the object is separated within a distance that can be detected from the pad.

In addition to the input through the sensing module 110, the input unit 140 receives an input signal for operating the display device 200 from a user. For example, the input unit 140 may be implemented as a keypad button.

The wireless transmitter 150 transmits an operation signal from the microcontroller 130 to the external display device 220. In one embodiment of the present invention, the wireless transmitter 150 may perform communication using various communication schemes. For example, the wireless transmitter 150 may transmit an operation signal using a communication method such as infrared communication, Bluetooth, or zigbee.

In the present invention, not only a state in which an object is in contact with the pad, but also an object spaced apart from the pad may be recognized and displayed on the screen as an interface object.

That is, as the distance between the pad and the object increases, the microcontroller 130 may indicate the distance between the pad and the object in such a manner as to adjust the transparency so that the corresponding object on the screen is dimly displayed.

Alternatively, when the object is within a detectable distance from the pad, the microcontroller 130 displays the corresponding object and the shadow of the object on the screen, and the distance between the object and the shadow is determined by the distance between the pad and the object. It can be displayed in proportion to the distance.

Alternatively, when an object is located within a detectable distance from the pad, the microcontroller 130 may display the corresponding object and the pad and the distance from the object together on the screen.

In an embodiment of the present disclosure, when an object contacts a pad, the microcontroller 130 may generate an operation signal for adding an outline to a corresponding object on the screen of the display device 200.

In one embodiment of the present invention, when the object is in contact with the pad and a pressure is applied to a predetermined pressure or more, the microcontroller 130 changes an operation signal for changing the color of the corresponding object on the screen of the display apparatus 200 to another color. Can be generated.

3 is a block diagram showing a detailed structure of a sensing module according to an embodiment of the present invention. FIG. 3A is a side cross-sectional view of the sensing module 110, and FIG. 3B is a view of the sensing module 110 viewed from above.

Referring to FIG. 3, the white circle represents the infrared LED 113_a constituting the LED matrix 113 and the black circle represents the photo transistor 115_a constituting the photo transistor array 115. The LEDs 113_a are connected in a matrix manner so that they can be blinked individually, and the photo transistors 115_a are all connected in parallel, and thus operate as a kind of optical sensor having a wide surface. In order to detect the position of the finger, the microcontroller 130 sequentially turns on the LEDs 113_a one at a time, and reads the output voltage of the phototransistor 115_a through the A / D converter input to allow a thumb on the sensing module 110. Acquire an image of 300. Through signal processing of the acquired thumb image, the position of the thumb 300 on the plane and the distance to the sensing module 110 are estimated.

That is, infrared light is emitted from the LED 113_a, and the infrared light reflected by the thumb 300 is returned to the photo transistor 115_a, and the microcontroller 130 reads the signal of the photo transistor 115_a to read the thumb. 300 images are acquired.

4 is a circuit diagram of an LED matrix and a photo transistor array according to an embodiment of the present invention. FIG. 4 (a) shows an example of a specific circuit diagram of the LED matrix, and FIG. 4 (b) shows an example of a specific circuit diagram of the photo transistor array.

Referring to FIG. 4A, both P1-P5 and Q1-Q7 represent digital output pins of the microcontroller 130. By appropriately adjusting these outputs, any LED can be turned on. For example, to light up the LEDs in the third row and the second column, the microcontroller 130 uses P1 = P2 = P4 = P5 = 0, P3 = 1, Q1 = Q3 = Q4 = Q5 = Q6 = Q7 = 1, You can adjust the output to Q2 = 0. In this way, one LED can be turned on one at a time.

Referring to FIG. 4 (b), all the photo transistors are connected in parallel so that all currents through the photo transistors flow through the upper resistor. Therefore, the output signal OUT output through the output pin maintains a value close to the power supply voltage VCC when there is no light, but when light is applied to the photo transistor array 115, the voltage drops in proportion to the amount. As described above, in the present invention, since the photo transistors are connected in parallel, the photo transistor array 115 may not be distinguished from which part of the photo transistor array 115, and the entire photo transistor array 115 may operate as a single planar optical sensor.

5 is a diagram illustrating a thumb object image obtained from the sensing module according to an embodiment of the present invention.

Four images of FIG. 5 show an example of a thumb image obtained from the sensing module 110. 5 (a) shows a state in which the thumb 300 touched the center of the sensing module 110, and FIG. 5 (b) shows a state in which the thumb 300 touched the upper left corner of the sensing module 110. 5 (c) shows the position of the thumb 300 as shown in FIG. 5 (a) but is separated from the sensing module 110 by about 5 mm, and FIG. 5 (d) shows the position of the thumb 300 of FIG. As in (b), but shows a state 5mm away from the sensing module 110. One pixel value on the thumb image represents an output value of the photo transistor array 115 with one LED of the sensing module 110 turned on.

As shown in FIG. 4, since the collector voltage of the photo transistor is an output, the output of the photo transistor array 115 is low when the reflection of LED light is high and the output of the photo transistor array 115 is maximum when the reflection is low. Becomes Since the pixel value is expressed in white when the output of the photo transistor is maximum and black when it is minimum, the color of the portion of the thumb 300 close to the pad is darker than that of other portions in the case of FIG. Was expressed.

In the present invention, the position and distance on the plane of the object spaced apart from the sensing module 110 may be calculated. Hereinafter, a method of calculating a position and a distance on a plane of an object spaced apart from the sensing module 110 will be described in detail.

In the present invention, the microcontroller 130 may process the generated object image to calculate a planar position of the object and a distance from the pad.

6 is a block diagram illustrating a process of calculating a position and a distance of a thumb according to an embodiment of the present invention. 6 shows a process of estimating the position and height of the thumb on the plane from the thumb image.

Referring to FIG. 6, the pixel f (i, j) of the thumb image represents the output of the photo transistor array 115 when the (i, j) th LED on the sensing module 110 is turned on. Where i = 0,1, ..., I-1, j = 0,1 ,,,, J-1, I is the number of columns, J is the number of rows, and I for the embodiment of FIG. = 7 and J = 5. f (i, j) has a maximum value when there is no reflector and a minimum value when the reflector is closest to the (i, j) th LED. The difference between the individual elements of the LED and the photo transistor and the sensing module 110 The difference in distribution between the LED and the phototransistor does not mean that f (i, j) has the same maximum and minimum values for all (i, j). In the present invention, the maximum value f _M (i, j) and the minimum value f _m (i, j) of f (i, j) are determined through experiments, and the thumb image g (i normalized using these values is normalized. , j) can be obtained (S601).

Next, the microcontroller 130 converts the normalized thumb image value to a level required for image processing (S603), obtains the center of gravity and flatness of the converted value (S607), and uses the center of gravity and flatness. The coordinates of the thumb image may be corrected (S609), and the corrected value may be filtered through an adaptive low pass filter (LPF) to calculate a planar position of the object (S617).

In other words,

[Equation 2]

는 0과 1사이의 값을 가진다. 그리고, 정규화된 엄지 이미지

의 픽셀 값을 단위 구간 간의 함수

를 통하여 변환한 값을

로 정의한다. 여기서 함수

는 화상 처리에서 사용하는 감마보정(gamma correction)과 같이

(

)의 형태를 포함하여 다양한 형태를 가질 수 있다. 이 단계의 목적은 엄지 이미지에서 엄지의 끝부분에 해당하는 값을 강조하여 결과적으로 얻어질 엄지의 위치가 엄지의 끝부분에 가깝게 하도록 하기 위함이다. 이렇게 구한

의 무게 중심

을 구한다.By Equation 2 above

Has a value between 0 and 1. And normalized thumb image

Function between pixel intervals of unit intervals

Converted value through

. Where function

Is the same as the gamma correction used in image processing.

(

It can have a variety of forms, including the form of). The purpose of this step is to emphasize the value corresponding to the tip of the thumb in the thumb image so that the resulting thumb position is close to the tip of the thumb. So obtained

Center of gravity

.

&Quot; (3) "

&Quot; (4) "

In an embodiment of the present invention, the microcontroller 130 may normalize the generated thumb image (S601), and estimate the height z of the thumb from the pad plane from the normalized thumb image value (S611).

In the present invention, the height z on the plane of the thumb from the thumb image uses the following equation.

[Equation 5]

는 0에서 1사이의 값을 가지며 엄지가 센싱모듈(110)에 접하여 최대 반사가 LED

위에서 일어 날 때 최대값 1을 가지게 된다. 따라서, z는 0에서 1의 값을 가진다. 즉, 엄지(300)가 센싱모듈(110)에 접했을 때 z는 0에 가까운 값을 가지게 되고 엄지가 센싱모듈(110)에서 충분히 멀어졌을 때 z는 1에 가까운 값을 가지게 되어 근접 정도를 근사적으로 나타내게 된다. 단, 이러한 값은 정확히 물리적 거리에 비례하는 값을 나타내는 것은 아니다.Previously defined

Has a value between 0 and 1 and the thumb is in contact with the sensing module 110 so that the maximum reflection of the LED

It will have a maximum of 1 when it rises from above. Thus, z has a value from 0 to 1. That is, when the thumb 300 is in contact with the sensing module 110, z has a value close to 0, and when the thumb is sufficiently far from the sensing module 110, z has a value close to 1, approximating the proximity. It is shown as an enemy. However, these values do not represent values that are exactly proportional to the physical distance.

는 엄지의 끝부분에 해당하지 않고 손 쪽으로 상당히 이동된 값을 가지게 된다. 이러한 이동을 보정하기 위하여 엄지 이미지의 길이를 나타내는 값을 이용하여

를 보정한다. 엄지 이미지의 길이에 비례하는 여러 가지 가능한 값의 하나로서 다음을 정의한다.In the example of FIG. 5B, the image of the thumb is elongated in the diagonal direction. This phenomenon is not limited only to this embodiment, the thumb will always appear long in this direction if the thumb is placed on the sensing module 110 in the posture holding the remote control device 100. However, the resultant position obtained when using the center of gravity as described above to determine the coordinates of the thumb

Does not correspond to the tip of the thumb and has a value that is significantly shifted toward the hand. To correct this shift, we use a value representing the length of the thumb image

Calibrate As one of several possible values proportional to the length of the thumb image, we define:

&Quot; (6) "

This value is 0 if the image is circular, positive for longer diagonals, and negative for longer diagonals. When the thumb is lying on the sensing module 110 in the state of holding it with the right hand, a long image in a diagonal direction comes down and the value of e has a positive large value. On the other hand, when the thumb is lying on the sensing module 110 in the state of grabbing with the left hand, a long image occurs in the diagonal direction ascending, and the value of e has a negative large value. Reflecting this fact, the coordinates are corrected as in Equation 7, Equation 8 below.

[Equation 7]

[Equation 8]

는 보정의 정도를 나타내는 양의 실수이고 실험에 의하여 결정할 수 있다.here

Is a positive real number indicating the degree of correction and can be determined by experiment.

In an embodiment of the present invention, a parameter of the adaptive low pass filter (LPF) may be determined by measuring the rate of change of the thumb height z. That is, the thumb height z is filtered through the low pass filter (S6123), the change rate is measured by differentiating the filtered values (S615), and the parameters of the adaptive low pass filter are determined using this (S617). .

의 변화가 클 경우에는 반응성의 유지가 떨림의 완화보다 더 중요하므로 저역 통과 필터를 약하게 하고, 반면,

의 변화가 작을 경우에는 반응성보다 떨림의 완화가 더 중요하므로 저역 통과 필터를 강하게 한다. 또한, 엄지(300)가 센싱모듈(110)을 터치하는 순간에 엄지 위치 추정치

를 일정하게 하여 포인팅을 안정화할 필요가 있다. 이러한 목적을 위하여 z의 변화율을 측정하고 z가 급격히 감소하는 순간에는 저역 통과 필터의 강도를 크게 하여

의 변화를 적게 할 수 있다. 종합하면,

의 분산을 측정하고, z의 변화율을 측정하여 저역 통과 필터의 파라미터를 결정하도록 한다. 먼저 z의 변화율을 측정하기 위하여 아래의 저역 통과 필터를 이용하여 z를 필터링하여

를 구한다. That is, the last step of FIG. 6 is an adaptive filter for coping with hand shaking and circuit noise. Simply using a low pass filter for this purpose reduces vibration, but at the same time a time delay occurs, resulting in poor responsiveness of the remote control device 100. To solve this problem, it is necessary to vary the strength of the low pass filter according to the size of the thumb movement. In other words,

If the change in is large, then maintaining the reactivity is more important than mitigating the vibrations, thus weakening the low pass filter.

When the change in is small, the relaxation of the vibration is more important than the reactivity, so the low pass filter is strengthened. In addition, the thumb position estimate when the thumb 300 touches the sensing module 110.

It is necessary to stabilize the pointing by keeping it constant. For this purpose, measure the rate of change of z and increase the strength of the low pass filter at the moment z decreases sharply.

You can make less changes. Overall,

Determine the parameters of the low pass filter by measuring the variance of and measuring the rate of change of z. First, to measure the rate of change of z, filter z using the low pass filter below.

.

[Equation 9]

는 원폴 필터의 폴의 위치를 나타낸다. 연속하는 두

값의 차이로 z의 변화율인 Vz를 계산한다. 다음은 최근 w샘플 동안의 x'_c의 표준 편차인

와, 최근 w샘플 동안의 y'_c의 표준 편차인

를 계산한다. 마지막으로,

를 아래의 저역 통과 필터를 이용하여 필터링하여

를 구한다.here

Denotes the position of the pole of the one-pole filter. Two consecutive

Calculate Vz, the rate of change of z, by the difference in values. The following is the standard deviation of x ' _c over the last w sample.

And the standard deviation of y ' _c over the last w samples

Calculate Finally,

Is filtered using the low pass filter below

.

[Equation 10]

[Equation 11]

Where a _x and a _y are determined as

[Equation 12]

[Equation 13]

는

의 변화율을 반영하는 정도를 나타내는 양의 파라미터이고, ρ는 z의 변화율을 반영하는 정도를 나타내는 양의 파라미터이다.Where a ₀ is a value from 0 to 1 that determines the strength of the overall low pass filter,

Is

Is a positive parameter indicating the degree of reflecting the rate of change of ρ, and ρ is a positive parameter indicating the degree of reflecting the rate of change of z.

,

,

의 일례이다. 세 그림 모두에서 0을 검은색으로, 1을 흰색으로, 그 사이의 값을 그 사이의 회색으로 표시하였다. (b)에서 엄지의 이미지를 선명하게 볼 수 있고, (c)에서는 (b)에 비하여 엄지의 끝부분이 상대적으로 강조된 것을 볼 수 있다.(A), (b) and (c) of FIG. 7 are respectively shown in FIG.

,

,

Is an example. In all three figures, 0 is black, 1 is white, and the values in between are gray. In (b) the image of the thumb can be seen clearly, and in (c) it can be seen that the tip of the thumb is relatively highlighted compared to (b).

8 is a state transition diagram illustrating four states that a pointing device of the present invention may have and transition conditions therebetween. “Off” indicates that the thumb 300 is sufficiently separated from the sensing module 110 to be detected. In the "Hovering" state, the thumb 300 enters the sensing range of the sensing module 110 but is not touched. In the "On" state, the thumb 300 touches the sensing module 110, but the force is greater than a certain level. "Pressed" is a state in which the thumb 300 exerts a predetermined force or more on the sensing module 110. The transition between the "Off" and "Hovering" states is based on the method described in FIG. Determined according to the value of z calculated. The transition between the “Hovering” and “On” states is determined by the output of the touch sensor circuit 120, and the transition between the “On” and “Pressed” states is determined by the output of the force sensor 117.

8, an example of the shape change of the thumb object which is the visualization of a thumb in each state is shown. The red circle represents the thumb and uses shading to express the distance between the thumb 300 and the sensing module 110. That is, the smaller the distance between the thumb 300 and the sensing module 110, the smaller the distance between the red circle and the shadow below it. When the sensing module 110 is touched, the shadow disappears and an outline is added to emphasize the change in the touch state. The red circle turns blue when it transitions to the "Pressed" state.

를 화면의 크기에 맞추어 적절히 선형 맵핑하고 화면의 범위를 벗어나지 않도록 클리핑하여 구한다. 즉,9 is an example of a TV user interface to be used with the pointing device of the present invention. The broadcasting menu is arranged along the lower side, the volume control bar is arranged along the right side, and the playback control bar of the current video is arranged on the upper side to move the thumb 300 along the periphery of the sensing module 110. It is easy to operate. In the embodiment of FIG. 9, the movie list is listed by selecting the “movie” button. Currently, the thumb object A is floating on the list, which shows a situation where the list can be dragged down by touching. The position (X, Y) of the thumb object on the screen is calculated by the method described in FIG.

Obtain a linear mapping appropriately to the size of the screen and clipping to keep it out of range of the screen. In other words,

[Equation 14]

[Equation 15]

와

는 화면의 가로 및 세로 크기를 나타내고,

와

은 화면의 좌단과 우단에 대응하는

의 값,

와

은 화면의 상단과 하단에 대응하는

의 값으로서 실험적으로 정한다.here

Wow

Indicates the horizontal and vertical size of the screen,

Wow

Corresponds to the left and right edges of the screen

, The value of,

Wow

Corresponds to the top and bottom of the screen

It is determined experimentally as the value of.

While the invention has been described using some preferred embodiments, these embodiments are illustrative and not restrictive. Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the invention and the scope of the rights set forth in the appended claims.

1 is a view showing a remote control device and a screen according to an embodiment of the present invention.

Figure 2 is a block diagram showing the internal configuration of a remote control device and a TV according to an embodiment of the present invention.

3 is a block diagram showing a detailed structure of a sensing module according to an embodiment of the present invention.

4 is a circuit diagram of an LED matrix and a photo transistor array according to an embodiment of the present invention.

5 is a diagram illustrating a thumb object image obtained from the sensing module according to an embodiment of the present invention.

6 is a block diagram illustrating a process of calculating a position and a distance of a thumb according to an embodiment of the present invention.

7 is a diagram illustrating a thumb image according to the calculation process of FIG. 6.

8 is a state transition diagram of a sensing module according to an embodiment of the present invention.

9 is an example TV screen according to an embodiment of the present invention.

* Description of the symbols for the main parts of the drawings *

100 remote control unit 200 screen

300 thumb 110 sensing module

120 touch sensor circuit 130 microcontroller

140 Input 150 Wireless Transmitter

111 Transparent Electrode 113 LED Matrix

115 Phototransistor Array 117 Force Sensors

210 Wireless Receiver 220 GUI

Claims (27)

In the remote control device for remotely controlling a display device including a screen, A pad is provided for sensing an object, and detects a position of an object spaced on the pad, detects whether an object is contacted on the pad, detects a position where the object is in contact with the pad, Sensing module for sensing the pressure to press the pad; A microcontroller for general control of the remote control device, controlling an operation of the sensing module, and generating an operation signal for a graphical user interface (GUI) using a signal output from the sensing module; And It includes a wireless transmission unit for transmitting the operation signal to an external display device, The sensing module may include a touch sensing unit for detecting whether an object contacts the pad, a light emitting unit for emitting light, and a light reflected from an object from among the light emitted from the light emitting unit. And a light receiving sensor unit and a pressure sensing unit for sensing a pressure of an object pressing the pad. delete Claim 3 was abandoned when the setup registration fee was paid. The method of claim 1, And a touch sensor circuit unit configured to detect whether or not an object contacts the pad by using the signal from the touch detector and transmit the detected touch to the microcontroller. Claim 4 was abandoned when the registration fee was paid. The method of claim 1, And the touch sensing unit is a transparent electrode. The method of claim 1, The light emitting unit is a remote control device, characterized in that the LED (Light Emitting Diode) matrix. The method of claim 1, And the light receiving sensor unit is a photo transistor array. Claim 7 was abandoned upon payment of a set-up fee. The method of claim 1, And the pressure sensing unit is a force sensor. The method of claim 1, And the pad is configured to have an aspect ratio equal to an aspect ratio of the screen. The method of claim 8, Each area of the pad and each area of the screen correspond to the one-to-one correspondence. The method of claim 1, And the microcontroller generates an operation signal for displaying a position of an object sensed on the pad in a corresponding area on the screen by using the signal output from the sensing module. The method of claim 1, And when the object is within a detectable distance from the pad, the microcontroller generates an operation signal for displaying the object in an area on the screen corresponding to the position of the detected object. The method of claim 11, And the microcontroller adjusts the transparency of the corresponding object on the screen to be faintly displayed as the distance between the pad and the object increases. The method of claim 11, The microcontroller displays a corresponding object and a shadow of the corresponding object on the screen when the object is separated within a detectable distance from the pad, and displays the distance between the object and the shadow from the pad and the object. Remote control device characterized in that the display proportional to the distance to. The method of claim 11, And when the object is within a detectable distance from the pad, the microcontroller displays the corresponding object on the screen and the distance between the pad and the object together. The method of claim 1, And when the object comes into contact with the pad, the microcontroller generates an operation signal for adding an outline to a corresponding object on the screen. The method of claim 1, And the microcontroller generates an operation signal for changing a color of the corresponding object on the screen to another color when an object comes into contact with the pad and a pressure is applied. The method of claim 5, The microcontroller is a remote control device, characterized in that for controlling each of the LEDs constituting the LED matrix sequentially turned on one at a time. The method of claim 6, And the microcontroller reads an output voltage of the photo transistor array to generate an image of an object spaced apart on the pad. The method of claim 18, And the microcontroller is configured to signal-process the generated object image to calculate an on-plane position of the object and a distance from the pad. The method of claim 19, The object is a thumb, The microcontroller normalizes the generated thumb image and estimates the height z of the thumb from the pad plane from the normalized thumb image value. 21. The method of claim 20, The object is a thumb, The microcontroller normalizes the generated thumb image, converts the normalized thumb image value to a level necessary for image processing, obtains a center of gravity and ellipticity of the converted value, and uses the center of gravity and flatness And correcting the coordinates of the thumb image and filtering the corrected values through an adaptive low pass filter to calculate the planar position of the object. The method of claim 21, And determine a parameter of the adaptive low pass filter by measuring the rate of change of the thumb height z. The method of claim 22, Measuring the rate of change of the thumb height z, And the thumb height z is filtered through a low pass filter, and the change rate is measured by differentially filtering the filtered values. Claim 24 is abandoned in setting registration fee. The method of claim 1, The wireless transmitter is a remote control device, characterized in that for transmitting a signal using infrared communication. Claim 25 is abandoned in setting registration fee. The method of claim 1, The wireless transmitter is a remote control device, characterized in that for transmitting a signal using Bluetooth. Claim 26 is abandoned in setting registration fee. The method of claim 1, The wireless transmitter is a remote control device, characterized in that for transmitting a signal using a zigbee (zigbee). The method of claim 1, And an input unit configured to receive an input signal for operating the display device from a user.

Images