KR20140037741A - A glove apparatus for hand gesture cognition and interaction, and therefor method - Google Patents

Abstract

A glove apparatus for hand gesture recognition and interaction according to the present invention is characterized by including a lighting unit including k number (k is a natural number equal to or greater than 1) of lighting units for irradiating n number of optical signals (n is a natural number equal to or greater than 1) spatially-split in mutually different patterns, a glove marker unit having the shape of a glove and having m number of markers (m is a natural number equal to or greater than 1) that are attached to a plurality of areas of the glove to detect the optical signals, respectively, irradiated from the lighting unit, and a location information outputting unit for recognizing information of locations of the markers based on the optical signals received from the glove markers and for wirelessly transmitting the information of the location of each recognized marker to the display device. [Reference numerals] (100) Lighting module; (200) Glove marker module; (300) Location information outputting module; (400) Haptic function performing module

Description

A glove apparatus for hand gesture recognition and interaction, and therefor method}

The present invention relates to a technology for recognizing hand gesture position information according to markers attached to a glove device, and providing position information of the recognized hand gesture to a display device. The present invention corresponds to an invention supported by a national research and development project as follows.

[Assignment number] 2N34990

[Ministry of Education] Ministry of Education, Science and Technology

[Research Management Specialized Organization] Realistic Exchange Human Response Solution Research Group

[Name of Research Project] Original Technology Development Project (Global Frontier R & D Project)

[Project name] Development of 4D + mirror world generation technology that can interact with objects

[Host] Korea Institute of Science and Technology

[Research Period] 2011.08.23 ~ 2012.08.31.

Recently, with the development of IT devices, there is an increasing demand for technology to recognize the position of a body or a specific object in real time and use it as a means of interaction with digital information. For example, KINECT, which was recently released by Microsoft, is a device that recognizes user's three-dimensional movements and plays games through interaction with digital contents. Also, as introduced in the Hollywood movie "Minority Report", next-generation technology that manipulates digital information on the screen through hand movements in the air has been commercialized by Oblong under the name of G-Speak. In the future, it is difficult to manipulate complex digital information in the screen with a conventional remote controller in addition to the development of 3D TV and IP TV, and it is becoming increasingly necessary to manipulate the information through the movement of the user from a remote place.

However, conventionally developed techniques have limitations in terms of price-to-location performance. G-Speak uses multiple expensive infrared cameras to measure the position of the infrared marker attached to the hand. The measurement accuracy is high but tens of millions of equipment are required. KINECT uses an inexpensive Time of Flight (TOF) 3D measurement camera, but its measurement accuracy is low in cm, and the measurement performance of small objects such as hands is far worse. In other words, the conventional techniques based on cameras require that a camera be expensive to achieve high performance (high resolution, high speed) because the performance of object positioning is determined by the camera performance (pixel resolution, 3D measurement resolution). There is a problem that the price of the equipment rises together.

The problem to be solved by the present invention is to wear a device in the form of a glove on the hand, to recognize the hand gesture position information according to the position of the markers (Passive Markers) attached to the glove, and to provide the position information of the recognized hand gesture to the display device In addition, the present invention relates to a glove apparatus and method for hand gesture recognition and interaction that provides a tactile sensing function to a user through a haptic element according to a response signal from a display device according to hand gesture position information.

In order to solve the above problems, the glove device for hand gesture recognition and interaction according to the present invention is mounted on the display device, and the n (n is a natural number greater than or equal to 1) optical signals divided into spaces in different patterns An illumination unit having k lighting units (k is a natural number greater than or equal to 1) each irradiated; A glove marker having a glove shape and having m markers (m is a natural number greater than or equal to 1) for detecting the optical signals irradiated from the illumination unit, respectively, attached to a plurality of areas of the glove; And a location information output unit for recognizing location information of each of the markers from the optical signals received from the glove marker unit, and wirelessly transmitting the location information of each of the recognized markers to the display device. do.

The lighting units are each provided with n lighting elements, and the lighting elements each include a light source and a space dividing element.

The spatial division element has a light transmission region and a light blocking region which are divided into barcode patterns at regular intervals.

The illumination elements are characterized in that to irradiate a plurality of optical signals that are each divided in a different pattern at the same time.

The lighting units are characterized by irradiating a plurality of optical signals sequentially with each other.

The illumination elements are characterized in that they comprise an illumination polarization filter having a different polarization direction.

The illumination elements are characterized in that they comprise an illumination bandpass filter for filtering different wavelength bands.

In order to recognize the two-dimensional position, at least one pair of the lighting unit is characterized in that the perpendicular to each other are mounted on the display device.

In order to recognize the three-dimensional position, at least two pairs of the lighting units are mounted to the display device at right angles to each other.

The lighting units are mounted to corner portions in the frame of the display device.

The markers may each include n photodiodes for receiving the optical signals.

The markers may include n marker polarization filters each having a different polarization direction.

The markers may each include n marker bandpass filters that filter different wavelength bands.

The markers may each include a camera module for receiving the optical signals.

The camera module may be equipped with n camera polarization filters having different polarization directions in an image sensor for receiving the optical signals.

The camera module is equipped with n camera bandpass filters for filtering different wavelength bands in the image sensor for receiving the optical signals.

 The location information output unit may include: a memory configured to store table information between binary codes and location coordinate values corresponding to the optical signals previously prescribed; A position calculation module for calculating position information of each of the markers provided by the glove marker unit; A wireless transmission module for wirelessly transmitting the location information calculated by the location calculation module to the display device; A control module for controlling the operation of the position calculating module and the wireless transmission module; And a power supply module for supplying power for driving the position calculation module, the wireless transmission module, and the control module.

The position calculation module may calculate any one of each of two-dimensional position information and three-dimensional position information of the markers.

The glove device for hand gesture recognition and interaction may further include a haptic operation unit that performs a tactile recognition operation in response to a response signal from the display device according to the transmission of the position information.

The display apparatus may include any one or more of a digital TV, a desktop PC, a tablet PC, a notebook computer, and a smartphone.

In order to solve the above problems, the method for hand gesture recognition and interaction according to the present invention is mounted on a display device and irradiates n optical signals each of which is spatially divided into different patterns (n is a natural number greater than or equal to 1). Illuminating each of the optical signals by an illumination unit having k (k is a natural number greater than or equal to 1) lighting units; Detecting each of the optical signals by a glove marker having a glove shape and having m markers (m is a natural number greater than or equal to 1) for detecting the optical signals irradiated from the illumination unit; And a location information output unit recognizing location information of each of the markers from the optical signals received from the glove marker unit, and wirelessly transmitting the location information of each of the recognized markers to the display device. It features.

In the irradiating the optical signals, the n lighting elements provided in each of the lighting units are irradiated simultaneously with a plurality of optical signals that are spatially divided in different patterns.

In the step of irradiating the optical signals, the illumination units are characterized in that to sequentially irradiate a plurality of optical signals with each other.

Recognizing the location information of each of the markers, characterized in that to recognize any one of each of the two-dimensional location information and three-dimensional location information for the markers.

The method for hand gesture recognition and interaction may further include a haptic performer performing a tactile recognition operation in response to a response signal from the display apparatus according to the transmission of the location information.

According to the present invention, the user can easily perform control of a display device such as a TV by recognizing a hand gesture and providing an external signal to the user through a haptic element at a relatively cheaper price than existing technologies.

That is, the present invention uses a photodiode array or a low-resolution high-speed camera, and irradiates a unique optical signal according to spatial position coordinates and recognizes it as a photodiode array or a camera. You can get it.

1 is a block diagram of a glove device for hand gesture recognition and interaction according to the present invention.
2 is a schematic reference diagram of any one of the lighting units of the lighting unit constituting the lighting unit.
FIG. 3 is a reference view showing detailed components of the lighting unit shown in FIG. 2.
4 is a reference diagram illustrating the components of any one lighting device constituting the lighting unit shown in FIG.
FIG. 5 is a reference diagram illustrating a pattern of spatial division elements P ₁ , P ₂ ,..., P _n shown in FIG. 2.
6 is a reference diagram illustrating an example of lighting units mounted on a display device.
7 is a reference diagram illustrating a glove marker unit and a location information output unit.
FIG. 8 is a reference view showing detailed components of any one of the markers constituting the glove marker unit.
FIG. 9 is a reference diagram of an embodiment of a marker element constituting the marker shown in FIG. 8.
FIG. 10 is a reference diagram showing another detailed component of the marker shown in FIG. 8.
FIG. 11 is a block diagram illustrating detailed components of the location information output unit illustrated in FIG. 1.
FIG. 12 is a reference diagram illustrating optical signals emitted through the space dividing elements illustrated in FIG. 5.
FIG. 13 is a reference diagram illustrating optical signal sensing at an armor marker unit for optical signals emitted through the spatial dividing elements illustrated in FIG. 12.
14 is a flowchart of an embodiment for explaining a method for hand gesture recognition and interaction according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, like reference numerals refer to like elements, and the size and thickness of each element may be exaggerated for clarity of explanation.

1 is a block diagram illustrating a glove device for hand gesture recognition and interaction according to the present invention, and includes an illumination unit 100, a glove marker unit 200, a location information output unit 300, and a haptic performing unit 400. .

The lighting unit 100 is mounted on the display device, and k (k is a natural number greater than or equal to 1) lights irradiating n (n is a natural number greater than or equal to 1) optical signals that are spatially divided into different patterns. Units are provided. That is, the lighting unit 100 is composed of the lighting unit 1 to the lighting unit k (I ₁ , I ₂ , ..., I _k ).

2 is a schematic reference diagram of any one of the lighting units of the lighting unit constituting the lighting unit. A lighting unit (I ₁₎ are each of the n number of illumination elements to each other, emitting an optical signal the space division in a different pattern comprises _{(I 11, I 12, ...} , I 1n), the illumination elements are, respectively, a light source and a spatial division elements It includes.

FIG. 3 is a reference view showing detailed components of the lighting unit shown in FIG. 2. The n illuminating device 3, the forming the light unit _{(I 1) (I 11,} I 12, ..., I 1n) is the n light sources that emit light _{(D 1, D 2, ...} , D _n ) and n spatial division elements P ₁ , P ₂ ,..., P _n may be arranged in pairs. In addition, it includes the n one trillion people polarization filters or light band-pass filter _{(F 1, F 2, F} 3, ... F n).

FIG. 4 is a reference diagram illustrating components of any one lighting element I ₁₁ constituting the lighting unit illustrated in FIG. 3. As shown in FIG. 4, a light source D ₁ that emits light, an illumination polarization filter F _1P or an illumination band pass filter F _1B , and a spatial division disposed on the light exit side of the light source D ₁ . It is composed of elements (P _1).

The n light sources comprising a light source _{(D 1) (D 1,} D 2, ..., D n) are respectively light-emitting diodes: may be composed of a light emitting element that emits light, such as (light emitting diode LED). The n light sources D ₁ , D ₂ ,..., and D _n may emit light sources of different wavelength bands λ ₁ , λ ₂ , λ ₃ ..., λ _n , respectively.

The n one trillion people polarizing filter comprising a light polarizing filter (F _{1P) (1P} F, F _{2P, 3P} F, ... F _nP), respectively, so the optical filter for passing only the linearly polarized light in a specific direction, a light source (D ₁ The light of the broadband emitted from) is filtered by light having a specific polarization while passing through the illumination polarization filter (F _1P ).

On the other hand, the illumination element I ₁₁ replaces the illumination bandpass filters F _1B , F _2B , F _3B , ... F instead of the illumination polarization filters F _1P , F _2P , F _3P , ... F _nP . _nB ) may be provided. At this time, since the illumination band-pass filter comprising a lighting-pass filter _{(F 1B) (F 1B,} F 2B, F 3B, ... F nB) corresponds to an optical filter which only a specific wavelength band as the pass band, the light source Light emitted from (D ₁ ) is filtered to light of a specific wavelength band through the illumination bandpass filters (F _1B ).

The n space dividing elements P ₁ , P ₂ ,..., P _n may be formed by, for example, printing a space dividing pattern on a transparent film. The spatial dividing elements P ₁ , P ₂ ,..., P _n are arranged on the light exit side of the light sources D ₁ , D ₂ ,..., D _n , respectively, so that the light sources D ₁ , D ₂ ,. The light emitted from D _n ) is spatially divided into different patterns.

FIG. 5 is a reference diagram illustrating a pattern of spatial division elements P ₁ , P ₂ ,..., P _n shown in FIG. 2. Referring to FIG. 5, the first space dividing element P ₁ has a one-dimensional barcode pattern obtained by dividing a space in one direction. At this time, the white region becomes the light transmission region and the black region becomes the light blocking region among the divided regions. Similarly, the second space dividing element P ₂ has a one-dimensional barcode pattern obtained by dividing the space three times in one direction, and the third space dividing element P ₃ has a one-dimensional barcode pattern obtained by dividing the space four directions in one direction. The dividing element P _n has a one-dimensional barcode pattern obtained by dividing a space n + 1 in one direction. The minimum spacing of the space dividing elements P ₁ , P ₂ ,..., P _n (that is, the spacing of the barcode patterns of the _nth space dividing element P _n ) results in determining the resolution of the position to be detected. . Therefore, if the position tracking is to be more precise, the minimum spacing of the spatial dividing elements P ₁ , P ₂ ,..., P _n may be made smaller. The one-dimensional barcode patterns of the space dividing elements P ₁ , P ₂ ,..., P _n are examples described for convenience of description, but are not limited thereto. For example, the pattern of the space dividing elements P ₁ , P ₂ ,..., P _n may have a two-dimensional barcode pattern in which the space is divided in two directions.

This is irradiated by the illumination unit (I _1), each of the illumination elements (I _11, I _12, ..., _1n I) are each a plurality of optical signals with different patterns at the same time the space division gloves marker 200. Meanwhile, the k lighting units I ₁ , I ₂ ,..., I _k sequentially irradiate a plurality of optical signals to the glove marker unit 200. That is, the plurality of lighting elements in the unit lighting unit simultaneously emits light, thereby outputting a plurality of optical signals that are spatially divided in different patterns, and between k lighting units I ₁ , I ₂ ,..., I _k . In irradiating light signals sequentially from the lighting unit 1 (I ₁ ) to the lighting unit k (I _k ), which are not simultaneously with each other, to the armor marker unit 200.

The lighting unit 100 is mounted on the display device. Here, the display device corresponds to a digital TV, a desktop PC, a tablet PC, a notebook or a smartphone. In particular, the lighting unit 100 may be mounted on the display device with at least one pair of lighting units at right angles to each other for two-dimensional position recognition, and for at least two pairs of the lighting units for each other for three-dimensional position recognition. It may be mounted on the display device at a right angle.

6 is a reference diagram illustrating an example of lighting units mounted on a display device. As shown in FIG. 6, lighting units 1 to 4 (I ₁ , I ₂ , I ₃ , and I ₄ ) corresponding to the components of the lighting unit 100 are respectively mounted to corner portions within the frame of the display device. You can check it.

For two-dimensional position recognition, a pair of lighting units, that is, lighting units 1 and 2 (I ₁ , I ₂ ), are mounted on the frame of the display device at right angles to each other, or lighting units 3 and 4 (I _3). , I ₄ ) are mounted on the frame of the display device at right angles to each other. Recognizing the two-dimensional position means that by placing at least two lighting units at right angles to each other, the position of each marker bent in the glove marker unit 200 can be confirmed by two-dimensional coordinates in the two-dimensional space. do.

Meanwhile, for three-dimensional position recognition, two pairs of lighting units, that is, lighting units 1 and 2 (I ₁ , I ₂ ) and lighting units 3 and 4 (I ₃ , I ₄ ) are respectively placed on the frame of the display device. They are mounted at right angles to each other. By arranging at least four lighting units that recognize three-dimensional positions at right angles, it means that the positions of the respective markers bent in the glove marker unit 200 can be confirmed in three-dimensional coordinates in three-dimensional space.

The glove marker unit 200 has a glove shape, and attaches m markers (m is a natural number greater than or equal to 1) to each of the plurality of regions of the glove, respectively detecting the optical signals emitted from the illumination unit 100. have.

FIG. 7 is a reference diagram illustrating a glove marker unit and a location information output unit. As shown in FIG. 7, the glove marker unit has a glove form that can be worn on a hand. The glove marker unit 200 includes m markers M ₁ , M ₂ ,..., M _m , and as shown in FIG. 7, markers 1 to 5 (M ₁ , M ₂ , M). ₃ , M ₄ , M ₅ ) are attached to the finger portion of the glove, respectively.

Each of the m markers M ₁ , M ₂ ,..., M _m has marker polarization filters together with photodiodes for receiving n optical signals irradiated simultaneously from any one lighting unit of the lighting unit 100. Or marker bandpass filters.

FIG. 8 is a reference diagram illustrating detailed components of any one marker 1 (M ₁ ) of markers constituting the glove marker unit. As shown in FIG. 8, marker 1 (M ₁ ) includes n photodiodes PD ₁ , PD ₂ , PD ₃ ,..., PD _n ) and n marker polarization filters or marker bandpass filters. (F ₁ , F ₂ , F ₃ , ..., F _n ).

FIG. 9 is a reference diagram of an embodiment of a marker element M ₁₁ constituting marker 1 M ₁ shown in FIG. 8. The marker element M ₁₁ includes a marker polarization filter F _1P having different polarization directions or a marker band pass filter F _1B for filtering different wavelength bands.

Marker polarizing filter (F _1P) n the markers polarizing filter that includes _{(F 1P, F 2P, F} 3P, ..., F nP) are illuminated with polarized light filter shown in Fig. 3 (F _1, F _2, A filter having a polarization direction corresponding to the polarization direction of F ₃ ,..., F _n ), and filters only an optical signal of a corresponding polarization direction among the incident optical signals and transmits the optical signals to the photodiodes. On the other hand, the marker element M ₁₁ is a marker band pass filters F _1B , F _2B , F _3B ,... Instead of marker polarization filters F _1P , F _2P , F _{3 ,.} , F _nB ). Since the marker bandpass filters F _1B , F _2B , F _3B ,..., And F _nB are optical filters having only a specific wavelength band as a pass band, among the incident optical signals Only optical signals of the corresponding wavelength band are filtered and transmitted to the photodiodes.

N photodiodes of the photodiode _{(PD 1, PD 2, PD} 3, ..., PD n) is to receive each of the n optical signals to be irradiated at the same time from any one of the lighting units, each comprising a (PD ₁₎ Corresponds to the optical sensor for The photodiodes PD ₁ , PD ₂ , PD ₃ ,..., PD _n respectively sense the optical signal filtered by the marker polarization filters or the marker bandpass filters.

FIG. 10 is a reference diagram showing another detailed component of the marker M ₁ shown in FIG. 8. As shown in FIG. 10, the marker M ₁ includes a camera module for simultaneously receiving n optical signals. The camera module refers to a module that detects an optical signal coming through an image sensor (eg, a CCD). Such a camera module is provided in m markers M ₁ , M ₂ ,..., M _m , respectively. As shown in FIG. 10, a camera module provided in each of the _m markers M ₁ , M ₂ ,..., And M _m has different polarizations in the image sensor to receive n different optical signals. N camera bandpass filters F _1B , F _2B equipped with n camera polarization filters F _1P , F _2P , F _{3 ,.} , F _3B , ..., F _nB ) are mounted. Accordingly, the m camera modules respectively detect optical signals emitted from the illumination unit 100 according to the corresponding polarization direction or wavelength band.

The glove marker unit 200 detects optical signals irradiated from the lighting unit 100 and transmits the detected optical signals to the location information output unit 300 connected in a wired manner.

The location information output unit 300 recognizes the location information of each of the _m markers M ₁ , M ₂ , ..., M _m from the optical signals received from the armor marker 200, and recognizes the location information. Each location information is wirelessly transmitted to the display device.

FIG. 11 is a block diagram illustrating detailed components of the location information output unit illustrated in FIG. 1, and includes a memory 310, a location calculation module 320, a wireless transmission module 330, a control module 340, and a power supply module. 350).

The memory 310 stores table information between binary codes and position coordinate values corresponding to the optical signals, which have been previously regulated.

The position calculation module 320 calculates the position information of each of the markers provided by the glove marker unit 200 with reference to the table information stored in the memory 310, and outputs the calculated result to the wireless transmission module 320. do.

Specific functions of the position calculation module 320 will be described with reference to FIGS. 12 and 13.

FIG. 12 is a reference diagram illustrating optical signals irradiated through the space dividing elements P ₁ , P ₂ ,..., P _n shown in FIG. 5, and FIG. 13 is a view of the space dividing elements shown in FIG. 12. P ₁ , P ₂ ,..., P _n ) is a reference diagram illustrating the detection of an optical signal at any one marker of the armor marker unit 200 with respect to the optical signals irradiated through P ₁ , P ₂ ,.

First, the illumination unit I ₁ simultaneously irradiates a plurality of optical signals that are spatially divided into different patterns. That is, the illumination unit I ₁ has a different wavelength band and a plurality of optical signals L ₁₁ , L ₁₃ ,…, L _1n spatially divided into different patterns P ₁ , P ₂ ,..., P _n . Irradiate simultaneously with the glove marker unit 200.

Since the optical signals irradiated simultaneously from the illumination unit I ₁ are spatially divided into different patterns, the markers M ₁ , M _{2 ,.} The position coordinate value of the attached finger can be converted.

When 12 and 13, a plurality of optical signals from the lighting unit _{(I 1) (L 11,} L 13, ..., L 1n) is irradiated at the same time, a plurality of light irradiated from the lighting unit (I ₁₎ The signals L ₁₁ , L ₁₃ ,..., L _1n simultaneously detect a plurality of optical signals L ₁₁ , L ₁₃ ,..., L _1n at the first marker M ₁ . In this case, the first optical signal L ₁₁ is irradiated with the barcode pattern of the first spatial dividing element P ₁ , and thus the first optical signal L ₁₁ is not detected where the first marker M ₁ is located. That is, where the first marker M ₁ is located, the first optical signal L ₁₁ will be detected as a "0" signal. In addition, the second optical signal L ₁₂ irradiated with the barcode pattern of the second space dividing element P ₂ will be detected as a "0" signal where the first marker M ₁ is located. However, the third optical signal L ₁₃ irradiated with the barcode pattern of the third space dividing element P ₃ will be detected as a "1" signal at the first marker M ₁ . In the same way, the first marker (M ₁₎ are each of a plurality of optical signal spatial division in a different pattern, and detecting the _{(L 11, L 13, ...} , L 1n), the detected light signal (L _11, L ₁₃ ,..., L _1n ) are transmitted to the location information output unit 300. Since the plurality of optical signals L ₁₁ , L ₁₃ ,..., L _1n are spatially divided into different patterns, if the position of the first marker M ₁ is moved, the plurality of optical signals L ₁₁ detected are moved. , L ₁₃ ,…, L _1n ) will be changed. Therefore, the signal values of the plurality of detected optical signals L ₁₁ , L ₁₃ ,..., L _1n have position information of the first marker M ₁ . Therefore, the location information output unit 300 processes the signal values of the plurality of optical signals L ₁₁ , L ₁₃ ,..., L _1n detected by the first marker M ₁ , and then executes a binary code (for example, 00100). ), And calculates the position coordinate value of the first marker M ₁ from a relational expression between the predefined binary codes and the position coordinate values or a lookup table using the generated binary code.

In the same way, the position calculation module 320 is m-1 markers (M ₂ ) for the plurality of optical signals L ₁₁ , L ₁₃ ,..., L _1n irradiated from the illumination unit I ₁ . , ..., M _m ) to calculate the position information of each marker (M ₂ , ..., M _m ) by binary coding the detected optical signals for each and comparing it with a reference table of position coordinate values. do. Thereafter, the position calculation module 320 respectively detects the detected optical signals of the glove marker unit 200 with respect to the optical signals sequentially irradiated from the remaining k-1 illumination units I ₂ ,..., I _k . by comparing the binary coded, and this, and the reference table of the position coordinate values, respectively, calculating the position information of the lighting units of each marker according to _{(I 2, ..., I k} ) (M 2, ..., M m) .

The position calculation module 320 calculates respective two-dimensional position information or three-dimensional position information for the markers. That is, as shown in Figure 6, a pair of lighting units, that is, lighting units 1 and 2 (I ₁ , I ₂ ) are mounted on the frame of the display device at right angles to each other, or lighting units 3 and 4 When (I ₃ , I ₄ ) are mounted on the frame of the display device at right angles to each other, when the optical signals irradiated from the pair of two lighting units are respectively received from the armor marker unit 200, The position calculation module 320 may check the position of each marker in two-dimensional coordinates in a two-dimensional space. Meanwhile, two pairs of lighting units, that is, lighting units 1 and 2 (I ₁ , I ₂ ) and lighting units 3 and 4 (I ₃ , I ₄ ) are mounted on the frame of the display device at right angles to each other. On the other hand, when receiving the light signals respectively irradiated from the pair of four lighting units from the armor marker unit 200, the position calculation module 320 is a three-dimensional coordinates on the three-dimensional space for the position of each marker You can check with

The wireless transmission module 330 wirelessly transmits the position information of each marker calculated by the position calculation module 320 to the display apparatus. To this end, the wireless transmission module 330 is provided with a wireless transmission and reception signal sensor for the wireless transmission of location information.

The control module 340 controls the operations of each of the memory 310, the location calculation module 320, and the wireless transmission module 330.

The power supply module 350 supplies power for driving the position calculation module 320, the wireless transmission module 330, and the control module 340.

The display apparatus transmits a response signal in the display apparatus according to the position recognition to the glove apparatus for hand gesture recognition and interaction in response to the position information transmitted from the position information output unit 300. Then, the haptic performing unit 400 provided with the glove marker unit 200 performs a tactile perception operation in response to the response signal from the display device according to the transmission of the position information. The tactile perception motion refers to an operation that allows the user to feel the touch by generating vibration, force, or impact on the glove device. For example, when the user performs an interaction operation of touching a virtual ball, when the haptic performer 400 receives a displacement value in the form of a ball, the haptic performer 400 may determine a tactile module that is proportional to the displacement amount. The mechanical value is transmitted to the user through the current value, giving the user a feeling similar to touching a real ball.

14 is a flowchart of an embodiment for explaining a method for hand gesture recognition and interaction according to the present invention.

The lighting unit having k (k is a natural number greater than or equal to 1) light units each of which is mounted on the display device and irradiates n (n is a natural number greater than or equal to 1) optical signals that are spatially divided into different patterns. Each of the optical signals is irradiated (operation 400).

Each lighting unit constituting the lighting unit includes n lighting elements that emit optical signals that are spatially divided in different patterns, and the lighting elements each include a light source and a space dividing element. In addition, it includes n illumination polarization filters or n illumination bandpass filters. The n light sources irradiate light sources of different wavelength bands, respectively. Since the n illumination polarization filters are optical filters that pass only linearly polarized light in a specific direction, only the specific polarized light is filtered out for the light emitted from the light source. On the other hand, the illumination elements may be provided with n illumination bandpass filters instead of the illumination polarization filters, respectively, since the illumination bandpass filters correspond to optical filters having only a specific wavelength band as a pass band, and thus are specific to light emitted from the light source. Filter by the light of the wavelength band and output. The n space dividing elements are respectively disposed on the light exit sides of the n light sources to spatially divide the light emitted from the light sources into different patterns.

The n illumination elements constituting the illumination unit irradiate n optical signals spatially divided in different patterns with the armor marker portion at the same time. On the other hand, the k lighting units are irradiated with a glove marker unit a plurality of light signals sequentially from each other. That is, the plurality of lighting elements in the unit lighting unit is irradiated with light at the same time, and outputs a plurality of optical signals that are spatially divided in different patterns, and between the lighting unit k from the lighting unit 1 that is not simultaneously with each other between the k lighting units In order to irradiate the optical signal to the armor marker portion sequentially.

After operation 400, an armored marker unit having m (m is a natural number greater than or equal to 1) markers each having the shape of a glove and detecting the optical signals emitted from the illumination unit detects the optical signals, respectively. Step 402).

Each of the m markers includes n marker polarization filters or n marker bandpass filters together with n photodiodes for receiving n optical signals irradiated simultaneously from either illumination unit of the illumination unit.

The n marker polarization filters filter only optical signals of a corresponding polarization direction among the incident optical signals and transmit them to the photodiodes. Meanwhile, n marker bandpass filters may be provided in place of the marker polarization filters. The marker bandpass filters filter only the optical signal of a corresponding wavelength band among the incident optical signals and transmit them to the photodiodes.

Each of the n photodiodes is an optical sensor for receiving n optical signals irradiated from any one lighting unit at the same time, and detects optical signals filtered by marker polarization filters or marker bandpass filters, respectively.

On the other hand, m markers may be provided with a camera module, respectively, instead of having photodiodes, the camera module is provided in each of the m markers. Each camera module included in m markers is equipped with n camera polarization filters having different polarization directions in the image sensor to receive n different optical signals, or n cameras filtering different wavelength bands. Band pass filters are mounted, and accordingly, optical signals emitted from the illumination unit are detected by filtering according to a corresponding polarization direction or wavelength band.

The armor marker unit detects optical signals irradiated from the lighting unit, and transmits the detected optical signals to the location information output unit.

After operation 402, the location information output unit recognizes location information of each of the markers from the optical signals received from the glove marker unit, and wirelessly transmits location information of each of the recognized markers to the display apparatus ( Step 404).

The position information output unit calculates position information of each of the markers provided by the glove marker unit with reference to the table information between the binary codes corresponding to the optical signals previously prescribed and the position coordinate values.

Since the optical signals irradiated simultaneously by any one lighting unit are spatially divided into different patterns, the position information output unit may convert the position coordinate values of the fingers to which the markers of the glove marker unit are attached. The location information output unit generates a binary code by processing the signal values of the n optical signals detected by the respective markers, and generates the binary code by using the generated binary code. Calculate the position coordinate value.

In the same manner, the position information output unit binary codes the detected optical signals of the armor marker unit for the optical signals sequentially irradiated from the remaining k-1 lighting units, and compares them with the reference table of the position coordinate values, The position information of each marker according to all the lighting units is respectively calculated.

The location information output unit calculates respective two-dimensional location information or three-dimensional location information for the markers. For example, as shown in FIG. 6, a pair of lighting units, that is, lighting units 1 and 2 (I ₁ , I ₂ ) are mounted on the frame of the display device at right angles to each other, or the lighting unit 3. And 4 (I ₃ , I ₄ ) are mounted on the frame of the display device at right angles to each other, the position information output section receives the optical signals irradiated from the pair of two lighting units from the armor marker section. Receive and calculate the position for each marker in two-dimensional coordinates in two-dimensional space. Meanwhile, two pairs of lighting units, that is, lighting units 1 and 2 (I ₁ , I ₂ ) and lighting units 3 and 4 (I ₃ , I ₄ ) are mounted on the frame of the display device at right angles to each other. The position information output unit receives the optical signals irradiated from the four lighting units of the two pairs from the armor marker unit, and calculates positions of the respective markers in three-dimensional coordinates in three-dimensional space.

The location information output unit wirelessly transmits the calculated location information of each marker to the display apparatus.

After the step 404, the haptic performer performs a tactile recognition operation in response to the response signal from the display device according to the transmission of the location information (step 406).

The display apparatus transmits a response signal in the display apparatus according to the position recognition to the haptic performing unit in response to the position information transmitted from the position information output unit. Then, the haptic performing unit provided with the glove marker unit performs a tactile perception operation in response to the response signal from the display device according to the transmission of the position information. The haptic performing unit generates a vibration, force, pressure, etc. as a tactile cognitive motion so that the user can feel the touch.

Meanwhile, the method inventions of the present invention described above can be implemented in computer-readable code / instructions / programs. For example, it may be implemented in a general-purpose digital computer that operates the code / instructions / program using a computer-readable recording medium. The computer-readable recording medium includes a storage medium such as a magnetic storage medium (e.g., a ROM, a floppy disk, a hard disk, a magnetic tape, etc.), an optical reading medium (e.g., a CD-ROM, a DVD, .

The above-described glove device and method for hand gesture recognition and interaction of the present invention have been described with reference to the embodiments shown in the drawings for clarity, but this is merely illustrative, and those skilled in the art will appreciate It will be appreciated that variations and other equivalent embodiments are possible. Accordingly, the true scope of the present invention should be determined by the appended claims.

100: lighting unit
200: gloves marker portion
300: location information output unit
310: memory
320: position calculation module
330: wireless transmission module
340: control module
350: power supply module
400: haptic performer

Claims (25)

Lighting unit mounted on the display device and having k (k is a natural number greater than or equal to 1) light units each of n (n is a natural number greater than or equal to 1) optical signals that are spatially divided into different patterns. ;
A glove marker having a glove shape and having m markers (m is a natural number greater than or equal to 1) for detecting the optical signals irradiated from the illumination unit, respectively, attached to a plurality of areas of the glove; And
And a location information output unit for recognizing location information of each of the markers from the optical signals received from the glove marker unit, and wirelessly transmitting the location information of each of the recognized markers to the display device. Armor device for hand gesture recognition and interaction. The method of claim 1,
The lighting units are each provided with n lighting elements, the lighting device comprises a light source and a space dividing element, respectively, gloves for hand gesture recognition and interaction. 3. The method of claim 2,
The space dividing element is a glove device for hand gesture recognition and interaction, characterized in that it has a light transmission area and a light blocking area divided into a bar code pattern at a predetermined interval. 3. The method of claim 2,
The lighting device is a glove device for hand gesture recognition and interaction, characterized in that to irradiate a plurality of optical signals spaced separately in different patterns at the same time. The method of claim 1,
The lighting unit is a glove device for hand gesture recognition and interaction, characterized in that for irradiating a plurality of optical signals sequentially with each other. 3. The method of claim 2,
The lighting device is a glove device for hand gesture recognition and interaction, characterized in that each of the illumination polarization filter having a different polarization direction. 3. The method of claim 2,
The lighting device is a glove device for hand gesture recognition and interaction, characterized in that each of the illumination bandpass filter for filtering different wavelength bands. The method of claim 1,
At least one pair of the illumination units for the two-dimensional position recognition, the armor device for hand gesture recognition and interaction, characterized in that the perpendicular to each other mounted on the display device. The method of claim 1,
At least two pairs of the lighting units to be perpendicular to each other for three-dimensional position recognition is mounted on the display device, the armor device for hand gesture recognition and interaction. 10. A method according to any one of claims 8 and 9,
The lighting unit is armored device for hand gesture recognition and interaction, characterized in that mounted on the corner portion of the frame of the display device. The method of claim 1,
And the markers each include n photodiodes for receiving the optical signals. 12. The method of claim 11,
And the markers each include n marker polarization filters having different polarization directions, respectively. 12. The method of claim 11,
And the markers each include n marker bandpass filters for filtering different wavelength bands. The method of claim 1,
The markers are gloves device for hand gesture recognition and interaction, characterized in that each of the camera module for receiving the optical signals. 15. The method of claim 14,
And the camera module is equipped with n camera polarization filters having different polarization directions in the image sensor for receiving the optical signals. 15. The method of claim 14,
And the camera module is equipped with n camera bandpass filters for filtering different wavelength bands in the image sensor for receiving the optical signals. According to claim 1, wherein the position information output unit
A memory for storing table information between binary codes and position coordinate values corresponding to the optical signals previously defined;
A position calculation module for calculating position information of each of the markers provided by the glove marker unit;
A wireless transmission module for wirelessly transmitting the location information calculated by the location calculation module to the display device;
A control module for controlling the operation of the position calculating module and the wireless transmission module; And
Armor apparatus for hand gesture recognition and interaction, characterized in that it comprises a power supply module for supplying power for driving the position calculation module, the wireless transmission module and the control module. The method of claim 17, wherein the position calculation module
Glove device for hand gesture recognition and interaction, characterized in that for calculating any one of the two-dimensional and three-dimensional position information for each of the markers. The apparatus of claim 1, wherein the glove device for hand gesture recognition and interaction
And a haptic operation unit for performing a tactile recognition operation in response to the response signal from the display device in response to the transmission of the position information. The method of claim 1,
The display device is a glove device for hand gesture recognition and interaction, characterized in that it comprises any one or more of a digital TV, desktop PC, tablet PC, notebook and smartphone. The lighting unit having k (k is a natural number greater than or equal to 1) light units each of which is mounted on the display device and irradiates n (n is a natural number greater than or equal to 1) optical signals that are spatially divided into different patterns. Irradiating the optical signals respectively;
Detecting each of the optical signals by a glove marker having a glove shape and having m markers (m is a natural number greater than or equal to 1) for detecting the optical signals irradiated from the illumination unit; And
And a location information output unit recognizing location information of each of the markers from the optical signals received from the glove marker unit, and wirelessly transmitting the location information of each of the recognized markers to the display device. Method for hand gesture recognition and interaction. The method of claim 21, wherein the step of irradiating the optical signals respectively
And n lighting elements provided in each of the lighting units irradiate a plurality of optical signals, each of which is spatially divided in different patterns, at the same time. The method of claim 21, wherein the step of irradiating the optical signals respectively
And the illumination units irradiate a plurality of optical signals sequentially with each other. The method of claim 21, wherein recognizing location information of each of the markers
Recognizing any one of the two-dimensional and three-dimensional position information for each of the markers for hand gesture recognition and interaction. 22. The method of claim 21, wherein the method for hand gesture recognition and interaction is
And performing a haptic perception operation in response to a response signal from the display device according to the transmission of the position information.

Images