KR200461366Y1 - Pointing Apparatus Using Image - Google Patents

Abstract

The present invention relates to a pointing device using motion recognition from an image, comprising: a display unit displaying an output target image; A camera which is fixed and captures a certain area in real time; And separating the background pixel and the object pixel entering the region from the real-time image photographed by the camera, and determining whether the object pixel is classified into a plurality of groups according to the position, and when the object pixel is specified as one group, Extract the coordinates in the real-time image of the group of and convert the coordinates in the real-time image to the coordinates in the display unit, position the pointer on the coordinates in the display unit in the display unit, and if the object pixels are classified into a plurality of groups, And a control unit for extracting the coordinates and converting the coordinates in each of the real-time images into coordinates in the display unit so that the display unit places a plurality of pointers in the coordinates in the display unit.

Description

Pointing Apparatus Using Image}

The present invention relates to a pointing device, and more particularly, to a pointing device using motion recognition from an image.

Many electronic devices such as computers have a GUI environment and use pointers to control their operation. The pointer on the screen can be moved using a mouse, a direction key of a keyboard, a joystick, a touch screen, and the like.

As a guide terminal, a sales terminal, or a kiosk used by many people, a touch screen type that does not require a separate device for a user to use is widely used. The touch screen method is a hygiene problem because many people touch the screen. In addition, the user should be as close as possible to the screen, there is a disadvantage that the size of the screen is limited. The same problem applies to other input methods such as a mouse, keyboard, and joystick.

Meanwhile, in a presentation in which a large screen is required for a large number of viewers and a projector screen is used, the presenter controls the pointer of the screen by using a pointing device such as a mouse. A pointing device such as a mouse has a hygienic problem because many people contact it. In addition, a pointing device such as a mouse restricts the movement of the presenter.

The present invention has been devised to solve the problems of the prior art, the object of the present invention is to move the point by the movement of an object such as a user's hand, using a separate device for pointing or do not need to touch It is to provide an apparatus and method.

Another object of the present invention is to allow the control unit to extract the distance from the camera to the object based on the size of the object (or the number of pixels classified as the object) to reproduce the three-dimensional motion of the object, such as a three-dimensional game or content It is to provide a pointing device and method that can be used to control the.

Still another object of the present invention is to provide an infrared camera and an infrared light emitter to determine the intensity of infrared rays reflected when the user enters not only the user's hand and arm but also other body parts in the photographing area of the camera. The present invention provides a pointing device and a method for classifying body parts so as to respond only to a user's hand or arm.

Still another object of the present invention is to point out that the control unit extracts the distance from the camera to the object based on the intensity of the reflected infrared rays so as to reproduce the three-dimensional movement of the object. It is to provide an apparatus and method.

Another object of the present invention, the camera and the infrared emitters are installed to be spaced apart from each other so that the infrared irradiation direction of the infrared emitters are directed toward the object, such as the user, hands and arms, so as not to include other parts of the body and the other body It is to provide a pointing device and a method for easily identifying the site.

Still another object of the present invention is to provide a thermal imaging camera that extracts temperature information of an object and when a user uses a hand and an arm, a pointing device that can easily extract an object from a captured image by extracting only pixels corresponding to body temperature. And to provide a way.

Another object of the present invention, when one or more pixels determined to be an object is adjacent to each other, grouping them into the same group and repeating the same process to assign two or more objects to each group, such as the user's two hands and arms, etc. It is to provide a pointing device and method capable of synchronizing and controlling the movement of a target of the object to the movement of a plurality of points.

In order to achieve the above object, the present invention, a display unit for displaying an output target image; a fixed and real-time camera to shoot a certain area; And separating an object entering the area from the background from the real-time image photographed by the camera, extracting a first coordinate of the object in the real-time image, converting the first coordinate into a second coordinate in the display unit, and displaying the same on the display unit. It provides a pointing device using an image comprising a; control unit for moving the pointer to the second coordinate.

It is preferable to make 1st coordinate the leftmost coordinate of the uppermost side of the said object.

The controller may store the initial image photographed by the camera in advance, and determine that the pixel of the pixel of the difference image of the initial image and the real-time image is outside the first threshold range as the object. The absolute value of the upper limit and the absolute value of the lower limit of the first threshold range may be different from each other. The first threshold range is set for each of the plurality of color channels of the difference image, and the controller may determine that the pixels having at least one chromaticity value among the plurality of color channels among the pixels of the difference image are outside the first threshold range. It is preferable to exclude from the object a pixel whose pixel value of adjacent pixels among the pixels determined as the object is within the first threshold range. The initial image is an image when the image photographed by the camera has not changed for a predetermined first time. The first threshold range may be set differently based on the pixel value of each pixel of the initial image.

Alternatively, the controller may determine, as an object, pixels having a pixel value within a predetermined second threshold range among pixels of the real time image.

The controller may separate the object based on a plurality of tracking pixels extracted at equal intervals from among the pixels of the camera.

The pointing device further comprises an infrared light emitter, and the camera is an infrared camera. The photographing area of the camera may be determined by adjusting the irradiation distance or the irradiation angle of the infrared light emitter.

Alternatively, the camera may be a thermal imaging camera.

The controller may measure the distance between the camera and the object based on the size or the measured intensity of the object, and control the pointer based on the measured distance.

The controller determines whether or not the object pixel is classified into a plurality of groups according to the position. When the object pixel is classified into a plurality of groups, the controller extracts coordinates in the real-time image for each group and displays the coordinates in each real-time image in the display unit. By converting the coordinates, the plurality of pointers may be positioned at the coordinates in the display unit.

When all two adjacent pixels are separated into objects, it is preferable to determine that two adjacent pixels are included in one group.

A plurality of pointers are used as a pointer for multi-pointing, a pointing device using an image, characterized in that used as a pointer for dragging a plurality of objects, or enlargement or reduction of an object.

The present invention has the effect of not having to use or touch a separate device for pointing by moving a point by the movement of an object such as a user's hand.

The present invention allows the control unit to extract the distance from the camera to the object based on the size of the object (or the number of pixels classified as the object) to reproduce the three-dimensional movement of the object, thereby controlling the three-dimensional game or content. An effect that can be used can be obtained.

The present invention is equipped with an infrared camera and an infrared light emitter to distinguish the user's hand, arm, and body parts by judging the intensity of infrared rays reflected when the user's hands and arms as well as other body parts enter the camera's shooting area. It has an effect that can respond only to the user's hand, arm.

The present invention allows the controller to extract the distance from the camera to the object based on the intensity of the reflected infrared rays to reproduce the three-dimensional movement of the object, thereby achieving an effect that can be used to control the three-dimensional game or content. have.

The present invention, the camera and the infrared emitters are installed to be spaced apart from each other so that the infrared irradiation direction of the infrared emitters toward the user and the object, such as hands and arms, and do not include other parts of the body to easily distinguish the object and other body parts It has the effect of making it possible.

The present invention is equipped with a thermal imaging camera to extract the temperature information of the object and when using the user's hands and arms only by extracting the pixels corresponding to the body temperature can achieve an effect that can easily extract the object from the captured image .

In the present invention, when one or more pixels determined to be objects are adjacent to each other, they are grouped into the same group and the same process is repeatedly performed to assign two or more objects to each group to move a plurality of objects such as two hands and arms of a user. It has the effect of synchronizing and controlling the movement of a plurality of points.

The present invention allows the user to move and click the pointer in a non-contact manner. In particular, it can be used where many unspecified publics use, and the effect is to provide a simple structure that uses video but does not require additional devices such as blue screens.

 According to the present invention, when the extracted tracking pixel is used, it is possible to obtain an effect of eliminating errors caused by bad pixels without significantly reducing the performance of the embedded hardware.

1 is a perspective view of a guidance terminal as a first embodiment of a pointing device using an image according to the present invention,
FIG. 2 is a block diagram of the guidance terminal of FIG. 1;
3A-3E are side views illustrating the embodiment of FIG. 1,
4 is a flowchart of an embodiment of a pointing method using an image according to the present invention;
5 is a view for explaining the principle of operation of the pointing method embodiment using an image according to the present invention,
6A and 6B are plan views illustrating an operation example of the guide terminal of FIG. 1;
7A and 7B are side views for explaining an operation example of the guide terminal of FIG. 1;
8 is a side view of an example of a guidance terminal as a second embodiment of a pointing device using an image according to the present invention, and
9 is a plan view of an example of a guide terminal as a second embodiment;
10 is an exemplary diagram of a pixel in the fourth embodiment,
11 is an exemplary view of a multi-pointing work screen in the fourth embodiment, and
12 is an exemplary view of a screen for dragging a plurality of windows in the fourth embodiment.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention. Additional configurations and effects of the present invention will be described in more detail below. However, the following examples are intended to illustrate the present invention and the present invention is not limited by the examples.

<First Embodiment>

1 is a perspective view of a guiding terminal 100 as a first embodiment of a pointing device according to the present invention, Figure 2 is a block diagram of the guiding terminal 100.

1 and 2, the guide terminal 100 may include a display unit 120 exposed on the surface of the case 110, a camera 130 installed on an upper side or a lower side of a front surface of the display unit 120, and a case ( The control unit 140 is installed on the inner surface of the 110.

The display unit 120 is a device for providing information to a user, and various monitors according to the prior art may be used. The display unit 120 is connected to the control unit and outputs a screen according to the instruction of the control unit.

The camera 130 is a digital camera capable of capturing a user's hand movements. In the present embodiment of the guidance terminal, the camera 130 is preferably fixed to the upper side or the lower side of the front of the display unit 120, but the present invention is not limited thereto, and the display unit 120 can be visually confirmed. When the user is located at a place, the camera 130 may be installed at a position where the user's hand movement can be easily observed. When used for presentation, for example, the user is located in a place where the screen can be seen without disturbing the viewer's gaze, and the camera is installed corresponding to the position of the user's hand.

The space in a predetermined area between the user and the display unit 120 is continuously photographed. Thus, when the user moves the hand, the camera 130 photographs the hand entered into the area. In the present invention, a separate backdrop such as a blue screen is not installed in the opposite direction of the camera 130.

The controller 140 receives and analyzes the captured image from the camera 130 to convert the position of the user's hand that has entered the shooting area of the camera 130 into pointing data and displays the display unit together with previously stored data. The memory 142 includes a CPU 141 transmitted to the 120 and a program and data for analyzing data to be displayed on the display 120 and an image captured by the camera 130.

3A to 3E, the user operates the guide terminal 100 at a position spaced apart from the display unit 120 by a predetermined interval. In FIG. 3A, the camera 130 is installed on the front ceiling of the display unit 120 to photograph the vertical downward direction, or is installed on the front bottom to photograph the vertical upward direction. In the case of FIG. 3A, a user's hand that has entered the area may be easily photographed. In FIG. 3B, the camera 130 is photographed by being installed at an angle inclined upward toward the lower side of the display unit 120. In the case of Figure 3b is easy to install and maintain the equipment compared to Figure 3a, but there is a fear of interference by other body parts than the user's hand. In FIG. 3C, the camera 130 is installed at an angle inclined to the lower side of the display unit 120, and a mirror 150 is installed below the camera 130 to secure a photographing area. In the case of FIG. 3C, the camera 130 may be prevented from being exposed to the maximum. In FIG. 3D, the camera 130 is installed in a separate single phase 160 installed in the front of the display unit 120. The upper surface of the single phase 160 is transparent or translucent. In the case of FIG. 3D, the user may operate the guide terminal 100 more intuitively. It can be used when a single phase is required for the convenience of the user. In FIG. 3E, the display unit 120 is inclined and the camera 130 is installed at the lower end of the display unit 120 to capture the vertical upward direction. 3A and 3D, as in the case of FIG. 3B and 3C, it is less likely to be interfered with by other body parts, and the equipment is easy to install and maintain. As shown in FIGS. 3A to 3E, the camera 130 may be spaced apart from the display unit 120 or the like, or may be integrally installed.

Hereinafter, a pointing method will be described in the guide terminal 100 having the above-described configuration.

4 is a flowchart illustrating a pointing method according to the present invention. FIG. 5 is a diagram for describing the flowchart of FIG. 4.

Referring to FIG. 4, the camera 130 continuously photographs a predetermined area (S10).

The image photographed by the camera 130 is transmitted to the controller 140, and the controller 140 separates an object (a hand, an arm, etc.) entering the area from the background from the photographed image (S20).

According to an embodiment, the background and the object are separated by comparing an image captured in real time with an initial image prepared and stored in advance.

The initial image is an image photographing only a background without an object in advance, and has a predetermined chromaticity value (RGB or CMYK) for each pixel. When the chromaticity value of each frame of the video photographed by the camera 130 does not change for a predetermined time, the initial image may be determined as the initial image by determining that only the background exists in the area photographed by the camera 130. have. The initial image may be updated from time to time when the environment changes over time, such as outdoors.

The difference image obtained by subtracting the image captured in real time from the initial image is obtained by the camera 130. When the chromaticity value of the difference image is within a predetermined first threshold range, it is classified as a background, and when it is outside the first threshold range, it is classified as an object (a hand, an arm, etc.) that has entered the area photographed by the camera 130. For example, when the RGB value of the initial image of one pixel is (196,196,204) and the RGB value of the real time image is (102,62,70), the RGB value of the difference image is (92,134,134), and the RGB value of the difference image is It is determined whether it is within a first threshold range.

The first threshold range is set for each color channel RGB. In addition, the first threshold range for each color channel may be different. If the chromaticity value of at least one of the color channels of one pixel of the difference image is outside the first threshold range, the pixel is classified as an object. For example, when the R value among the RGB values of one pixel of the difference image is outside the first threshold range, and the G and B values are within the first threshold range, the pixel is classified as an object.

The first threshold range may have a positive upper limit and a negative lower limit with the same absolute value. Alternatively, the first threshold range may have an upper limit value and a lower limit value at which absolute values are different from each other. At this time, the upper limit value is necessarily a positive value and the lower limit value is not necessarily a negative value. For example, when the background photographed in the initial image is bright and the object is dark, both the upper limit value and the lower limit value of the first threshold range may be positive values. That is, the threshold value when the chroma value of the real time image increases and decreases from the initial image may be set differently. As described above, the upper limit value and the lower limit value may be determined from the brightness of the surrounding environment and the background.

In addition, the first threshold range of each pixel may be set differently. In the initial image (background), chromaticity values of each pixel may be different. When the same first threshold range is set for pixels having different chromaticity values, sensitivity for determining an object may be different for each pixel. For example, if the background is divided into relatively bright and dark parts, when setting the first threshold range based on the dark parts, the bright parts are sensitive to changes in chromaticity values according to the surrounding environment, When setting the first threshold range as a reference, the dark portion may not respond to a change in chromaticity value due to the entry of the object. Therefore, it is preferable that the light portion sets the first threshold range relatively wide and the dark part sets the first threshold range relatively narrow.

According to another embodiment, the background and the object are separated based on the real-time image captured by the camera 130 without the initial image. That is, among the pixels of the image photographed by the camera 130, pixels whose chromaticity values are within a predetermined second threshold range are classified as objects that enter the area photographed by the camera 130. For example, when the RGB value of one pixel of the real-time image is (102, 62, 70), it is determined whether the RGB value is within the second threshold range. The second threshold range is determined based on skin color. In this case, although there is an advantage that the initial image is not used, there is a risk of malfunction when the user operates while wearing gloves or the like.

The second threshold range is set for each color channel RGB. And, the second threshold range for each color channel may be different. If the chromaticity values of all the color channels of one pixel of the real-time image are within the second threshold range, the pixel is classified as an object. For example, when the R value of the RGB value of one pixel of the real time image is within the second threshold range, and the G value and the B value are outside the second threshold range, the pixel is classified as a background.

As described above, when the background and the object are separated by using the chromaticity value, the background and the object may be incorrectly separated by the bad pixel. Therefore, it is preferable that all of the pixels surrounding the background classified among the pixels classified as the object are not the object.

When the background and the object are separated, the coordinates of the object are extracted in the captured frame (S30). The coordinate is preferably the coordinate of the position of the fingertip. The coordinates of the fingertips are determined by the coordinates of the uppermost pixel among the pixels classified as the object, and when there are a plurality of the uppermost pixels, the coordinates of the leftmost pixel of the uppermost pixel are determined (based on the right-handedness). ).

The object separation and coordinate extraction step (steps S20 and S30) may be performed for all pixels, but may be performed only for a predetermined tracking pixel in order to reduce the execution time and the load of the controller 140. For example, if the pixels of the camera 130 are 706432 (1024X768), the steps S20 and S30 may be performed only for 768 (32X24) tracking pixels arranged at equal intervals.

Coordinates in the frame photographed by the camera 130 are converted into coordinates in the display unit 120 (step S40). The area photographed by the camera 130 corresponds to the display unit 120 one-to-one. For example, when the coordinates of the fingertip are extracted by 32 × 24 pixels in a frame, (1,1) pixels among the 32 × 24 pixels are the leftmost of the uppermost part of the display unit 120, and (32,24) pixels are displayed on the display unit ( It corresponds to the bottommost right side of 120). The controller 140 moves the pointer to the coordinates in the converted display unit 120.

As described above, the user may move the pointer by moving a hand in an area photographed by the camera 130 without using a touch screen or a mouse.

6A and 6B are diagrams illustrating controlling a pointer by using the above-described guide terminal 100. 6A and 6B are plan views when the camera 130 is installed below as an example.

6A and 6B, the user moves his or her hand horizontally in the camera imaging area 170 to move the pointer 121 in the display unit 120. The camera imaging area 170 may be intuitively acquired by the user based on the movement of the pointer 121 in the display unit 120 or may be limited by a mechanism such as the single phase 160 as shown in FIG. 3D. In FIG. 6A, when the user moves his hand to the left side of the camera imaging area 170, the pointer 121 moves to the upper left side of the display unit 120. In FIG. 6B, when the user moves his hand to the right rear side of the camera imaging area 170, the pointer 121 moves to the lower right side of the display unit 120.

7A and 7B are diagrams illustrating controlling a pointer by using the above-described guide terminal 100. 6A and 6B are plan views when the camera 130 is installed below as an example.

6A and 6B illustrate the user moving his hand horizontally within the camera imaging area 170, while in FIGS. 7A and 7B the user moves the pointer 121 by changing the inclination of his arm. In FIG. 7A, when the user makes his hand horizontal, the pointer 121 moves to the upper side of the display unit 120. In FIG. 7B, when the user tilts his hand, the pointer 121 moves to the lower side of the display unit 120. Move.

The controller 140 may extract a distance from the camera 130 to the object based on the size of the object (or the number of pixels classified as the object). Since the actual size of the object is unknown, the distance is not an absolute value but a relative value between each frame. The measured distance can be used to reproduce the three-dimensional movement of the object, and can be used to control the three-dimensional game or content.

On the other hand, the user wants not only to move the pointer but also to click at the desired position of the pointer. If the position of the pointer determined as described above does not change for a predetermined time (for example, 1 second), the controller 140 determines that the user wants to click at the position. On the other hand, when the number of tracking pixels in the camera 130 frame is arranged in a large number, the position of the pointer may be minutely changed due to camera shake. In this case, it is determined that the user wants to click when the position of the pointer remains within a predetermined range for a predetermined time.

Alternatively, when the controller 140 can extract the distance from the camera 130 to the object, the controller 130 may determine that the user wants to click when the user moves the body in the shooting direction of the camera 130. Can be.

&Lt; Embodiment 2 >

8 is a side view of a guidance terminal as a second embodiment of a pointing device according to the present invention. The configuration of the guide terminal according to the second embodiment is the same as that of the first embodiment, except that the camera 130 is an infrared camera and further includes an infrared light emitter 180 for emitting infrared light.

The camera 130 is configured by using a sensor that detects only infrared rays or a filter that passes only infrared rays.

The infrared light emitter 180 emits infrared rays of the wavelength band photographed by the camera 130.

The controller 140 extracts only pixels having a predetermined third or more threshold value from the real-time infrared image captured by the camera 130 and moves the pointer in the display unit 120.

Referring to FIG. 8, an infrared light emitter 180 is installed adjacent to the camera 130, and an area irradiated with infrared light from the infrared light emitter 180 substantially corresponds to an imaging area of the camera 130. When the subject enters the camera imaging area 170, the intensity of the infrared rays irradiated on the subject decreases as the distance from the infrared emitter 180 to the subject increases, and the infrared rays reflected from the subject are measured by the camera 130. The intensity is reduced as the distance from the camera 130 to the subject increases. Since the camera 130 and the infrared light emitter 180 are installed adjacent to each other, the intensity of the infrared light measured by the camera 130 decreases as the distance to the subject increases.

As shown in FIG. 8, not only the user's hand and arm but also other body parts may enter the imaging area of the camera 130. However, since the intensity of infrared rays reflected from the user's hand and arm is stronger than the intensity of infrared rays reflected from other body parts, the intensity of the infrared rays reflected from other body parts and the intensity of infrared rays reflected from the user's hand and arm The third threshold value is set so that it can be distinguished. In FIG. 8, the portion photographed above the third threshold is defined as the camera imaging area 170. Therefore, in the second embodiment, since the control unit extracts only the portion that is greater than or equal to the third threshold value, the control unit 140 can easily separate not only the background but also other body parts other than the user's hand and arm. The control unit extracts the coordinates of the user's hand and arm, converts the coordinates into the coordinates in the display unit 120, and moves the pointer in the display unit 120 according to the converted coordinates.

The controller 140 may measure the distance between the camera 130 and the object based on the intensity of the reflected infrared light. Since the position of the camera 130 and the infrared light emitter 180 and the luminous intensity of the infrared light emitter 180 may be known in advance, the measured distance may be an absolute value. The measured distance can be used to reproduce the three-dimensional movement of the object, and can be used for the three-dimensional game, the control of the content, the click, and the like. It is also possible to measure the distance by the size of the object as in the first embodiment.

Meanwhile, although the camera 130 and the infrared light emitter 180 are shown to be installed adjacent to each other in FIG. 8, the camera 130 and the infrared light emitter 180 may be spaced apart from each other. Referring to FIG. 9, the photographing direction of the camera 130 includes not only a user's hand and arm but also other body parts, but an infrared irradiation direction of the infrared light emitter 180 spaced apart from the camera 130 indicates another body part of the user. Does not include Thus, the area actually photographed by the camera 130 is defined as the camera imaging area 170 of FIG. 9. The photographing area corresponding to the display unit 120 is determined in the camera imaging area 170. In this case, it becomes easier to distinguish the user's hands, arms and other body parts. In FIG. 9, the positions of the camera 130 and the infrared light emitter 180 may be changed.

Third Embodiment

The configuration of the guide terminal according to the third embodiment is the same as that of the first embodiment, except that the camera 130 is a thermal imaging camera.

In the present embodiment, the camera 130 extracts temperature information of an object in the camera imaging area 170. Thus, when the user uses the hand, only the pixels corresponding to the body temperature can be extracted, thereby easily extracting the pixels corresponding to the object.

The controller 140 may measure the distance between the camera 130 and the object based on the intensity of the thermal image. In this case, the measured distance may be an absolute value. The measured distance can be used to reproduce the three-dimensional movement of the object, and can be used for the three-dimensional game, the control of the content, the click, and the like. It is also possible to measure the distance by the size of the object as in the first embodiment.

According to the present embodiment, by using the thermal imaging camera, pixels corresponding to the body can be easily separated, and free from the influence of ambient lighting as compared with the first embodiment, and the infrared light emitter described in the second embodiment, etc. No additional configuration is required. In addition, since it is less influenced by a fuselage or the like surrounded by clothing, only hand movements can be easily captured.

<Fourth Embodiment>

The pointing device according to the first to third embodiments described above may be provided to control a plurality of pointers. In the above-described embodiment, the user enters one hand and arm into the camera imaging area 170 to control one pointer. However, in the present embodiment, the user enters both hands and both arms into the camera imaging area 170 so that two points are controlled. You can control the pointer.

In the present embodiment, the controller 140 classifies and manages pixels determined as objects in the image photographed by the camera 130 into one or more groups. If it is determined that two adjacent pixels are all objects, it is determined that the two adjacent pixels belong to one group. In this way, the group is expanded to determine all pixels included in one group.

For example, when (50, 100) pixels are determined to be an object and when the group identifier 'A' is specified for (50, 100) pixels, (51, 100) pixels are determined to be an object (51, 100) The pixel is also assigned a group identifier 'A', and when (51, 101) pixels are determined to be objects, the (51, 101) pixels are also assigned a group identifier 'A', and in this way The pixels to be connected are assigned to the group 'A'. On the other hand, if (200, 100) pixels are determined to be an object, and (200, 100) pixels and the pixels connected thereto are not adjacent to the pixels belonging to the group 'A' (that is, connected to (50, 100) pixels) If not), the (200, 100) pixels and the pixels connected thereto are designated with a separate group identifier 'B' (see FIG. 10).

This group designation process may include the bad pixel removal process described above. That is, when all adjacent pixels of the pixel classified as the object are classified as the background, the adjacent pixels are assigned to the same group only when the adjacent pixels are classified as the object, and therefore, no group is assigned to the pixel having no adjacent pixel classified as the object. , The pixel to which the group is not designated may be determined as a bad pixel.

When all the pixels determined to be the object are included in one group, the following process is as described in the first to third embodiments.

When the pixel determined as the object is classified into a plurality of groups, the controller 140 converts coordinates corresponding to each group into coordinates of the display unit 120, and converts a plurality of coordinates into coordinates of the plurality of converted display units 120. Position the pointer.

This group classification is performed for each frame of the video. And if there is continuity between the groups classified in each frame, they are assigned to the same group. Continuity is determined to be continuity if the coordinate change is within a predetermined threshold. Thus, each pointer may move on the display 120 in succession.

Meanwhile, while the user initially uses only one hand and only one pointer is displayed on the display unit 120, the user may use the other hand. In this case, a new group is assigned to a newly created group without continuity, and a new pointer according to the new group is created in the display unit 120.

Similarly, some of the plurality of pointers may disappear within the display unit 120. For example, when a user controls two pointers using two hands and pulls one of the hands out of the camera photographing area 170, the pixels recognized as objects belong to only one group and the display unit 120. One of the two pointers in) disappears. Alternatively, when two hands are overlapped while controlling two pointers using two hands, pixels classified into two groups belong to only one group, and one of the two pointers in the display unit 120 disappears. . In the present invention, when the coordinate of the object is preferably set to the leftmost coordinate on the uppermost side, the pointer on the upper side or the left side of the two pointers will be left and the pointer on the lower side or the right side will disappear.

Such a plurality of pointers may be used as multi-pointers. For example, referring to FIG. 11, when a user executes a program such as a word processor, a spreadsheet, a presentation, an image processor, one pointer 121a is positioned in the work area 191 and the other pointer 121b is used. May be located in the menu area 192. Alternatively, when the user works in a plurality of windows, each pointer may be placed in each window. Thus, the trouble of the user moving the pointer from time to time can be reduced. Although the multi-pointer according to the related art is implemented using one pointer moving device such as a mouse, in the present invention, the control is easily performed by controlling the multi-pointer using two hands.

Alternatively, the plurality of pointers may be used to move, enlarge or reduce the object. For example, referring to FIG. 12, when a user moves a plurality of objects, each of the pointers 121a and 121b is positioned at each of the objects 193 and 194, and the respective pointers 121a and 121b are moved. Each object 193, 194 can be moved. As another example, when a user zooms in or out of an object, places one pointer at one corner of the object, another pointer at a diagonal corner of one corner, and then moves the pointers to enlarge the size of the object or Can be shrunk.

The above-mentioned plurality of pointers can be generated only in a situation in which a plurality of pointers are required to reduce the risk of malfunction.

Although the above description has been made of the user controlling two pointers using two hands, the present invention is not limited thereto and can control two or more pointers. For example, when a plurality of users use their bodies, it is possible to control two or more pointers. This provides a new environment that allows multiple users to collaborate or play games using one screen.

In the above-described embodiment, the guide terminal is described, but this is only an example, and the present invention may be applied to various devices such as a sales terminal, a kiosk, a presentation device, an advertisement device, and the like.

Although the present invention has been described in connection with the above-described preferred embodiments, it will be readily apparent to those skilled in the art that various modifications and variations are possible without departing from the spirit and scope of the present invention, all of which are attached to the appended claims. It is obvious that it belongs to the scope of the claims.

100: information terminal
110: case
120: display unit
121: pointer
130: camera
140:
141: CPU
142: memory
150: mirror
160: single phase
170: camera imaging area
180: infrared emitter

Claims (15)

A display unit which displays an output target image;
A camera which is fixed and captures a certain area in real time;
Separating a background and an object entering the area from the real-time image photographed by the camera, extracting a first coordinate of the object in the real-time image, converting the first coordinate into a second coordinate in the display unit, and A controller configured to move the pointer displayed on the display unit to the second coordinates; And
Includes an infrared light emitter,
The camera is an infrared camera, and the controller distinguishes and reacts an object such as a user's hand and an arm and other parts of the body based on an infrared ray intensity reflected from an object such as a user's hand and arm and other parts of the body. Pointing device using images. The method of claim 1,
The controller may store the initial image photographed by the camera in advance, and if the pixel value among the pixels of the difference image between the initial image and the real-time image is outside a predetermined first threshold range, determine the pixels as the object. Pointing device using an image, characterized in that. The method of claim 2,
Pointing device using an image, characterized in that the absolute value of the upper limit and the lower limit of the first threshold range is different. The method of claim 3,
The first threshold range is set for each of the plurality of color channels of the difference image,
And the controller determines the pixels as the object when the chromaticity value of at least one of the plurality of color channels among the pixels of the difference image is outside the first threshold range. The method of claim 4, wherein
The first threshold range is set differently based on a pixel value of each pixel of the initial image. The method of claim 1,
The control unit is a pointing device using an image, characterized in that the pixel value of the pixel of the real-time image is within the predetermined second threshold range to determine the object. The method of claim 1,
The control unit is a pointing device using an image, characterized in that for separating the object based on a plurality of tracking pixels extracted at equal intervals among the pixels of the camera. delete delete The pointing device of claim 1, wherein the controller is configured to extract a distance from a camera to an object to reproduce a three-dimensional motion of the object. The method of claim 1,
A camera and an infrared light emitter are installed to be spaced apart from each other, the infrared irradiation direction of the infrared light emitter pointing to the object and do not include other parts of the body pointing device using an image that can easily distinguish the object and other parts of the body. delete The method of claim 1,
The controller classifies the pixels of the objects into a plurality of groups by classifying and classifying the pixels of the objects into the same group when the pixels determined to be objects in the image photographed by the camera are adjacent to each other and in a different group when they are not adjacent to each other. Extract coordinates in the real-time image and convert coordinates in each real-time image into coordinates in the display unit so that the plurality of pointers are positioned at the coordinates in the display unit in the display unit to synchronize the movement of the plurality of objects with the movement of the plurality of points. Pointing device using video. delete delete

Images