KR20140031105A - Method for extracting three dimensional distance information from recognition object and apparatus therefor - Google Patents

Abstract

Provided are a method for extracting information on a three-dimensional distance from a recognition object and an apparatus therefor capable of efficiently saving electric power necessary to detect a gesture input while accurately determining the gesture input which is inputted by a user using information on a distance from a recognition object. The method for extracting information on a three-dimensional distance using the apparatus therefor according to the present invention comprises the steps of: determining whether a recognition object exists within a preset range or not; generating a three-dimensional image of the recognition object in case the recognition object exists within the preset range; and calculating a distance from the recognition object using the three-dimensional image. [Reference numerals] (AA) Start; (BB) No; (CC) Yes; (DD) End; (S202) Generate a two-dimensional image by using a two-dimensional camera; (S204) Is a recognition object included in the two-dimensional image?; (S206) Emit light for measuring a distance from the recognition object; (S208) Generate a three-dimensional image by photographing the recognition object by a three-dimensional camera; (S210) Calculate a distance from the recognition object by using the three-dimensional image

Description

A method for extracting three-dimensional distance information from an object to be recognized and an apparatus therefor {METHOD FOR EXTRACTING THREE DIMENSIONAL DISTANCE INFORMATION FROM RECOGNITION OBJECT AND APPARATUS THEREFOR}

The present invention relates to a method for extracting three-dimensional distance information, and more particularly, to accurately determine a gesture input input from a user by using distance information with a recognition target, and to efficiently determine the power required to detect the gesture input. The present invention relates to a method for extracting three-dimensional distance information from a recognition object that can be saved, and an apparatus therefor.

The method of extracting 3D distance information can be largely divided into an active method and a passive method. The active scheme may be achieved by a device for extracting 3D distance information by firing energy for extracting the 3D distance information. Herein, the energy may be an optical wave, an ultrasonic wave, a microwave, or the like. A device which wants to extract three-dimensional distance information extracts the distance from the device to the scene or object by projecting the light wave onto a specific scene or object and photographing the projected scene or object.

On the other hand, the manual method is a method in which the device to extract the 3D distance information extracts the 3D distance information using only the image captured without emitting energy. The apparatus photographs a scene or an object for which the 3D distance information is to be extracted using an image capturing apparatus, and processes a photographed image to extract a distance from the apparatus to the scene or object.

Gesture control is a technique for controlling a device by recognizing a user's gesture, and there are methods using distance information and methods not using it. The method that does not use the distance information recognizes the user's gesture using only the captured 2D image, and it is necessary to separate the object to be recognized, that is, the hand, the arm, the leg, the face, the object, and the like from the background. On the other hand, the method using distance information not only separates the object to be recognized from the background using the distance information but also obtains the shape information more accurately than the method using the distance information. It is possible to accurately determine the gestures to ensure better performance.

Accordingly, an active method, that is, a method of emitting energy and detecting distance information by using reflected energy, must continuously emit energy in order to detect the distance information. Since the user's gesture may not be known, the device to extract the 3D distance information must continuously emit energy regardless of the user's gesture. Continuous firing of energy even when the user's gesture does not occur not only generates unnecessary power waste, but also shortens the use time of the device.

Accordingly, an object of the present invention is to accurately determine the gesture input input from the user by using the distance information with respect to the recognition object and at the same time three-dimensional distance information from the recognition object that can efficiently save the power required to detect the gesture input. To provide a method and apparatus for extracting the same.

The 3D distance information extraction method of the apparatus for extracting 3D distance information according to an embodiment of the present invention may include determining whether a recognition object exists within a preset range, and wherein the recognition object exists within the preset range. In this case, the method may include generating a 3D image of the object to be recognized, and calculating a distance from the object to be recognized using the 3D image.

The apparatus for extracting 3D distance information according to an embodiment of the present invention may include a camera controller for determining whether a recognition object exists within a preset range, and when the recognition target exists within the preset range, And a 3D camera for generating a 3D image of the 3D image, and a distance calculator configured to calculate a distance to the object of recognition using the 3D image.

According to an embodiment of the present invention, a terminal may include: an object detecting unit detecting a recognition object existing within a preset range, a 3D camera generating a 3D image corresponding to the recognition object, and the recognition object using the 3D image. A distance calculator configured to calculate a distance between the controller and the controller, and a controller configured to determine a gesture input corresponding to a shape of the object to be recognized based on the distance between the 3D image and the object to be recognized, and to perform an operation according to the gesture input. .

A method of performing an operation corresponding to a gesture input by the terminal includes detecting a recognition object existing within a preset range, generating a 3D image corresponding to the recognition object, and using the 3D image to recognize the recognition object. Calculating a distance from an object, determining a gesture input corresponding to the shape of the object to be recognized based on the distance between the 3D image and the object to be recognized, and performing an operation according to the gesture input Include.

According to the present invention, three-dimensional distance information from an object to be recognized can be efficiently determined by using the distance information with respect to the object to be recognized accurately and at the same time efficiently saving power required for detecting the gesture input. It provides a method of extraction and an apparatus therefor.

1 is a block diagram showing the configuration of a three-dimensional distance information extraction apparatus according to an embodiment of the present invention;
2 is a flowchart illustrating an example of a method of extracting 3D distance information of the apparatus for extracting 3D distance information shown in FIG. 1;
3 is a flowchart illustrating another example of a method of extracting 3D distance information of the apparatus for extracting 3D distance information shown in FIG. 1;
4 is a flowchart illustrating still another example of a 3D distance information extraction method of the apparatus for extracting 3D distance information shown in FIG. 1;
5 is a flowchart illustrating still another example of a 3D distance information extraction method of the apparatus for extracting 3D distance information shown in FIG. 1;
FIG. 6 is a block diagram illustrating a configuration of a terminal including the apparatus for extracting three-dimensional distance information illustrated in FIG. 1;
7 is a flowchart illustrating an example of a method in which a terminal illustrated in FIG. 1 performs an operation corresponding to a gesture input;
8 is a flowchart illustrating another example of a method in which a terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.
9A is a diagram illustrating a terminal for detecting a recognition object using a 2D camera;
9B is a diagram illustrating a terminal for detecting a recognition object using a proximity sensor;
FIG. 10 is a flowchart illustrating still another example of a method in which a terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.
FIG. 11 is a flowchart illustrating another example of a method in which a terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input. FIG.
FIG. 12 is a flowchart illustrating still another example of a method in which the terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention. And will be apparent to those of ordinary skill in the art.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention. And will be apparent to those of ordinary skill in the art.

1 is a block diagram showing the configuration of a three-dimensional distance information extraction apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the apparatus for extracting 3D distance information 100 may include a target detector 120, an illumination unit 104, a 3D camera 106, a distance calculator 108, a camera controller 110, and a camera memory 114. It may include.

The object detector 120 detects the object to be recognized before the 3D camera 106 recognizes the 3D image of the object to be recognized. The object detector 120 may be implemented as a 2D camera 102, or in some embodiments, may be implemented as a proximity sensor 112 instead of the 2D camera 102. In addition, according to another exemplary embodiment, the object detector may include both the 2D camera 102 and the proximity sensor 112. In FIG. 1, for convenience of description, the object detector 120 implemented as a form including both the 2D camera 102 and the proximity sensor 112 is illustrated.

The 2D camera 102 performs a photographing operation by converting an image input through a lens (not shown) provided in advance into a 2D image. According to an exemplary embodiment, the 2D camera 102 may convert an image input through a lens (not shown) into a video.

The 2D camera 102 according to the present exemplary embodiment captures an image input through a lens and converts the image into a 2D image to capture a recognition target included in the image. As described above, the 2D camera 102 may generate a 2D image including a recognition target.

The lighting unit 104 emits light for allowing the 3D camera 106 to generate the 3D image before the photographing operation by the 3D camera 106 is performed. The lighting unit 104 according to the present exemplary embodiment may emit light having different amounts of light to the recognition target according to the size of the recognition target under the control of the camera controller 110. In addition, the lighting unit 104 may emit light having a pattern, which is preset and stored in the camera memory 114, toward the recognition target under the control of the camera controller 110. Can be.

The 3D camera 106 generates a 3D image of the object to be recognized. When light is emitted by the lighting unit 104, the 3D camera 106 may generate a 3D image by receiving reflected light reflected from a recognition object through a lens (not shown).

The distance calculator 108 calculates the distance between the object to be recognized, that is, the distance between the 3D distance information extracting apparatus 100 and the object to be recognized using the 3D image generated by the 3D camera 106, and calculates the distance based on the calculated distance. The information is stored in the camera memory 114 as information.

Since the 3D image generated by the 3D camera 106 is generated by receiving reflected light reflected from the object, that is, the subject, the size or shape of the object included in the 3D image according to the distance between the object and the 3D camera 106. May vary. In addition, the time at which the reflected light reflected from the recognition object reaches the 3D camera 106 may vary according to the distance between the recognition object and the 3D camera 106. The 3D image according to the present exemplary embodiment may include, as time information, a time required for the light emitted from the illumination unit 104 to be reflected by the object to be recognized and returned to the 3D camera 106.

The distance calculator 108 according to the present exemplary embodiment may calculate the distance between the 3D camera 106 and the object to be recognized using the time information. In addition, since the 3D camera 106 is mounted on the 3D distance information extracting apparatus 100, the distance calculated by the distance calculating unit 108 is substantially referred to as the distance between the 3D distance information extracting apparatus 100 and the recognition object. Can be.

Meanwhile, the 3D image generated by the 3D camera 106 is generated by the 3D camera 106 receiving the reflected light reflected from the object to be recognized, that is, the subject. Therefore, even if it is one recognition object, the time taken for the reflected light to reach the 3D camera 106 by the angle, the bending, etc. of the recognition object is different for each part of the recognition object. For example, assuming that the object to be recognized is a hand, the time taken for the reflected light from the first node of the finger to reach the 3D camera 106 and the time taken for the reflected light from the folded palm to reach the 3D camera 106 are Each one is different.

The distance calculating unit 108 is based on the arrival time of the reflected light, which is different depending on which part of the object is reflected as described above, that is, for each part of the 3D camera 106 and the object to be recognized based on time information of each part of the object to be recognized. You can also calculate the distance.

In addition, the distance calculating unit 108 may store, as the distance information, the three-dimensional distance information extracting apparatus 100 and the distance of each part of the object to be recognized as distance information under the control of the camera controller 110 as the distance information. have.

The camera controller 110 controls the overall operation of the 3D distance information extraction apparatus 100.

The camera controller 110 according to the present exemplary embodiment determines whether the recognition target is included in the 2D image generated by the 2D camera 102. When the object to be recognized is included in the 2D image, the camera controller 110 controls the lighting unit 104 and the 3D camera 106 to generate a 3D image of the object to be recognized. On the other hand, when the recognition object is not included in the 2D image, the camera controller 110 does not drive the lighting unit 104 and the 3D camera 106. As described above, the 3D distance information extraction apparatus 100 according to the present invention may drive the lighting unit 104 and the 3D camera 106 only when the recognition object is included in the 2D image. By driving the illumination unit 104 and the 3D camera 106 only when the recognition object is included in the 2D image, the three-dimensional distance information extraction apparatus 100 according to the present embodiment is the illumination unit 104 and the 3D camera 106. It is possible to save the power required to drive the.

According to an exemplary embodiment, the camera controller 110 controls the illumination unit 104 and the 3D camera 106 such that the illumination unit 104 and the 3D camera 106 are driven only when a recognition object is detected through the proximity sensor 112. can do. When the object to be detected is not detected through the proximity sensor 112, the camera controller 110 does not drive the lighting unit 104 and the 3D camera 106. By driving the illumination unit 104 and the 3D camera 106 only when the recognition object is detected through the proximity sensor 112 as described above, the three-dimensional distance information extraction apparatus 100 according to the present embodiment is the illumination unit 104 And power required to drive the 3D camera 106.

In addition, according to another exemplary embodiment, the camera controller 110 may control the 3D distance information extraction apparatus 100 to detect a recognition object using both the proximity sensor 112 and the 2D camera 102. When a specific object is detected through the proximity sensor 112, the camera controller 110 controls the 2D camera 102 to photograph the specific object to generate a 2D image including the specific object. The camera controller 110 determines whether the specific object detected by the proximity sensor 112 is an object of recognition using the 2D image generated by the 2D camera 102.

When a specific object photographed by the 2D camera 102 is a recognition target, that is, when a recognition target is included in the 2D image, the camera controller 110 operates the lighting unit 104 and the 3D camera 106 to Generate 3D images. On the other hand, when a specific object photographed by the 2D camera 102 is not a recognition target, that is, when the recognition target is not included in the 2D image, the camera controller 110 does not operate the lighting unit 104 and the 3D camera 106. Do not. As described above, the 3D distance information extracting apparatus 100 according to the present invention first detects the recognition target through the target detection unit 120, so that the illumination unit 104 and the lighting unit 104 do not know whether the object to be photographed is the recognition target. The power consumption can be reduced as compared to the conventional operation of operating the 3D camera 106.

The camera memory 114 stores various data for controlling the operation of the 3D distance information extraction apparatus 100.

In the camera memory 114 according to the present embodiment, identification data for identifying a recognition target may be stored. For example, assume that the recognition target is a user's finger. In the camera memory 114, identification data, for example, various images including a finger or feature descriptors of a finger may be stored in advance. The camera controller 110 compares the 2D image generated by the 2D camera 102 with the identification data. The camera controller 110 determines that the recognition object is included in the 2D image when the user's finger image is included according to the comparison result. At this time, the feature discriminator of the finger may be a feature for determining the shape of the finger (for example, the presence of a fingerprint, the presence of a fingernail, the shape of a finger), etc. in order to determine whether the recognition target is a finger.

According to an exemplary embodiment, the recognition data may be data previously input and stored by a user. The 2D camera 102 may store, for example, a 2D image generated by photographing a user's hand image in the camera memory 114 as recognition data.

According to an exemplary embodiment, the amount of light required from the lighting unit 104 according to the size of the recognition target may be stored in the camera memory 114. For example, when the size of the recognition target included in the 2D image is level 1 to 10 with reference to the camera memory 114, the camera controller 110 outputs the amount of light at level 1, and if the size is 11 to 20, the amount of light at level 2 and size. Is 21 to 30, the lighting unit 104 may be controlled to emit light having a light amount of level 3.

FIG. 2 is a flowchart illustrating an example of a 3D distance information extraction method of the apparatus for extracting 3D distance information shown in FIG. 1.

Referring to FIG. 2, first, the 3D distance information extraction apparatus 100 generates a 2D image by photographing at least one object using the 2D camera 102 (S202). When the 2D image is generated, the camera controller 110 determines whether a recognition target (for example, a user's hand) is included in the 2D image (S204).

As a result of the determination in step S204, if the recognition target is not included in the 2D image (S204: NO), the 2D camera 102 may continuously generate the 2D image by photographing at least one object. According to an exemplary embodiment, the camera controller 110 may control the 2D camera 102 to continue shooting until an image corresponding to the recognition target, for example, a 2D image including the recognition target, is generated. According to another exemplary embodiment, the camera controller 110 may control the 2D camera 102 to continuously generate a 2D image by repeating shooting every pre-stored time (for example, every minute).

In addition, according to an exemplary embodiment, the camera controller 110 determines whether all of the recognition targets are included in the plurality of 2D images photographed during the reference time (for example, 1 second) preset and stored in the camera memory 114. can do. If not all of the 2D images photographed during the reference time includes the recognition target, the camera controller 110 determines that the recognition target does not exist in the field of view range of the lens included in the 2D camera 102 and the 2D camera. The operation of 102 can be terminated.

 As a result of the determination in step S204, when the recognition target is included in the 2D image (S204: YES), the camera controller 110 controls the lighting unit 104 to extract the distance between the recognition target, that is, the recognition target and the three-dimensional distance information extraction device. The light is emitted to measure the distance between the 100 (S206). When light is emitted from the lighting unit 104 as described above, the 3D camera 106 generates a 3D image by photographing a recognition target (S208). The light emitted in step S206 is reflected by the recognition object, and the photographing sensor (not shown) of the 3D camera 106 captures the recognition object by receiving the reflected light reflected from the recognition object. The imaging of the 3D camera 106 in step S208 in the present embodiment can be performed by receiving the reflected light reflected from the recognition object and converting the received reflected light into a 3D image.

When a 3D image corresponding to the object to be recognized is generated through the 3D camera 106, the distance calculator 108 calculates a distance to the object to be recognized using the 3D image (S210). As a result, the 3D distance information extraction apparatus 100 may know the distance between the 3D distance information apparatus 100 and the recognition object.

3 is a flowchart illustrating another example of a 3D distance information extraction method of the apparatus for extracting 3D distance information shown in FIG. 1.

Referring to FIG. 3, first, a recognition object is detected using the proximity sensor 112 (S302). The proximity sensor 112 according to the present exemplary embodiment may detect that the recognition target enters a preset range (for example, within a radius of 10 cm) from the proximity sensor 112.

When the object to be detected is detected, the camera controller 110 controls the lighting unit 104 to emit light for measuring the distance between the object to be recognized, that is, the distance between the object and the three-dimensional distance information extracting apparatus 100 (S304). ). When light is emitted from the lighting unit 104 as described above, the 3D camera 106 generates a 3D image by photographing a recognition target (S306).

Light emitted from the illumination unit 104 in step S304 is reflected by the recognition object, and the photographing sensor (not shown) of the 3D camera 106 receives the reflected light reflected from the recognition object. In step S306 according to the present embodiment, the 3D camera 106 may be configured by receiving the reflected light reflected by the 3D camera 106 from the object to be recognized and converting the received reflected light into a 3D image.

When a 3D image including a recognition object is generated through the 3D camera 106, the distance calculating unit 108 uses the 3D image to determine a distance from the recognition object, that is, between the 3D distance information extracting apparatus 100 and the recognition object. The distance is calculated (S308).

4 is a flowchart illustrating still another example of a 3D distance information extraction method of the apparatus for extracting 3D distance information shown in FIG. 1.

Referring to FIG. 4, the apparatus for extracting 3D distance information 100 detects an object that enters within a sensing range (eg, within a radius of 10 cm) of the proximity sensor 112 using the proximity sensor 112. It is detected (S402). When the object is detected, the camera controller 110 generates a 2D image including the object by photographing the object by operating the 2D camera 102 (S404).

The object detected through the proximity sensor 112 in step S402 may not be a recognition target. If the object is not recognized, it is not necessary to generate a 3D image including the object. Therefore, prior to generating the 3D image including the object, the 3D distance information extraction apparatus 100 according to the present embodiment first determines whether the object detected by the proximity sensor 112 is a recognition target. In order to determine whether the object detected in step S402 is a recognition target, the camera controller 110 photographs the object in step S404 to generate a 2D image corresponding to the object, for example, a 2D image including the object. 2D camera 102 can be controlled. When a 2D image corresponding to the object detected in step S402 is generated, the camera controller 110 determines whether a recognition target (for example, a user's hand) is included in the 2D image (S406).

As a result of the determination in step S406, when the recognition target is not included in the 2D image (S406: NO), the camera controller 110 may perform the proximity sensing again to detect another object that has entered the sensing range. Control (S402).

According to another exemplary embodiment, the camera controller 110 recognizes when a specific object (eg, a user's hand) is not detected for a reference time (eg, 1 minute) preset and stored in the camera memory 114. It may be determined that the subject does not enter the sensing range of the proximity sensor 112. In addition, when it is determined that the object of recognition does not enter within the sensing range, the camera controller 110 may terminate the operation of the proximity sensor 112.

As a result of the determination in step S406, when the recognition object is included in the 2D image (S406: Yes), the camera controller 110 controls the lighting unit 104 to extract a distance between the recognition object, that is, the recognition object and the three-dimensional distance information extraction device. The light is emitted to measure the distance between the 100 (S408). When light is emitted from the lighting unit 104 as described above, the 3D camera 106 generates a 3D image corresponding to the recognition target by photographing the recognition target (S410). When a 3D image corresponding to the object to be recognized is generated through the 3D camera 106, the distance calculator 108 calculates a distance to the object to be recognized using the 3D image (S412).

FIG. 5 is a flowchart illustrating still another example of a 3D distance information extraction method of the apparatus for extracting 3D distance information shown in FIG. 1.

Referring to FIG. 5, first, the 2D camera 102 photographs at least one object to generate a 2D image corresponding to the object, for example, a 2D image including the object (S502). When the 2D image is generated, the camera controller 110 determines whether a recognition target (for example, a user's hand) is included in the 2D image (S504).

As a result of the determination in step S504, when the recognition object is not included in the 2D image (S504: NO), the 2D camera 102 may continuously generate at least one object to generate the 2D image. According to an exemplary embodiment, the camera controller 110 may control the 2D camera 102 to continue shooting until a 2D image including the object of recognition is generated.

As a result of the determination in step S504, when the recognition target is included in the 2D image (S504: YES), the camera controller 110 determines the amount of light to be emitted from the lighting unit 104 based on the size of the recognition target included in the 2D image. (S506)

In the present embodiment, the camera controller 110 may determine an approximate distance between the 3D distance information extraction apparatus 100 and the object to be recognized using the 2D image generated in operation S502. For example, it is assumed that the total number of pixels of the 2D image generated by the 2D camera 102 is '800 × 600' pixels, that is, 480,000 pixels. As shown in Table 1 below, the camera memory 114 has a distance between the object to be recognized and the apparatus for extracting 3D distance information 100 according to the size of the object to be recognized, and the amount of light required by the illumination unit 104 to generate a 3D image for each distance. May be stored.

Recognition target size (pixel) Distance to recognition object Required light quantity 10,001 to 50,000 1.1 to 3.0 Level 1 50,001 to 100,000 3.1 to 6.0 Level 2 100,001-150,000 6.1 to 9.0 Level 3 150,001 to 200,000 9.1-12.0 Level 4 ... ... ...

Referring to Table 1 as an example, assuming that the size of the recognition object included in the 2D image generated in step S502 is 120,000 pixels, the distance between the recognition object and the 3D distance information extraction apparatus 100 becomes 6.1 to 9.0 cm. . In addition, the required amount of light to be emitted from the lighting unit 104 is Level 3 to obtain a 3D image corresponding to the recognition target located between 6.1 and 9.0 cm. The camera controller 110 according to the present exemplary embodiment may determine the amount of light to be emitted from the lighting unit 104 by using information previously set and stored in the camera memory 114 as shown in Table 1 above.

When the amount of light to be emitted from the lighting unit 104 is determined, the lighting unit 104 determines the distance of the light amount determined in step S506 to measure the distance between the object to be recognized, that is, the distance between the 3D distance information extracting apparatus 100 and the object to be recognized. The light is emitted toward the recognition target (S508). When light is emitted from the lighting unit 104, the 3D camera 106 photographs a recognition object to generate a 3D image corresponding to the recognition object (S510). The light emitted in step S508 is reflected by the object to be recognized, and a photographing sensor (not shown) of the 3D camera 106 may generate a 3D image by receiving and converting the reflected light reflected from the object to be recognized.

When a 3D image corresponding to the object to be recognized, for example, a 3D image including the object to be recognized is generated through the 3D camera 106, the distance calculator 108 calculates a distance from the object to be recognized using the 3D image ( S512).

6 is a block diagram illustrating a configuration of a terminal including the apparatus for extracting 3D distance information shown in FIG. 1.

As illustrated in FIG. 6, each component of the 3D distance information extracting apparatus 100 may be implemented in a form embedded in the terminal 600. Referring to FIG. 6, the terminal 600 includes a 2D camera 102 and a proximity sensor 112, a target detector 120, an illumination unit 104, a 3D camera 106, a distance calculator 108, and a camera controller. 110, a camera memory 114, a user input unit 604, a display unit 606, a memory 608, a control unit 610, and may further include a communication interface 602.

In FIG. 6, the object detector 120 includes both the 2D camera 102 and the proximity sensor 112 for convenience of description. However, according to an exemplary embodiment, the object detector 120 may include the 2D camera 102 and the 2D camera 102. It may be implemented by including at least one of the proximity sensors 112. Also, the camera controller 110 may be implemented in a form included in the controller 610, and the camera memory 114 may be implemented in a form included in the memory 608.

The communication interface 602 performs wired or wireless communication of the terminal 600. The communication interface 602 according to the present embodiment may perform short-range wireless communication such as Bluetooth, Wi-Fi, and the like. The communication interface 602 may also receive identification data for identifying a recognition object by performing wired or wireless communication.

According to an embodiment, the communication interface 602 may receive a call from another terminal or receive a message (eg, a short message service (SMS)).

The user input unit 604 receives an input from a user. The user input unit 604 according to the present exemplary embodiment may receive a user input for measuring the distance between the terminal 600 and the recognition target. In addition, the user input unit 604 may receive a user input for receiving a gesture input.

In the present embodiment, 'gesture input' refers to a user input that is input by, for example, a user making a specific gesture for controlling the terminal 600 by using a hand. That is, the user may input a gesture input to the terminal 600 by using a hand to take a gesture corresponding to any one of gesture inputs preset and stored in the memory 608. According to an embodiment, the user drives the 3D distance information extraction apparatus 100 of the terminal 600, and the lens of the 2D camera 102 or the 3D camera 106 included in the 3D distance information extraction apparatus 100. The gesture input may be input to the terminal 600 by taking a gesture toward the terminal 600. The terminal 600 according to the present exemplary embodiment receives a user input for receiving a gesture input from a user, that is, a user input for driving the 3D distance information extracting apparatus 100, and thus the 3D distance information extracting apparatus 100. Can be driven.

The display unit 606 displays various data stored in the terminal 600. According to an exemplary embodiment, the display 606 may display a gesture input input from a user in real time. In addition, the terminal 600 may display various data for performing an operation corresponding to the gesture input. For example, if a gesture input is for playing a video file stored in the memory 608, the display 606 may play and display the video under the control of the controller 610.

The memory 608 stores various data for controlling the terminal 600. The memory 608 according to the present embodiment may store a plurality of gesture inputs and commands or operations corresponding to each of the gesture inputs.

The controller 610 controls the overall operation of the terminal 600. The controller 610 according to the present exemplary embodiment may be implemented by including the camera controller 110 of the 3D distance information extraction apparatus 100 to perform various operations executed by the camera controller 110. The controller 610 determines the shape of the object to be recognized based on the 3D image generated through the 3D camera 106 and the distance information generated by the distance calculator 108, and inputs a gesture input corresponding to the shape of the object to be recognized. You can decide. In addition, when the gesture input is determined, the controller 608 determines and performs an operation of the terminal 600 corresponding to the gesture input.

According to another exemplary embodiment, the controller 610 may determine the shape of the object to be recognized using only the 3D image. Even when the shape of the object to be recognized is determined using only the 3D image generated by the 3D camera 106, the controller 610 may determine a gesture input corresponding to the 3D image. In addition, the terminal 600 may be controlled to perform an operation according to the gesture input.

According to an exemplary embodiment, the controller 610 may determine whether to drive the 3D distance information extraction apparatus 100 to execute a motion recognition function according to the state of the terminal 600. When the terminal 600 maintains a standby state or when the terminal 600 maintains a ready state, the controller 610 determines whether to execute a motion recognition function.

The gesture recognition function refers to a function of allowing the terminal 600 to recognize a user's gesture input to the terminal 600 as a gesture input using the 3D distance information extracting apparatus 100. The standby state refers to a state in which the terminal 600 is powered on, but no other event (eg, a state in which a user input is not input) does not occur in the terminal 600. The ready state indicates a state in which the terminal 600 executes a function other than the motion recognition function. For example, if the terminal 600 is transmitting or receiving a message, accessing the Internet, displaying an e-book, playing music, playing a game, or the like, the terminal 600 is ready.

According to an embodiment, the controller 610 may determine whether a wake up event occurs in the terminal 600. For example, when the terminal 600 which is in the standby state receives a call or receives a message, the controller 610 may determine that a wake up event has occurred. In addition, even when the terminal 600, which has maintained the standby state, receives a user input through the user input unit 604, the controller 610 may determine that a wake up event has occurred. In this case, the user input may be a user input for executing a gesture recognition function or a user input for executing a function other than the gesture recognition function. In addition, when a wake up event occurs, the state of the terminal 600 may be switched from the standby state to the ready state under the control of the controller 610.

When the wake-up event occurs in the terminal 600 as described above, the controller 610 may drive the 3D distance information extraction apparatus 100 to control the terminal 600 to receive a gesture input from the user. . When a gesture input is input, the controller 610 controls the terminal 600 to perform an operation corresponding to the gesture input.

Even when the terminal 600 maintains the ready state, the controller 610 may determine whether to execute the motion recognition function. According to an embodiment, when a user input for executing a gesture recognition function is input from the user input unit 604, the controller 610 may control the terminal 600 to execute the gesture recognition function. In addition, even when the terminal 600 is switched from the standby state to the ready state, the controller 610 may execute a motion recognition function.

According to an embodiment, when the terminal 600 is in the ready state, the controller 610 may determine whether a user input for executing a gesture recognition function is input. When the user input is input, the controller 610 controls the terminal 600 to execute a motion recognition function. For example, the terminal 600 may receive a user input for executing a gesture recognition function from a user while playing a music file. When the user input is input as described above, the controller 610 may execute the 3D distance information extraction apparatus 100 to receive a gesture input from the user. According to another exemplary embodiment, the terminal 600 is ready. In this case, it may be determined whether the recognition target enters within a preset range (eg, within 10 cm from the terminal 600) from the target extractor 120. When the recognition target enters within the set range, the controller 610 may control the terminal 600 to execute a motion recognition function.

In the ready state, for example, when a recognition target (for example, a user's hand) enters an angle of view of the lens of the 2D camera 102 or when a recognition target is detected in the proximity sensor 112, the controller 610. The terminal 600 may control the terminal 600 to execute a gesture recognition function. That is, the terminal 600 according to the present embodiment may, for example, play a music file and receive a gesture input from a user. According to an exemplary embodiment, the controller 610 may execute the gesture recognition function when the object to be recognized exists within the angle of view of the lens of the 2D camera 102 more than a time (for example, 1 second) previously stored in the memory 608. 600 can be controlled. In addition, the controller 610 may control the terminal 600 to execute the motion recognition function when the object to be detected is continuously detected by the proximity sensor 112 for more than a predetermined time (eg, 1 second) or more.

For example, it is assumed that the terminal 600 is in a ready state playing a music file. The user may gesture to allow the object to be recognized, that is, the hand, to exist within the angle of view of the lens of the 2D camera 102 for at least 1 second. The 2D camera 102 may detect a recognition object, and the controller 610 may control the terminal 600 to execute a motion recognition function. When the gesture recognition function is executed, the terminal 600 receives a gesture input from the user. Since the terminal 600 is playing the music file, the user may input a gesture input associated with playing the music file to the terminal 600. For example, the user may input play, pause, forward, rewind, stop, and the like into the terminal 600 as a gesture input. In addition, the user may input a gesture input, which is not related to playing the music file, to the terminal 600. For example, in a ready state in which a music file is being played, the terminal 600 may receive a gesture input for executing another application (for example, an internet browser, a game, an e-book, a call origination, etc.) from the user. have. In this case, the controller 610 controls the terminal 600 to play a music file and to drive another application.

In the standby state or the ready state, the controller 610 may control the 3D distance information extracting apparatus 100 to detect a recognition object every predetermined time (for example, 2 seconds). In addition, when the recognition object is detected, it may be determined whether the recognition object stays within the recognition range of the 3D distance information extraction apparatus 100 for a predetermined time (for example, 1 second) or more. If the recognition target stays within the recognition range for a predetermined time or more, the controller 610 assumes that the user wants to input a gesture input and executes the recognition target function. The recognition range indicates the range in which the object of view of the lens of the 2D camera 102 or the object of recognition by the proximity sensor 112 can be detected.

According to an embodiment, the controller 610 determines whether the user gazes at the display unit 606 of the terminal 600 using the 2D camera 102, and executes the motion recognition function when the gaze is gazed. ) Can also be controlled. When the 2D camera 102 is implemented to face the user of the terminal 600, the user's face may be located within the angle of view of the 2D camera 102. Also, the 2D camera 102 may receive a user's face through the lens and convert the user's face into a 2D image (hereinafter, referred to as a face image). The controller 610 may control the operation of the terminal 600 according to a face pose of the user or movement of the pupil included in the face image. For example, when the eyes of the user included in the face image are closed, the controller 610 may turn off the power of the terminal 600, and when the pupil included in the face image moves downward, the display unit 606 You can scroll down.

FIG. 7 is a flowchart illustrating an example of a method in which a terminal illustrated in FIG. 1 performs an operation corresponding to a gesture input.

Referring to FIG. 7, the controller 610 of the terminal 600 detects a recognition target by using a 2D image (S702). To this end, the controller 610 may control the 2D camera 102 to generate a 2D image corresponding to the specific object by photographing a 2D recognition target. Since the controller 610 may be implemented to include the camera controller 110, it may be determined whether the 2D image generated by the 2D camera 102, that is, the 2D image corresponding to the specific object is a 2D image corresponding to the recognition target. can do. The controller 610 may determine whether the recognition object is included in the 2D image, thereby determining whether the 2D image is a 2D image corresponding to the recognition object. When the 2D image includes an object to be recognized, the controller 610 controls the lighting unit 104 and the 3D camera 106 to generate a 3D image corresponding to the object to be recognized (S704).

The controller 610 controls the distance calculator 108 to calculate a distance between the object to be recognized, that is, a distance between the terminal 600 and the object to be recognized using the 3D image (S706). The distance calculated by the distance calculator 108 in step S706 is a distance between the 3D camera 106 and the object to be recognized, and may be substantially a distance between the terminal 600 on which the 3D camera 106 is installed and the object to be recognized. The distance calculator 108 calculates the distance between the 3D camera 106 and the object to be recognized and generates distance information including the calculated distance value. In addition, the distance calculator 108 may store the distance information in the camera memory 114.

The controller 610 of the terminal 600 may determine a shape of a recognition object photographed by the 3D camera 106 and a gesture of the recognition object based on the distance information and the 3D image. The controller 610 determines a gesture input corresponding to the gesture of recognition (S708).

The controller 610 according to the present exemplary embodiment compares the gesture inputs preset and stored in the memory 608 with the type of the recognition object photographed by the 3D camera 106, that is, the gesture, to be identical to (or similar to) the gesture. It may be determined whether a gesture input having a shape exists in the memory 608. If the gesture input having the same shape as the gesture of the recognition target is stored among the gesture inputs preset and stored in the memory 608, the controller 610 may determine the gesture of the recognition target as the gesture input. On the other hand, if the gesture input having the same shape as the gesture of the recognition target is not stored in the memory 608, it may be determined that the gesture of the recognition target is not a gesture input. If it is determined that the gesture to be recognized is not a gesture input, the controller 610 may control the terminal 600 not to perform a separate operation.

As described above, the controller 610 determines whether the shape of the recognition object captured by the 3D camera 106, that is, the gesture of the recognition object is one of preset gesture inputs. In addition, when the gesture to be recognized is determined as one of preset gesture inputs, the controller 610 may search the memory 608 to determine a command or operation of the terminal 600 corresponding to the gesture input. have.

When the gesture input corresponding to the gesture of the recognition target is determined in step S708, the controller 610 determines and performs an operation corresponding to the gesture input (S710).

FIG. 8 is a flowchart illustrating another example of a method in which the terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.

Referring to FIG. 8, the control unit 610 of the terminal 600 detects a recognition object through the proximity sensor 112 (S802). As illustrated in FIG. 6, the control unit 610 of the terminal 600 may include the camera control unit 110, so that the recognition object detected by the proximity sensor 112 may be confirmed. When the object to be detected is detected by the proximity sensor 112 as described above, the controller 610 controls the lighting unit 104 and the 3D camera 106 to generate a 3D image corresponding to the object to be recognized (S804).

The controller 610 controls the distance calculator 108 to calculate a distance between the object to be recognized, that is, a distance between the terminal 600 and the object to be recognized using the 3D image (S806). The distance between the 3D camera 106 and the object to be recognized may be stored in the camera memory 114 as distance information. The controller 610 of the terminal 600 determines the shape of the recognition object photographed by the 3D camera 106 based on the distance information and the 3D image. The controller 610 determines a gesture input corresponding to the type of the recognition target, that is, the gesture of the recognition target (S808). When the gesture input is determined, the controller 610 determines and performs an operation corresponding to the gesture input (S810).

9A is a diagram illustrating a terminal for detecting a recognition object using a 2D camera, and FIG. 9B is a diagram for a terminal for detecting a recognition object using a proximity sensor. It is assumed that the lighting unit 104 of FIG. 9A and FIG. 9B, the 3D camera 106, the 2D camera 102 of FIG. 9A, and the proximity sensor 112 of FIG. 9B are all disposed in front of the terminal 600.

Referring to FIG. 9A, a user's hand 900 that is a recognition target may be first detected through a 2D image generated by the 2D camera 102. When the user's hand 900 is detected, the lighting unit 104 of the terminal 600 generates a 3D image corresponding to the hand 900, for example, a 3D image including the hand 900. (Shown in solid line). The light emitted from the illumination unit 104 is reflected by the hand 900 that is the object to be recognized, and the reflected light (shown by the dotted line) reflected by the hand 900 is input to the 3D camera 106. The 3D camera 106 receives the reflected light reflected by the hand 900 to generate a 3D image of the hand 900.

Referring to FIG. 9B, the user's hand 900 that is a recognition target is first detected by the proximity sensor 112. When the user's hand 900 is detected, the illumination unit 104 of the terminal 600 emits light (shown in solid line) to generate a 3D image of the hand 900. The light emitted from the illumination unit 104 is reflected by the hand 900 that is the object to be recognized, and the reflected light (shown by the dotted line) reflected by the hand 900 is input to the 3D camera 106. The 3D camera 106 receives the reflected light reflected by the hand 900 and generates a 3D image corresponding to the hand 900.

FIG. 10 is a flowchart illustrating an example of a method in which the terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.

Referring to FIG. 10, the terminal 600 maintains a standby state (S1002), and when a wake-up event occurs (S1004), the target detector 120 of the 3D distance information extracting apparatus 100 is used. In operation S1006, the recognition target is detected.

In the present embodiment, the wake up event refers to an event for changing the state of the terminal 600 which has maintained the standby state. The wake up event indicates, for example, receiving a key input or a touch input from a user, the terminal 600 receiving a call, receiving a message, or the like. In the present exemplary embodiment, the 3D distance information extracting apparatus 100 may perform an operation for detecting a distance before the wake-up event occurs, that is, when the terminal 600 is in a standby state (for example, the 2D camera 102 may be operated. Operation, operation of the proximity sensor 112, light emission operation of the lighting unit 104, operation of the 3D camera 106, etc.) may not be performed.

According to an embodiment, in step S1006, the terminal 600 may detect the recognition object using the 2D camera 102 of the object detector 120. Accordingly, the controller 610 of the terminal 600 may drive the 2D camera 102 to capture a recognition object, thereby generating a 2D image including the recognition object. In addition, the controller 610 determines whether a recognition target is included in the 2D image. When the recognition target is included, the controller 610 controls the illumination unit 104 to emit light for measuring a distance from the recognition target to generate a 3D image of the recognition target. The 3D camera 106 photographs the recognition object by receiving light reflected from the recognition object.

According to another exemplary embodiment, in operation S1006, the terminal 600 may detect a recognition object using the proximity sensor 112 of the object detection unit 120. Accordingly, the proximity sensor 112 may detect that the object of recognition enters a preset range (for example, within 10 cm of the radius) from the proximity sensor 112. When the object to be detected is detected, the camera controller 110 controls the lighting unit 104 to emit light for measuring the distance between the object to be recognized, that is, the distance between the object and the three-dimensional distance information extraction apparatus 100. The 3D camera 106 photographs the recognition object by receiving light reflected from the recognition object.

When the object to be detected is detected as described above, the terminal 600 generates a 3D image corresponding to the object to be recognized (S1008). The distance calculator 108 calculates the distance between the recognition object, that is, the distance between the terminal 600 and the recognition object, using the 3D image (S1010). The distance between the 3D camera 106 and the object to be recognized may be stored in the camera memory 114 as distance information.

The controller 610 of the terminal 600 determines the shape of the recognition object photographed by the 3D camera 106 based on the distance information and the 3D image. The controller 610 determines a gesture input corresponding to the type of the recognition target, that is, the gesture of the recognition target (S1012). When the gesture input is determined, the controller 610 determines and performs an operation corresponding to the gesture input (S1014).

According to an embodiment, the terminal 600 may maintain a standby state and determine whether a wake-up event has occurred in the terminal 600 at every pre-stored time.

FIG. 11 is a flowchart illustrating another example of a method in which a terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.

Referring to FIG. 11, first, the terminal 600 maintains a standby state (S1102). In the present exemplary embodiment, the standby state refers to a state in which the power of the terminal 600 is turned on but no other operation is performed. The controller 610 determines whether a wake up event occurs in the terminal 600 (S1104).

When the wakeup event is generated as a result of the determination of step S1104 (S1104: YES), the terminal 600 is switched to the ready state (S1106). The wake up event may be, for example, receiving a call, receiving a message, or receiving a user input to transition to a ready state. In the ready state, the terminal 600 executes a motion recognition function under the control of the controller 610 (S1108). In this case, the controller 610 may recognize the user's gesture by using the 3D distance information extraction apparatus 100.

The terminal 600 receives a gesture input from the user (S1110) and performs an operation corresponding to the corresponding gesture input (S1112).

On the other hand, when the wakeup event is not generated as a result of the determination of step S1104 (S1104: NO), the terminal 600 maintains a standby state (S1102).

FIG. 12 is a flowchart illustrating still another example of a method in which the terminal illustrated in FIG. 6 performs an operation corresponding to a gesture input.

Referring to FIG. 12, first, the terminal 600 maintains a ready state (S1202). In this embodiment, the ready state indicates a state in which the terminal 600 executes a function other than the motion recognition function or a state after receiving a user input for executing various functions including the motion recognition function. In the ready state, the controller 610 determines whether an object to be detected is detected within the recognition range of the object detector 120 (S1204).

When the recognition object is detected within the recognition range in the determination result of step S1204 (S1204: YES), the terminal 600 executes a motion recognition function (S1206). In this case, the controller 610 may recognize the user's gesture as the gesture input by using the 3D distance information extraction apparatus 100. The terminal 600 receives a gesture input from the user (S1208), and performs an operation corresponding to the corresponding gesture input (S1210).

On the other hand, if the recognition object is not detected within the recognition range as a result of the determination in step S1204 (S1204: NO), the terminal 600 maintains the ready state (S1202).

It will be appreciated that the method of extracting 3D distance information according to an embodiment of the present invention may be realized in the form of hardware, software, or a combination of hardware and software. Such arbitrary software may be stored in a memory such as, for example, a volatile or non-volatile storage device such as a storage device such as ROM or the like, or a memory such as a RAM, a memory chip, a device or an integrated circuit, , Or a storage medium readable by a machine (e.g., a computer), such as a CD, a DVD, a magnetic disk, or a magnetic tape, as well as being optically or magnetically recordable. The method of extracting 3D distance information according to an embodiment of the present invention may be implemented by a computer or a portable terminal including a controller and a memory, wherein the memory includes a program or instructions including implementing the embodiments of the present invention. It will be appreciated that this is an example of a machine-readable storage medium suitable for storing programs. Accordingly, the invention includes a program comprising code for implementing the apparatus or method as claimed in any of the claims, and a storage medium readable by a machine (such as a computer) for storing such a program. In addition, such a program may be electronically transported through any medium such as a communication signal transmitted via a wired or wireless connection, and the present invention appropriately includes the same.

In addition, the apparatus for extracting 3D distance information or the terminal including the apparatus may receive and store the program from a program providing apparatus connected by wire or wirelessly. The program providing apparatus includes a memory for storing a program, information necessary for a content protection method, and the like, including instructions for causing the 3D distance information extracting device or a terminal including the device to perform a preset content protection method; A communication unit for performing wired or wireless communication of a 3D distance information extracting device or a terminal including the device, and a request or automatic transmission of a corresponding program or a corresponding program of the 3D distance information extracting device or a terminal including the device It may include a control unit for transmitting to.

The scope of the present invention should not be limited by the embodiments described, but should be determined by equivalents of the claims and the appended claims.

100: 3D distance information extraction device 102: 2D camera
104: lighting unit 106: 3D camera
108: distance calculation unit 110: camera control unit
112: camera memory 114: proximity sensor
120: target detection unit 600: terminal
602: communication interface 604: user input unit
606: display unit 608: memory
610: control unit

Claims (24)

Determining whether a recognition object exists within a preset range,
Generating a 3D image of the recognition object when the recognition object exists within the preset range;
And calculating the distance from the object to be recognized using the 3D image. The method of claim 1,
The process of determining whether the recognition object exists within the preset range,
Receiving a video input through the lens to generate a 2D image,
Determining whether the recognition object is included in the 2D image;
And determining that there is the recognition object within the preset range when the recognition object is included in the 2D image. 3D distance information extraction method of the 3D distance information extraction apparatus. The method of claim 1,
The process of determining whether the recognition object exists within the preset range,
Detecting a specific object entered within the preset range;
And determining that there is the object of recognition within the preset range when the specific object is detected. 3D distance information extraction method of a 3D distance information extraction apparatus. The method of claim 1,
The process of determining whether the recognition object exists within the preset range,
Detecting a specific object entered within the preset range;
Generating a 2D image including the specific object when the specific object is detected;
Determining whether the recognition object is included in the 2D image;
And determining that there is the recognition object within the preset range when the recognition object is included in the 2D image. 3D distance information extraction method of the 3D distance information extraction apparatus. A camera controller which determines whether a recognition object exists within a preset range,
A 3D camera generating a 3D image of the recognition object when the recognition object exists within the preset range;
And a distance calculator configured to calculate a distance from the object to be recognized using the 3D image. 6. The method of claim 5,
Further comprising a 2D camera for receiving an image input through the lens to generate a 2D image,
The camera control unit,
It is determined whether the recognition object is included in the 2D image, and when the recognition object is included in the 2D image, the 3D distance information is determined to be within the preset range. Extraction device. 6. The method of claim 5,
Further comprising a proximity sensor for detecting a specific object entered within the predetermined range,
The camera control unit,
And determining that the recognition object is within the preset range when the specific object enters within the preset range. 6. The method of claim 5,
A proximity sensor that detects a specific object entered within the preset range;
And a 2D camera for photographing the specific object through a lens to generate a 2D image corresponding to the specific object.
The camera control unit,
When the specific object enters within the preset range, the 2D camera is controlled to generate a 2D image corresponding to the specific object,
It is determined whether the recognition object is included in the 2D image, and when the recognition object is included in the 2D image, the 3D distance information is determined to be within the preset range. Extraction device. An object detecting unit which detects a recognition object existing within a preset range,
A 3D camera generating a 3D image corresponding to the recognition object;
A distance calculator configured to calculate a distance from the object to be recognized using the 3D image;
And a controller configured to determine a gesture input corresponding to a shape of the object to be recognized based on the distance between the 3D image and the object to be recognized, and to perform an operation according to the gesture input. 10. The apparatus according to claim 9,
Search for a gesture input including a gesture that matches the shape of the object to be recognized among prestored gesture inputs, and correspond to a shape of the object to be recognized as a result of the search; The terminal characterized in that determined by the gesture input. 10. The method of claim 9,
The target detection unit,
It includes a 2D camera for receiving an image input through the lens to generate a 2D image,
The control unit,
And determining whether the recognition target is included in the 2D image, and when the recognition target is included in the 2D image, determining that the recognition target is within the preset range. 10. The method of claim 9,
The target detection unit,
Proximity sensor for detecting a specific object entered within the preset range,
The camera control unit,
And when the specific object enters within the preset range, determining that the recognition target is within the preset range. 10. The method of claim 9,
The target detection unit,
A proximity sensor that detects a specific object entered within the preset range;
And a 2D camera for photographing the specific object through a lens to generate a 2D image of the specific object.
The control unit,
When the specific object is detected by the proximity sensor, the 2D camera is controlled to generate a 2D image of the specific object, it is determined whether the recognition target is included in the 2D image, and the 2D image is And when the recognition target is included, determining that the recognition target is within the preset range. 10. The apparatus according to claim 9,
And determining whether a wake-up event has occurred in the terminal in a standby state, and if the wake-up event occurs, controlling the target detector to detect the recognition object. 10. The apparatus according to claim 9,
And when the terminal is in the ready state, controlling the target detection unit to detect the recognition target. 16. The apparatus of claim 15,
And if the recognition object is not detected within a preset reference time, determining that the recognition object does not exist within the preset range. In the method for the terminal to perform an operation corresponding to the gesture input,
Detecting a recognition object existing within a preset range;
Generating a 3D image corresponding to the object to be recognized;
Calculating a distance from the object to be recognized using the 3D image;
Determining a gesture input corresponding to a shape of the object to be recognized based on the distance between the 3D image and the object to be recognized;
And a step of performing an operation according to the gesture input. 18. The method of claim 17,
The detecting of the recognition object existing within the preset range may include: receiving a video input through a lens and generating a 2D image;
Determining whether the recognition object is included in the 2D image;
If the recognition object is included in the 2D image, determining that the recognition object is within the preset range. 18. The method of claim 17,
The detecting of the recognition object existing within the preset range may include: detecting a specific object that has entered within the preset range;
And when the specific object is detected, determining that the recognition target is within the preset range. 18. The method of claim 17,
Detecting a recognition object existing within the preset range,
Detecting a specific object entered within the preset range;
Generating a 2D image including the specific object when the specific object is detected;
Determining whether the recognition object is included in the 2D image;
If the recognition object is included in the 2D image, determining that the recognition object is within the preset range. 18. The method of claim 17,
The determining of the gesture input corresponding to the shape of the object to be recognized is based on the distance between the 3D image and the object to be recognized.
Searching for a gesture input including a gesture that matches the shape of the object to be recognized among preset gesture inputs;
And determining, as a result of the search, a gesture input including a gesture corresponding to the shape of the object to be recognized as a gesture input corresponding to the shape of the object to be recognized. 18. The method of claim 17,
Before the process of detecting the recognition object existing within the preset range,
Determining whether a wake-up event has occurred in the terminal in a standby state;
Detecting a recognition object existing within the preset range,
And detecting, by the at least one of the 2D camera and the proximity sensor, the recognition object entering the predetermined range when the wake up event occurs. 18. The method of claim 17,
Before the process of determining whether a recognition object exists within the preset range,
If the terminal is in the ready state, using the at least one of the 2D camera and the proximity sensor further comprises the step of detecting the recognition object entered into the preset range. 24. The method of claim 23,
And if the recognition object is not detected within the preset reference time, determining that the recognition object does not exist within the preset range.

Images