KR102437979B1 - Apparatus and method for interfacing with object orientation based on gesture - Google Patents

Abstract

A gesture-based object-oriented interfacing method of the present invention comprises: an input step of receiving image data from a camera device; an object recognition step of parsing the image data and recognizing an object; and an execution control step of controlling a specific command to be executed according to an event generated by the object. The object recognition step includes: a vertical object recognition step of recognizing a first vertical object having a shape extending more than a first reference length in a vertical direction from the image data; a point object recognition step of recognizing a point object located on either the left side or the right side of a first vertical line corresponding to the first vertical object; and an event detection step of detecting a first event in which the point object moves to the opposite side with respect to the first vertical line. In the execution control step, when the first event is detected, a command mapped to the first event is controlled to be executed. The object recognition rate can be dramatically improved.

Description

Gesture-based object-oriented interfacing method and device

The present invention relates to a gesture-based interfacing method and the like, and more particularly, by organically grafting the mutual relationship between absolute and relative movement of an object to be recognized, interfacing between a user and an electronic device or It relates to a gesture-based object-oriented interfacing method that can implement an interaction more clearly and precisely.

In order to meet the user's heightened needs and to further enhance user convenience and usability of the device, poses, actions, gestures, and specific shapes or movements of body parts (hands, arms, etc.) (hereinafter referred to as 'gestures') Gesture recognition technology that recognizes image matching, artificial intelligence, deep learning, and deep neural network (DNN) models and controls interfacing with electronic devices based on this is widely applied.

This gesture recognition technology has the advantage of not requiring additional means for interfacing as well as the basic features of the non-contact method, and has the advantage of being implemented even at a remote distance.

Looking at the prior art, from a simple technique that detects only the overall movement of a hand or arm and applies it to interfacing, a fairly complex object recognition algorithm system is applied to image data generated from a camera device to recognize delicate motions or shape features. Various embodiments up to techniques are disclosed.

However, since the latter techniques are based on the consistency of unique and complex gestures of human objects as in Korea Patent Publication No. 10-1785650, or because complex recognition algorithms are applied, the clarity of gesture recognition is ensured if the shape features of the object are not precisely recognized. It is difficult to do so, and interfacing response delay, operation stop, malfunction, etc. may occur frequently, and it can be said that it is difficult to implement immediate responsiveness in operation processing.

Of course, some of these problems may be solved if expensive hardware resources with high resolution (sensor, CPU, camera, etc.) are sufficiently supported. It can lead to another intrinsic problem that limits its versatility and accessibility.

In addition, in the case of the prior art, since unfamiliar gestures are mainly applied to the user to improve resolution, etc., it can be said that the user convenience is low because it is difficult for the user to intuitively adapt unless continuously learning with a sense of purpose.

Korean Patent Publication No. 10-1785650 (2017.09.29.) Korean Patent Publication No. 10-1812605 (2017.12.20) Korean Patent Publication No. 10-2320754 (October 27, 2021)

The present invention was devised to solve the above-mentioned problems in the background as described above, and optimizes user convenience by intuitive recognition based on a gesture familiar to the user, and dynamically changes the relative positional relationship of individual objects. The purpose is to provide a gesture-based object-oriented interfacing method that can dramatically improve computational efficiency as well as object recognition rate by effectively grafting it to object recognition.

Other objects and advantages of the present invention may be understood by the following description, and will be more clearly understood by the embodiments of the present invention. In addition, the objects and advantages of the present invention can be realized by the configuration shown in the claims and the combination of the configuration.

According to an embodiment of the present invention for achieving the above object, a gesture-based object-oriented interfacing method includes an input step of receiving image data from a camera device; an object recognition step of recognizing an object by parsing the image data; and an execution control step of controlling a specific command to be executed according to an event generated by the object.

Specifically, the object recognition step may include: a vertical object recognition step of recognizing a first vertical object having a shape extending more than a first reference length based on a vertical direction in the image data; a point object recognition step of recognizing a point object located on one of the left and right sides with respect to a first vertical line corresponding to the first vertical object; and an event detecting step of detecting a first event in which the point object moves to the opposite side with respect to the first vertical line. In this case, the execution control step of the present invention is performed when the first event is detected. and execute a command mapped to the first event.

In addition, the step of recognizing the vertical object of the present invention is an object located on the opposite side of the point object among the left or right sides of the first vertical line, and has a shape extending in the vertical direction, more than the first reference length. It may be configured to further recognize a second vertical object having a small length.

In this case, it is preferable that the first event of the event detection step of the present invention is set to a case where the location information of the point object corresponds to a second vertical line formed by the second vertical object.

Preferably, in the event detection step of the present invention, a first sub-event in which the location information of the point object corresponds to an upper point of the second vertical line or the location information of the point object corresponds to a lower point of the second vertical line and detect one of the second sub-events. In this case, the execution control step of the present invention controls the command mapped to each of the first sub-event or the second sub-event to be executed.

Specifically, it is preferable that the first vertical object is an open index object among the user's fingers, the point object is a thumb object, and the second vertical object is a folded middle object.

According to an embodiment, the present invention provides a distance calculation step of calculating first distance information between eyes in the image data; an estimation step of calculating separation distance information between a display apparatus and a user by using a ratio between the average distance information between the actual eyes and the first distance information; and a recognition area setting step of setting a reference recognition area using the separation distance information and the size information of the display device.

In this case, the object recognition step of the present invention is preferably configured to use image data of an area corresponding to the reference recognition area.

In addition, the present invention provides a camera control step of controlling the camera device to be activated when the sound detected by the microphone module corresponds to a wake-up sound; and a directionality information input step of generating direction information, which is information on a location or directionality of the starting sound.

In this case, the object recognition step of the present invention performs processing for object recognition using n unit images in which the image data is divided into n (n is a natural number greater than or equal to 2) regions, but among the n unit images It may be configured to perform processing for object recognition with priority on a unit image corresponding to the directionality information.

Furthermore, the present invention may further include a recognition area setting step of setting an area formed by the extension line of the right angle shape as a reference recognition area when a hand object including a right angle shape is recognized at a position symmetrical in the diagonal direction. In this case, the object recognition step of the present invention may be configured to use image data of an area corresponding to the reference recognition area.

More preferably, the recognition region setting step of the present invention comprises: a position setting step of setting face position information at a time point when the reference recognition region is set; and a variable setting step of moving the position of the reference recognition area to correspond to the movement when the set face position information moves.

A gesture-based object-oriented interfacing apparatus according to another aspect of the present invention includes an input unit for receiving image data from a camera device; an object recognition unit for recognizing an object by parsing the image data; and an execution control unit for controlling a specific command to be executed according to an event generated by the object.

Specifically, the object recognition unit may include: a vertical object recognition unit for recognizing a first vertical object having a shape extending more than a first reference length in the vertical direction in the image data; a point object recognition unit for recognizing a point object located on one of the left and right sides with respect to a first vertical line corresponding to the first vertical object; and an event detector configured to detect a first event in which the location information of the point object moves to the opposite side with respect to the first vertical line.

In this case, when the first event is detected, the execution control unit of the present invention controls the command mapped to the first event to be executed.

According to a preferred embodiment of the present invention, since a gesture having a simple displacement characteristic is applied, the efficiency of computational processing can be further increased, and changes in the relative positional relationship between individual objects constituting the gesture can be accurately specified. And since it can be grafted to the interfacing, clarity and precision of interfacing or interaction between the user and the electronic device can also be implemented.

According to the present invention, there is no need to change the positional reference for gesture recognition or detection in time series because the relative positional relationship of individual objects that can move together and move independently at the same time are applied as a gesture according to the present invention. The efficiency of processing can be further improved.

In addition, according to an embodiment of the present invention, since a shape or movement corresponding to a UX familiar to the user is applied as a gesture, user convenience through intuitive recognition can be further optimized.

Furthermore, according to another embodiment of the present invention, the image of the region corresponding to the user's current location or direction, etc. is not applied to the entire image data or pre-processing for pre-selecting or specifying the region for object recognition as an object. Through processing that is preferentially used for recognition, it is possible to dramatically improve the processing speed of calculations, so that immediate responsiveness of interfacing can be realized.

The following drawings attached to the present specification illustrate preferred embodiments of the present invention, and serve to more effectively understand the technical spirit of the present invention together with the detailed description of the present invention to be described later, so the present invention is described in these drawings It should not be construed as being limited only to the matters.
1 is a block diagram showing a detailed configuration of an interfacing device according to a preferred embodiment of the present invention;
Figure 2 is a block diagram showing the detailed configuration of the object recognition unit shown in Figure 1;
3 and 4 are flow charts illustrating a processing procedure for embodiments of the present invention;
5 and 6 are views for explaining detailed configurations of vertical objects and point objects and events generated based on them;
7 is a flowchart illustrating a processing process of the present invention in which a reference recognition area for object recognition is set;
8 is a flowchart illustrating a processing process of the present invention in which sound direction information and the like are utilized;
9 is a view for explaining another embodiment of the present invention in which a reference recognition area for object recognition is set.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, the terms or words used in the present specification and claims should not be construed as being limited to conventional or dictionary meanings, and the inventor should properly understand the concept of the term in order to best describe his invention. Based on the principle that it can be defined, it should be interpreted as meaning and concept consistent with the technical idea of the present invention.

Therefore, the configuration shown in the embodiments and drawings described in the present specification is only the most preferred embodiment of the present invention and does not represent all the technical spirit of the present invention, so various equivalents that can replace them at the time of the present application It should be understood that there may be water and variations.

1 is a block diagram illustrating a detailed configuration of a gesture-based object-oriented interfacing device (hereinafter referred to as an 'interfacing device') 100 according to a preferred embodiment of the present invention, and FIG. 2 is an object shown in FIG. It is a block diagram showing a detailed configuration of the recognition unit 120, FIGS. 3 and 4 are flowcharts showing a processing process according to an embodiment of the present invention, and FIGS. 5 and 6 are detailed configurations of vertical objects and point objects and an event generated using the same.

As shown in FIG. 1 , the interfacing device 100 of the present invention includes an input unit 110 , an object recognition unit 120 , an execution control unit 130 , a region setting unit 140 , and a sound processing unit 150 , etc. can be configured.

In addition, as shown in FIG. 2, the object recognition unit 120, which is a component of the interfacing device 100 of the present invention, specifically includes an AI processing unit 121, a vertical object recognition unit 123, and a point object recognition unit ( 125), a tracking unit 127, and an event detection unit 129 may be included.

The gesture-based object-oriented interfacing method (hereinafter referred to as 'interfacing method of the present invention') according to the present invention may be implemented in the form of software mounted and driven in a specific terminal or device, and may include storage means, arithmetic processing means, etc. It may also be implemented in the form of hardware such as a module or an independent device designed to implement the technical idea according to the present invention using electronic devices, parts, and the like.

In addition, according to an embodiment, the interfacing method (device) of the present invention is a TV, a display device, a navigation device, an imaging medical device, an AR or VR device, a computer (monitor), a screen system, a kiosk ( Of course, it can also be implemented in the form of being embedded in facilities or devices such as kiosks) and automation systems.

In this regard, each component of the interfacing device 100 of the present invention shown in FIG. 1 and the object recognition unit 120 shown in FIG. 2 is to be understood as a logically distinct component rather than a physically separate component. do.

That is, since each component shown in the drawing corresponds to a logical configuration for effectively explaining the technical idea according to the present invention, the function performed by the logical configuration of the present invention is realized even if each component is integrated or separated. If possible, it should be construed to be within the scope of the present invention, and if it is a component that performs the same or similar function, it should be construed as being within the scope of the present invention regardless of whether the names are identical.

The interfacing apparatus 100 or method of the present invention analyzes image data input from the camera apparatus 50 and applies an algorithm such as a matching algorithm or deep learning to recognize and detect a specific object in the image data, and detect It corresponds to an apparatus (method) for controlling a specific command mapped to a shape feature, a movement pattern, a movement characteristic, etc. of the object to be executed in the device 70 .

When image data is input from the camera device 50 through the input unit 110 (see S300, S400, FIG. 3, etc.), the object recognition unit 120 of the present invention parses the input image data to recognize the object. perform (S310).

Specifically, the vertical object recognition unit 123, which is a component of the object recognition unit 120, recognizes or detects a first vertical object, which is an object having a shape extending more than a first reference length in the vertical direction in the image data. do (S311).

According to the embodiment, as illustrated in FIG. 3 , the vertical object recognition unit 123 may be configured to interwork with the AI processing unit 121 so that reference data or learning modeling data can be effectively utilized.

As a method of recognizing an object, various methods including deep learning may be applied, and a technique using a skeleton or joint points, etc. may be used for the shape, shape, and movement of the object.

The first vertical object is an object for determining the criteria for object recognition and gesture recognition as will be described later, and the left area (LA, LA, 5) and the right area (RA, see FIG. 5) can be clearly distinguished so that the relative positional relationship of other objects can be precisely specified, as well as the movement of the object based on the relative positional relationship of the objects is more distinct and can be clearly specified.

Of course, if it is a human object having a shape extending in the vertical longitudinal direction, various objects such as a human body and an arm may be applied as the first vertical object (VO1, see FIG. 5, etc.).

However, in order to organically reflect the correlation between absolute movement and relative movement between individual objects and further enhance the user's intuitive recognition and UX-based convenience, the first vertical object VO1 is the index object spread among the user's fingers. (unfolded index finger object) is preferably set.

In this regard, of course, the first reference length may be variably set, and may be determined through a functional relationship such as the average physical length of the index finger, the distance between the image and the user. In addition, according to an embodiment, when a hand object is recognized as a skeleton or the like, it goes without saying that the first vertical object VO1 corresponding to the index finger may be detected using the shape features of the hand.

Although a detailed description is omitted, the recognition or detection of the first vertical object VO1 as in general object recognition technology means that the corresponding location information, area data, or vector data is generated (S313), and the generated data is tracked. ) means that it can be The tracking unit 127 shown in FIG. 2 means that this function can be implemented in the object recognition unit 120 of the present invention.

As such, when the first vertical object VO1 is recognized, the point object recognition unit 125 of the present invention performs a first vertical line (first vertical region) corresponding to the first vertical object VO1 (VL1, see FIG. 5 ). ), the point object PO located on either the left or the right side is recognized or detected (S315).

Of course, the first vertical line may be composed of data representing the line, but may also be composed of data representing the area occupied by the first vertical object VO1 depending on the embodiment. Accordingly, the first vertical line may be set as a left, right, or center line among the regions of the first vertical object VO1 . The second vertical line VL2 by the second vertical object VO2, which will be described later, is also the same.

As described above, when the first vertical object VO1 is set with the index finger, the point object PO is preferably set as the thumb or the tip (point or end region) of the thumb.

As such, when the first vertical object VO1 and the point object PO are recognized, the event detection unit 129 of the present invention detects that the point object PO moves to the opposite side with respect to the first vertical line VL1. It is monitored whether a moving moving or an action (hereinafter referred to as a 'first event') is made (S317).

That is, in the case of the present invention, whether or not the point object PO is moved is determined by determining the vertical region or line formed by the first vertical object VO1 that belongs to the same human object (hand) and corresponds to a different individual object. Since it is determined based on the reference, whether the movement (LA→RA, see FIG. 5) can be recognized more accurately and clearly.

When the first event is detected as described above, the execution control unit 130 of the present invention controls a predetermined command mapped to the first event to be executed in the device 70 (S320). It goes without saying that the command mapped to the first event may be set in various ways according to the type of device, the type and characteristics of content displayed on the device, user registration, and the like.

Of course, the above-described processing of the present invention may be configured to be applied cyclically as long as termination conditions such as device deactivation, termination command, etc. are not satisfied (S330 and S460).

For the implementation of a more preferred embodiment, the vertical object recognition unit 123 of the present invention may further recognize the second vertical object VO2 (S410).

The second vertical object VO2 is an object (refer to FIG. 5, etc.) located in an area where the point object PO is not located among the left area or the right area of the first vertical line VL1 (see FIG. 5, etc.) and is extended in the vertical direction. It corresponds to an object having a shape but having a length smaller than the first reference length (the length in the vertical direction of the first vertical object VO1).

That is, according to this embodiment, the point object PO and the second vertical object VO2 are located in opposite areas with respect to the first vertical object VO1 (default).

As described above, when the second vertical object VO2 is recognized, it goes without saying that various information and data corresponding to the second vertical object VO2 may be generated and tracked (S420).

In the embodiment described above, a gesture in which the location information of the point object PO is changed from one side to the other side based on the area of the first vertical line VL1 by the first vertical object VO1 becomes the first event.

In order to more clearly and clearly configure the change in the positional relationship, in this embodiment, the point object PO crosses the first vertical line VL1 and the first condition and the point object PO in which the position is changed from one side to the other side ) is configured such that the first event occurs when all the second conditions corresponding to the second vertical line VL2 formed by the second vertical object VO2 are satisfied (S430).

In other words, in this embodiment, the location information of the point object PO, etc. is located within the area of the second vertical line VL2 or corresponds to the outer line of the second vertical line VL2. becomes the first event of

As described above, when the first vertical object VO1 is an open index object and the point object PO is composed of a thumb object, the second vertical object VO2 is composed of a folded middle finger. can be

In this configuration, since the posture or shape of the hand taken by the user when using a mouse, which is the input means most familiar to the user, is similar to the user, the ease of use by intuitive recognition can be further improved.

Therefore, looking at this embodiment from the user's point of view, the gesture of serving by moving the thumb PO across the region where the index finger VO1 is located to the middle finger located on the opposite side of the thumb PO (based on the default position) is the first becomes an event. Since such a gesture corresponds to an action of clicking a mouse button, an interfacing environment conforming to UX can be implemented.

More specifically, when the position of the point object PO corresponds to the second vertical line VL2 ( S430 ), the event detection unit 129 of the present invention determines that the position of the point object PO corresponds to the second vertical line VL2 . It may be configured to detect which of the upper point or lower point of the has moved (S440).

In this way, the event (first sub-event) in which the point object PO crosses the first vertical object VO1 and moves to the upper point P1 of the second vertical object VO2 by the event detection unit 129 (Fig. 6) or an event (second sub-event) moving to the lower point P2 of the second vertical object VO2 (the right diagram of FIG. 6) is detected, the execution control unit 130 of the present invention A corresponding command mapped to the first or second sub-event is controlled to be executed in the device 70 (S450).

According to the embodiment, when the position to which the point object PO is moved is not clearly distinguished as one of the upper or lower points of the second vertical object VO2, guide information for inducing the user to adjust the position is displayed as an expression means (visual medium). Or it may be configured to be expressed through an auditory medium (S445).

Hereinafter, an embodiment of the present invention in which a reference recognition area for object recognition is set will be described in detail with reference to FIG. 7 and the like.

As shown in FIG. 1 , the area setting unit 140 of the present invention may specifically include a distance calculating unit 141 , an estimation calculating unit 142 , and a recognition area setting unit 143 . Of course, as described above, these components also correspond to logical configurations centered on their functions.

When image data is input from the camera device 50 (S600), the distance calculator 141 selects and selects an object corresponding to an eye from the image data by applying algorithms such as feature point extraction, vector operation, and deep learning. The first distance information between the eye objects is calculated ( S610 ).

As such, when the first distance information is calculated using the pixel position of the image data, the estimation operation unit 142 of the present invention uses the ratio information between the actual physical average distance information between the eyes and the first distance information to the display device. The distance information between the (device 70) and the user is estimated and calculated (S620).

When the separation distance information is estimated, the recognition area setting unit 143 of the present invention performs the separation distance information, the size information of the display device 70, the size information of the portable object in the image data, or the human object (hand) considered in terms of ergonomics. The reference recognition area is set by comprehensively considering the movement range information of the object) (S630).

This reference recognition area is an area set in consideration of the physical distance between the current device (display apparatus) 70 and the user, the size of the device (display apparatus) 70, etc., and corresponds to a virtual area where the user's gesture is highly likely. do.

When the reference recognition area is set in this way, the object recognition unit 120 of the present invention does not parse the entire image data for each frame input from the camera device 50, but only a specific area corresponding to the reference recognition area. Alternatively, object recognition processing may be performed with a priority on this specific area (S670), thereby improving the efficiency of computational processing.

More preferably, the recognition area setting unit 143 may be configured to store and utilize information on the position of the face at the point in time when the reference recognition area is set (S640).

If the information on the face position is stored in this way, it monitors whether or not the face position information moves in the future (S650), and when the position of the face moves, the reference recognition area is moved to a position determined as a function thereof (S660). As the reference recognition area can be naturally changed according to the movement of

When the face recognition algorithm is implemented in the present invention, it may be configured such that the above-described method is applied to a registered user first.

8 is a flowchart illustrating a processing process according to an embodiment of the present invention utilizing sound direction information and the like.

The size of the image data for each frame varies depending on the hardware resource, and the proportion of human objects included in the image data according to the physical distance between the camera device 50 (including the form installed on the device 70, etc.) and the user this will be different

Therefore, when the object recognition algorithm is applied to the entire image data, the efficiency of calculation processing is low, and the size of the human object detected in the image data is relatively small, so the accuracy and precision of object detection may be deteriorated.

In consideration of this point, image data may be divided into n (n is a natural number greater than or equal to 2) regions, and object detection or recognition processing may be applied to the n-divided individual unit images.

For example, if the image data for each frame is divided into four unit regions or unit images, object detection or recognition processing is applied to the fourth unit image from the first unit image by a sequential method, etc., if the #3 If an intended object is detected in the unit image, the detection processing for the #1 unit image and the #2 unit image is processing that reduces the efficiency of the calculation processing.

The embodiment of the present invention shown in FIG. 8 is an embodiment for effectively overcoming this problem, and object recognition processing is prioritized for a specific unit image among a plurality of unit images by utilizing the directionality information of the sound acquired by the microphone module. Corresponds to an embodiment in which

Specifically, when sound information is input from the microphone module 60 (S710), the sound processing unit 150 of the present invention determines whether the input sound information corresponds to a preset wake-up sound (S720). ), the camera device 50 is controlled to be activated (wake mode) in case of a start sound (S730).

The startup sound may be registered in advance by a registered user, etc., and may be a voice-based sound composed of specific words and phrases, or may be a sound other than voice, such as a predetermined number of applause.

In this configuration, the camera device 50 can be normally maintained in an inactive mode (sleep mode, sleep mode) (S700), so that the efficiency of power use can also be increased.

According to the embodiment, when the microphone module 60 is a directional microphone or is composed of a plurality of microphone units provided at different positions, it is possible to generate directional information on the position or the position where the starting sound is generated (S740). ).

Of course, the direction information may be generated by the microphone module 60 itself or by parsing the sound information in the sound processing unit 150 of the present invention, depending on the embodiment.

As such, when the direction information, which is information about the location or direction of the starting sound, is generated, the object recognition unit 120 of the present invention corresponds to the direction information among n unit images that are images in which the image data is divided into n pieces. It may be configured to select a unit image (S750) and perform processing for object recognition with the unit image as a priority (S760).

In this configuration, as described above, the amount of operations processed per unit time can be drastically reduced, so that the instantaneous responsiveness of object recognition and interfacing based thereon can be more effectively implemented.

9 is a view for explaining another embodiment of the present invention in which a reference recognition area for object recognition is set.

This embodiment corresponds to an embodiment in which an area conveniently determined by a user based on a user's intention is set as a reference recognition area, which is an area to which a gesture for object recognition and interfacing control is input.

When the image data is input from the camera device 50, the recognition area setting unit 143 of the present invention is located at a position where the hand object H including the right-angled shape B in the input image data is diagonally symmetrical. Monitor whether it is recognized.

When the hand object having the above-described shape is recognized, an area formed by the extension line of the right-angled shape B is generated and the generated area is set as the reference recognition area A. As shown in FIG.

When the reference recognition area A is set in this way, as in the embodiment described with reference to FIG. 7 , the object recognition unit 120 of the present invention parses the entire image data for each frame input from the camera device 50 . It may be configured to target only a specific area corresponding to the reference recognition area without a target, or to perform object recognition processing with this specific area as a priority target.

Furthermore, in this embodiment, whether the stored face location information is moved or not, and if the location of the face is moved, the reference recognition area may be configured to naturally move to a location determined functionally.

In the above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited thereto and will be described below with the technical idea of the present invention by those of ordinary skill in the art to which the present invention pertains. It goes without saying that various modifications and variations are possible within the scope of equivalents of the claims.

In the above description of the present invention, modifiers such as first and second are only instrumental terms used to relatively distinguish between components, so they are used to indicate a specific order, priority, etc. It should not be construed as a term that

The accompanying drawings for the purpose of explaining the present invention and illustrating examples thereof may be shown in a somewhat exaggerated form in order to emphasize or highlight the technical contents of the present invention, but the above-described contents and matters shown in the drawings, etc. It should be construed as apparent that various types of modification application examples may be possible at the level of those skilled in the art in consideration of this.

The gesture-based object-oriented interfacing method of the present invention described above may be implemented as a computer-readable code on a computer-readable recording medium. Computer-readable recording media includes all types of recording devices (CD-ROM, RAM, ROM, floppy disk, magnetic disk, hard disk, magneto-optical disk, etc.) in which computer-readable data is stored, and wired and wireless Internet It also includes a server for transmission.

50: camera device 60: microphone module
70: device (display apparatus) 100: interfacing apparatus of the present invention
110: input unit 120: object recognition unit
121: AI processing unit 123: vertical object recognition unit
125: point object recognition unit 127: tracking unit
129: event detection unit 130: execution control unit
140: area setting unit 141: distance calculation unit
142: estimation calculation unit 143: recognition area setting unit
150: sound processing unit

Claims (9)

an input step of receiving image data from a camera device;
a distance calculation step of calculating first distance information between eyes in the image data;
an estimation step of calculating separation distance information between a display apparatus and a user by using a ratio between the average distance information between the actual eyes and the first distance information;
a recognition area setting step of setting a reference recognition area using the separation distance information and the size information of the display device;
an object recognition step of recognizing an object by parsing the image data, but recognizing an object by parsing image data of an area corresponding to a reference recognition area among the image data; and
and an execution control step of controlling a specific command to be executed according to an event generated by the object. According to claim 1, wherein the object recognition step,
a vertical object recognition step of recognizing a first vertical object having a shape extending over a first reference length in the vertical direction from the image data;
a point object recognition step of recognizing a point object located on one of the left and right sides with respect to a first vertical line corresponding to the first vertical object; and
An event detection step of detecting a first event in which the point object moves to the opposite side with respect to the first vertical line,
In the execution control step, when the first event is detected, the gesture-based object-oriented interfacing method, characterized in that controlling the command mapped to the first event to be executed. The method of claim 2, wherein the step of recognizing the vertical object comprises:
A second vertical object, which is an object located on the opposite side of the point object from the left or right side of the first vertical line, has a shape extending in the vertical direction, and has a length smaller than the first reference length, is further recognized do,
The first event of the event detection step is,
The gesture-based object-oriented interfacing method, characterized in that the positional information of the point object corresponds to a second vertical line formed by the second vertical object. The method of claim 3, wherein the event detection step comprises:
Detecting one of a first sub-event in which the location information of the point object corresponds to an upper point of the second vertical line or a second sub-event in which the location information of the point object corresponds to a lower point in the second vertical line, and ,
The execution control step is
The gesture-based object-oriented interfacing method, characterized in that controlling the execution of a command mapped to each of the first sub-event or the second sub-event. According to claim 4, The first vertical object,
A gesture-based object-oriented interfacing method, characterized in that the index object is an open index object among the user's fingers, the point object is a thumb object, and the second vertical object is a folded middle object object. delete an input unit for receiving image data from a camera device;
a distance calculator for calculating first distance information between eyes in the image data;
an estimation calculator configured to calculate separation distance information between a display device and a user by using a ratio between the average distance information between the eyes and the first distance information;
a recognition area setting unit for setting a reference recognition area using the separation distance information and the size information of the display device;
an object recognition unit for recognizing an object by parsing the image data, but recognizing an object by parsing image data of an area corresponding to a reference recognition area among the image data; and
and an execution control unit for controlling a specific command to be executed according to an event generated by the object. delete A computer-readable recording medium in which a program for performing the method according to any one of claims 1 to 5 is recorded.

Images