KR20230126158A - Apparatus and method for interfacing using directional information of wake-up sound - Google Patents

Abstract

A gesture-based object-oriented interfacing method of the present invention includes an input step of receiving image data from a camera device; an object recognition step of parsing the image data and recognizing an object; and an execution control step of controlling execution of a specific command according to an event generated by the object, wherein the object recognition step includes a first vertical image extending from the image data to a first reference length or more in a vertical direction. a vertical object recognition step of recognizing an object; a point object recognizing step of recognizing a point object located on one of the left and right sides of a first vertical line corresponding to the first vertical object; and an event detection step of detecting a first event in which the point object moves to the opposite side with respect to the first vertical line, wherein the execution control step performs mapping to the first event when the first event is detected. It is characterized in that the command is controlled to be executed.

Description

Interfacing method and apparatus using directional information of starting sound

The present invention relates to a gesture-based interfacing method and the like, and more specifically, interfacing between a user and an electronic device by organically combining the mutual relationship between absolute movement and relative movement of an object to be recognized. It relates to a gesture-based object-oriented interfacing method that can more clearly and elaborately implement an interaction.

In order to meet the increased needs of users and further enhance user convenience and device usability, poses, actions, gestures, specific shapes or movements of body parts (hands, arms, etc.) (hereinafter referred to as 'gestures') are used. Gesture recognition technology that recognizes image matching, artificial intelligence, deep learning, deep neural network (DNN) models, etc. and controls interfacing with electronic devices based on this is widely applied.

Since this gesture recognition technology has the basic characteristics of the non-contact method as well as the advantage of not requiring additional means for interfacing and being implemented at a remote distance, it is extensively applied without being limited to the category or type of electronic device.

Looking at the prior art, from a simple technique of detecting only the overall movement of a hand or arm and applying it to interfacing, to a fairly complex object recognition algorithm system for image data generated from a camera device to recognize delicate motions or shape features. Various embodiments have been disclosed, including techniques.

However, since the latter technologies are based on the consistency of unique and complex gestures of human objects or complex recognition algorithms are applied, as in Korean Patent Registration Publication No. 10-1785650, the clarity of gesture recognition is secured if the geometric features of the object are not precisely recognized. It is difficult to achieve, and interfacing response delay, operation stoppage, malfunction, etc. may occur frequently, and it is difficult to implement immediate responsiveness even in calculation processing.

Of course, some of these problems may be solved if expensive hardware resources (sensor, CPU, camera, etc.) with high resolution are sufficiently supported. It can cause other intrinsic problems that limit versatility and accessibility.

In addition, in the case of the prior art, since unfamiliar gestures unfamiliar to the user are mainly applied to improve resolution, etc., it is difficult for the user to intuitively adapt unless he or she continuously learns with a sense of purpose, so user convenience is low.

Korean Registered Patent Publication No. 10-1785650 (2017.09.29.) Korean Registered Patent Publication No. 10-1812605 (2017.12.20) Korean Registered Patent Publication No. 10-2320754 (2021.10.27.)

The present invention was devised to solve the above-mentioned problems in the background as described above, and optimizes user convenience by intuitive recognition based on gestures familiar to the user, and provides dynamic changes to the relative positional relationship of individual objects. Its purpose is to provide a gesture-based object-oriented interfacing method that can dramatically improve not only computational efficiency but also object recognition rate by effectively grafting .

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. In addition, the objects and advantages of the present invention can be realized by the configuration shown in the claims and the combination of the configuration.

According to an embodiment of the present invention for achieving the above object, a gesture-based object-oriented interfacing method includes an input step of receiving image data from a camera device; an object recognition step of parsing the image data and recognizing an object; and an execution control step of controlling a specific command to be executed according to an event generated by the object.

Specifically, the object recognizing step may include: a vertical object recognizing step of recognizing a first vertical object having a shape extending more than a first reference length with respect to the vertical direction in the image data; a point object recognizing step of recognizing a point object located on one of the left and right sides of a first vertical line corresponding to the first vertical object; and an event detection step of detecting a first event in which the point object moves to the opposite side with respect to the first vertical line. In this case, the execution control step of the present invention is carried out when the first event is detected. and execute a command mapped to the first event.

In addition, the vertical object recognition step of the present invention is an object located on the opposite side of the point object on the left or right side of the first vertical line, and has an extended shape based on the vertical direction, but is longer than the first reference length. It may be configured to further recognize the second vertical object having a small length.

In this case, it is preferable that the first event in the event detection step of the present invention is set when the location information of the point object corresponds to the second vertical line formed by the second vertical object.

Preferably, in the event detection step of the present invention, the location information of the point object corresponds to a first sub-event corresponding to a higher point of the second vertical line or the location information of the point object corresponds to a lower point of the second vertical line. It may be configured to detect one of the second sub-events. In this case, the execution control step of the present invention controls commands mapped to each of the first sub-event and the second sub-event to be executed.

Specifically, it is preferable that the first vertical object is a user's finger extended index object, the point object is a thumb object, and the second vertical object is a folded stop object.

According to an embodiment, the present invention provides a distance calculation step of calculating first distance information between eyes in the image data; an estimation step of calculating separation distance information between the display device and the user by using a ratio between the average distance information between actual eyes and the first distance information; and a recognition area setting step of setting a reference recognition area using the separation distance information and size information of the display device.

In this case, it is preferable that the object recognition step of the present invention is configured to use image data of an area corresponding to the reference recognition area.

In addition, the present invention includes a camera control step of controlling the camera device to be activated when the sound detected by the microphone module corresponds to a wake-up sound; And it may further include a direction information input step of generating direction information that is information on the generation location or direction of the startup sound.

In this case, the object recognition step of the present invention performs object recognition processing using n unit images in which the image data is divided into n (n is a natural number of 2 or more) regions, and among the n unit images The unit image corresponding to the directional information may be configured to perform object recognition processing first.

Furthermore, the present invention may further include a recognition area setting step of setting an area formed by the extension line of the right angle shape as a reference recognition area when a hand object including a right angle shape is recognized at a diagonally symmetrical position. In this case, the object recognition step of the present invention may be configured to use image data of an area corresponding to the reference recognition area.

More preferably, the recognition area setting step of the present invention includes a position setting step of setting face location information at the time when the reference recognition area is set; and a variable setting step of moving the location of the reference recognition region to correspond to the movement when the set face location information moves.

A gesture-based object-oriented interfacing device according to another aspect of the present invention includes an input unit for receiving image data from a camera device; an object recognition unit that recognizes an object by parsing the image data; and an execution control unit that controls execution of a specific command according to an event generated by the object.

Specifically, the object recognizing unit includes a vertical object recognizing unit recognizing a first vertical object having a shape extending more than a first reference length with respect to the vertical direction in the image data; a point object recognition unit for recognizing a point object located on either the left side or the right side of a first vertical line corresponding to the first vertical object; and an event detection unit configured to detect a first event in which the location information of the point object moves to the opposite side with respect to the first vertical line.

In this case, when the first event is detected, the execution control unit of the present invention controls a command mapped to the first event to be executed.

According to a preferred embodiment of the present invention, since a gesture having a simple displacement feature is applied, the efficiency of calculation processing can be further increased, and a change in the relative positional relationship between individual objects constituting the gesture can be accurately specified. And it can be grafted to interfacing, so the clarity and precision of interfacing or interaction between a user and an electronic device can also be implemented.

In the case of the present invention, since the relative positional relationship of individual objects that have motion characteristics that move together as a whole and can move independently is applied as a gesture, there is no need to change the positional reference for gesture recognition or detection in a time-series manner. The processing efficiency can be further improved.

In addition, according to an embodiment of the present invention, since a shape or movement corresponding to a user-familiar UX is applied as a gesture, user convenience by intuitive recognition can be further optimized.

Furthermore, according to another embodiment of the present invention, the image of the region corresponding to the user's current location or direction is converted into an object without pre-processing to select or specify the region for object recognition in advance or without applying the entire image data as it is. Through processing that is prioritized for recognition, the operation processing speed can be drastically improved, so that the immediate response of interfacing can be implemented.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and together with the detailed description of the present invention serve to more effectively understand the technical idea of the present invention, the present invention is described in these drawings should not be construed as limited to
1 is a block diagram showing the detailed configuration of an interfacing device according to a preferred embodiment of the present invention;
Figure 2 is a block diagram showing the detailed configuration of the object recognition unit shown in Figure 1;
3 and 4 are flow charts showing processing procedures for embodiments of the present invention;
5 and 6 are diagrams for explaining detailed configurations of vertical objects and point objects and events generated based thereon;
7 is a flowchart showing the processing procedure of the present invention in which a reference recognition area for object recognition is set;
8 is a flowchart showing the processing process of the present invention in which sound direction information and the like are utilized;
9 is a diagram for explaining another embodiment of the present invention in which a reference recognition area for object recognition is set.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, the terms or words used in this specification and claims should not be construed as being limited to the usual or dictionary meaning, and the inventor appropriately uses the concept of the term in order to explain his/her invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that it can be defined.

Therefore, since the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent all of the technical ideas of the present invention, various equivalents that can replace them at the time of the present application It should be understood that there may be waters and variations.

1 is a block diagram showing the detailed configuration of an interfacing device (hereinafter referred to as 'interfacing device') 100 using directional information of a starting sound according to a preferred embodiment of the present invention, and FIG. 2 is shown in FIG. 3 and 4 are flowcharts showing a processing procedure according to an embodiment of the present invention, and FIGS. 5 and 6 are for vertical objects and point objects. It is a diagram explaining the detailed configuration and events generated by using it.

As shown in FIG. 1, the interfacing device 100 of the present invention includes an input unit 110, an object recognition unit 120, an execution control unit 130, a region setting unit 140, and a sound processing unit 150. can be configured.

In addition, as shown in FIG. 2, the object recognition unit 120, which is one component of the interfacing device 100 of the present invention, specifically includes the AI processing unit 121, the vertical object recognition unit 123, the point object recognition unit ( 125), a tracking unit 127, an event detection unit 129, and the like.

The interfacing method according to the present invention (hereinafter referred to as the 'interfacing method of the present invention') may be implemented in the form of software that is loaded and driven on a specific terminal or device, and may include electronic elements such as storage means and calculation processing means, parts, etc. It can also be implemented in the form of hardware such as a module or an independent device designed to implement the technical idea according to the present invention using .

In addition, according to the embodiment, the interfacing method (device) of the present invention is a TV, display device, navigation, medical imaging device, AR or VR device, computer (monitor), screen system, kiosk ( Of course, it can also be implemented in the form of being embedded in facilities or devices such as a kiosk) and an automation system.

In this respect, each component of the interfacing device 100 of the present invention shown in FIG. 1 and the object recognition unit 120 shown in FIG. 2 should be understood as a logically separated component rather than a physically separated component. do.

That is, since each component shown in the drawings corresponds to a logical configuration for effectively explaining the technical idea according to the present invention, even if each component is integrated or separated, the function performed by the logical configuration of the present invention is realized. If possible, it should be interpreted as being within the scope of the present invention, and components performing the same or similar functions should be construed as being within the scope of the present invention regardless of whether or not their names are identical.

The interfacing device 100 or method of the present invention analyzes (parses) image data input from the camera device 50, recognizes and detects a specific object in the image data by applying an algorithm such as a matching algorithm or deep learning, and detects it. It corresponds to a device (method) for controlling a specific command mapped to a shape feature, movement pattern, or movement characteristic of an object to be executed in the device 70 .

When image data is input from the camera device 50 through the input unit 110 (see S300, S400, FIG. 3, etc.), the object recognition unit 120 of the present invention parses the input image data and performs processing for recognizing an object. It is performed (S310).

Specifically, the vertical object recognition unit 123, which is one component of the object recognition unit 120, recognizes or detects a first vertical object, which is an object having a shape extending beyond a first reference length in the vertical direction, in the image data. Do (S311).

According to the embodiment, as illustrated in FIG. 3 , the vertical object recognition unit 123 may be configured to interwork with the AI processing unit 121 so that reference data or learning modeling data can be effectively utilized.

A method of recognizing an object may be applied using various methods including deep learning, and a technique using a skeleton or joint points may be used for the shape, shape, and motion of an object.

As will be described later, the first vertical object is an object for determining the standards for object recognition and gesture recognition, and the left area (LA, 5) and the right area (RA, see FIG. 5) can be clearly distinguished, so that the relative positional relationship of other objects can be accurately specified, and the movement of objects based on the relative positional relationship between objects is more distinct and can be clearly specified.

Of course, various objects such as the body and arms of a human body can be applied as the first vertical object (VO1, see FIG. 5, etc.) if it is a human object having a shape extending in the vertical length direction.

However, in order to organically reflect the correlation between absolute movement and relative movement between individual objects, and to further enhance the user's intuitive recognition and UX-based convenience, the first vertical object (VO1) is the user's finger. (un-folded index finger object).

In this regard, the first reference length can be set variably, and can be determined through a functional relationship such as a physical average length of an index finger and a distance between an image and a user. In addition, according to the embodiment, when a hand object is recognized as a skeleton or the like, it goes without saying that the first vertical object VO1 corresponding to the index finger may be detected using the shape characteristics of the hand.

Although a detailed description is omitted, recognizing or detecting the first vertical object VO1 as in general object recognition technology means that corresponding location information, area data, or vector data are generated (S313) and the generated data is tracked. ) means that it can be The tracking unit 127 shown in FIG. 2 means that these functions can be implemented in the object recognition unit 120 of the present invention.

In this way, when the first vertical object VO1 is recognized, the point object recognition unit 125 of the present invention receives a first vertical line (first vertical area) (VL1, see FIG. 5) corresponding to the first vertical object VO1. ) is recognized or detected (S315).

Of course, the first vertical line may be composed of data representing a line, but may also be composed of data representing an area occupied by the first vertical object VO1 according to an embodiment. Accordingly, the first vertical line may be set to the left, right, or center line of the area of the first vertical object VO1. The second vertical line VL2 by the second vertical object VO2 to be described later is also the same.

As described above, when the first vertical object VO1 is set to the index finger, the point object PO is set to the thumb or the tip (point or end area) of the thumb.

In this way, when the first vertical object VO1 and the point object PO are recognized, the event detection unit 129 of the present invention moves the point object PO to the opposite side with respect to the first vertical line VL1. It is monitored whether a moving movement or action (hereinafter referred to as a 'first event') is performed (S317).

That is, in the case of the present invention, whether or not the point object PO is moved depends on the vertical area or line formed by the first vertical object VO1 belonging to the same human object (hand) but corresponding to another individual object. Since it is determined based on the criteria, whether or not the movement (LA→RA, see FIG. 5) can be recognized more accurately and clearly.

When the first event is detected in this way, the execution control unit 130 of the present invention controls a predetermined command mapped to the first event to be executed in the device 70 (S320). Of course, the command mapped to the first event can be set in various ways according to the type of device, the type and characteristics of content displayed on the device, user registration, and the like.

It goes without saying that the processing of the present invention described above may be configured to be applied cyclically unless termination conditions such as device deactivation, termination command, and the like are satisfied (S330 and S460).

In order to implement a more preferred embodiment, the vertical object recognition unit 123 of the present invention may further recognize the second vertical object VO2 (S410).

The second vertical object VO2 is an object located in an area where the point object PO is not located among the left or right areas of the first vertical line VL1 (see FIG. 5, etc.) and is extended in the vertical direction. It corresponds to an object having a shape but having a length smaller than the first reference length (the length of the first vertical object VO1 in the vertical direction).

That is, in the case of this embodiment, the point object PO and the second vertical object VO2 are located in opposite areas relative to the first vertical object VO1 (default).

As described above, when the second vertical object VO2 is recognized, various information and data corresponding to the second vertical object VO2 can be created and tracked (S420).

In the above-described embodiment, the first event is a gesture in which the location information of the point object PO is changed from one side to the other side based on the area of the first vertical line VL1 by the first vertical object VO1.

In order to make this change in positional relationship more distinct and clear, in this embodiment, the first condition that the point object PO crosses the first vertical line VL1 and its position changes from one side to the other and the point object PO ) is configured to be the first event when all the second conditions corresponding to the second vertical line VL2 formed by the second vertical object VO2 are satisfied (S430).

In other words, in terms of data processing, in this embodiment, the location information of the point object PO is located within the area of the second vertical line VL2 or corresponds to the outer line of the second vertical line VL2. becomes the first event of

As described above, when the first vertical object VO1 is an unfolded index object and the point object PO is composed of a thumb object, the second vertical object VO2 is composed of a folded middle finger. It can be.

In the case of this configuration, since the posture or shape of the user's hand when using a mouse, which is the most familiar input means to the user, is similar, the convenience of use by intuitive recognition can be further enhanced.

Therefore, when looking at this embodiment from the user's point of view, the gesture of moving the thumb PO across the area where the index finger VO1 is located to the middle finger located on the opposite side of the thumb PO (based on the default position) to treat is the first gesture. become an event Since these gestures correspond to the act of clicking a mouse button, an interfacing environment suitable for UX can be implemented.

More specifically, when the position of the point object PO corresponds to the second vertical line VL2 (S430), the event detection unit 129 of the present invention determines that the position of the point object PO corresponds to the second vertical line VL2. It may be configured to detect which of the upper or lower points of the has moved (S440).

In this way, an event (first sub-event) in which the point object PO moves to the upper point P1 of the second vertical object VO2 across the first vertical object VO1 by the event detector 129 (FIG. 6) or an event moving to the lower point P2 of the second vertical object VO2 (second sub-event) (right diagram of FIG. 6) is detected, the execution control unit 130 of the present invention The command mapped to the first or second sub-event is controlled to be executed in the device 70 (S450).

Depending on the embodiment, when the position to which the point object PO has moved is not clearly divided into either an upper or lower point of the second vertical object VO2, guidance information for inducing the user to adjust the position is an expression means (visual medium). Or, it may be configured to be expressed (S445) through an auditory medium, etc.).

Hereinafter, an embodiment of the present invention in which a reference recognition area for object recognition is set will be described in detail with reference to FIG. 7 and the like.

As shown in FIG. 1 , the area setting unit 140 of the present invention may include a distance calculation unit 141 , an estimation operation unit 142 , a recognition area setting unit 143 , and the like. As described above, of course, these components also correspond to a logical configuration centered on their functions.

When the image data is input from the camera device 50 (S600), the distance calculation unit 141 selects and selects an object corresponding to the eye from the image data by applying algorithms such as feature point extraction, vector operation, and deep learning. First distance information between the eye objects is calculated (S610).

In this way, when the first distance information is calculated using the pixel position of the image data, the estimation operation unit 142 of the present invention uses the ratio information between the physical average distance information between the eyes and the first distance information to display the device. Information on the separation distance between (the device 70) and the user is estimated and calculated (S620).

If the separation distance information is estimated, the recognition area setting unit 143 of the present invention determines the separation distance information, the size information of the display device 70, the size information of the portable object in the image data, or the human object (hand hand) considered in terms of ergonomics. The reference recognition area is set by comprehensively considering the movement range information of the object) (S630).

This reference recognition area is an area set in consideration of the physical distance between the current device (display device) 70 and the user, the size of the device (display device) 70, and the like, and corresponds to a virtual area in which the user's gesture is likely to be made. do.

When the reference recognition area is set in this way, the object recognition unit 120 of the present invention does not parse the entire image data for each frame input from the camera device 50, but only the specific area corresponding to the reference recognition area. Alternatively, object recognition processing may be performed (S670) targeting this specific area first, thereby improving the efficiency of calculation processing.

More preferably, the recognition area setting unit 143 may be configured to store (S640) information on the face position at the time when the reference recognition area is set and utilize it.

When the information on the face position is stored in this way, whether the face position information moves in the future is monitored (S650), and when the position of the face moves, the reference recognition area is moved to a position determined functionally (S660). As the reference recognition area can be naturally varied according to the movement of , it is possible to provide a user-oriented environment as well as efficiency in calculation processing.

When the face recognition algorithm is implemented in the present invention, the above-described method may be applied to registered users first.

8 is a flowchart illustrating a processing process according to an embodiment of the present invention utilizing sound direction information and the like.

The size of the image data for each frame varies according to hardware resources, and the ratio of human objects included in the image data according to the physical distance between the camera device 50 (including the type installed in the device 70) and the user. this becomes different

Therefore, when the object recognition algorithm is applied to the entire image data, not only the efficiency of operation processing is low, but also the accuracy and precision of object detection may be deteriorated because the size of a human object detected in the image data is relatively small.

Considering this point, image data may be divided into n (n is a natural number equal to or greater than 2) regions, and object detection or recognition processing may be applied to individual unit images divided into n regions.

For example, if image data for each frame is divided into 4 unit regions or unit images, object detection or recognition processing is applied to the fourth unit image from the first unit image in a sequential manner. If an intended object is detected in the unit image, the detection processing for the #1 unit image and the #2 unit image becomes processing that reduces the efficiency of calculation processing.

The embodiment of the present invention shown in FIG. 8 is an embodiment for effectively overcoming such a problem, and object recognition processing is performed by prioritizing a specific unit image among a plurality of unit images by utilizing the directional information of sound acquired by the microphone module. Corresponds to an embodiment that performs

Specifically, when sound information is input from the microphone module 60 (S710), the sound processing unit 150 of the present invention determines whether the input sound information corresponds to a preset wake-up sound (S720 ) When corresponding to the startup sound, the camera device 50 is controlled to be activated (wake mode) (S730).

The startup sound may be registered in advance by a registered user, etc., and may be a voice-based sound composed of a specific word or phrase, or may be a sound other than voice, such as a set number of claps.

In this configuration, the camera device 50 can be maintained in an inactive mode (sleep mode) during normal times (S700), so that the efficiency of power use can also be increased.

Depending on the embodiment, when the microphone module 60 is a directional microphone or is composed of a plurality of microphone units provided at different locations, it is possible to generate a location where a startup sound is generated or directional information about the corresponding location (S740). ).

Of course, the directional information may be generated in the microphone module 60 itself or generated through parsing of sound information in the sound processing unit 150 according to the embodiment.

In this way, when directional information, which is information on the location or direction of occurrence of the activation sound, is generated, the object recognition unit 120 of the present invention corresponds to the directional information among n unit images, which are images in which image data is divided into n pieces. It may be configured to select a unit image (S750) and perform processing for object recognition with the unit image as a priority (S760).

In the case of this configuration, as described above, the amount of computations processed per unit time can be drastically reduced, so that object recognition and immediate responsiveness of interfacing based thereon can be implemented more effectively.

9 is a diagram for explaining another embodiment of the present invention in which a reference recognition area for object recognition is set.

This embodiment corresponds to an embodiment in which an area conveniently determined by a user based on a user's intention is set as a reference recognition area, which is an area where a gesture for object recognition and interfacing control is input.

When image data is input from the camera device 50, the recognition area setting unit 143 of the present invention is located at a position where the hand object H including the rectangular shape B in the input image data is diagonally symmetrical. It monitors whether it is recognized.

When a hand object having the above-described shape is recognized, an area formed by the extension line of the rectangular shape (B) is created and this created area is set as the reference recognition area (A).

When the reference recognition area A is set as described above, the object recognition unit 120 of the present invention parses the entire image data for each frame input from the camera device 50 as in the embodiment described with reference to FIG. It may be configured to target only a specific region corresponding to the reference recognition region without targeting it, or to perform object recognition processing with a priority targeting this specific region.

Furthermore, it goes without saying that in this embodiment, whether or not the stored face location information is moved is monitored, and when the location of the face moves, the reference recognition area can be naturally moved to a location determined functionally thereto.

Although the present invention has been described above with limited examples and drawings, the present invention is not limited thereto and will be described below and the technical spirit of the present invention by those skilled in the art to which the present invention belongs. Of course, various modifications and variations are possible within the scope of the claims.

In the description of the present invention described above, modifiers such as first and second are only terms of instrumental concepts used to relatively distinguish components from each other, so they are used to indicate a specific order, priority, etc. It should be interpreted that it is not a term that

Although the accompanying drawings and the like for illustration of the description of the present invention and its embodiments may be shown in a slightly exaggerated form in order to emphasize or highlight the technical contents according to the present invention, the above-described contents and matters shown in the drawings Taking into account, it should be interpreted that it is obvious that various types of modifications can be applied at the level of those skilled in the art.

The above-described gesture-based object-oriented interfacing method of the present invention can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices (CD-ROM, RAM, ROM, floppy disk, magnetic disk, hard disk, magneto-optical disk, etc.) in which data that can be read by a computer is stored. It also includes a server for transmission.

50: camera device 60: microphone module
70: device (display device) 100: interfacing device of the present invention
110: input unit 120: object recognition unit
121: AI processing unit 123: vertical object recognition unit
125: point object recognition unit 127: tracking unit
129: event detection unit 130: execution control unit
140: area setting unit 141: distance calculation unit
142: estimation operation unit 143: recognition area setting unit
150: sound processing unit

Claims (4)

a camera control step of controlling a camera device to be activated when the sound detected by the microphone module corresponds to a wake-up sound;
a direction information input step in which direction information, which is information on the generation location or direction of the startup sound, is generated;
an input step of receiving image data from the camera device;
an object recognition step of parsing the image data and recognizing an object; and
An execution control step of controlling a specific command to be executed according to an event generated by the object,
The object recognition step,
Processing for object recognition is performed using n unit images in which the image data is divided into n (n is a natural number of 2 or more) regions, and among the n unit images, the unit image corresponding to the directional information is firstly targeted. Interfacing method characterized in that performing processing for object recognition with. The method of claim 1, wherein the object recognition step,
a vertical object recognizing step of recognizing a first vertical object having a shape extending beyond a first reference length in a vertical direction from the image data;
a point object recognizing step of recognizing a point object located on one of the left and right sides of a first vertical line corresponding to the first vertical object; and
An event detection step of detecting a first event in which the point object moves to the opposite side with respect to the first vertical line,
Wherein the execution control step controls a command mapped to the first event to be executed when the first event is detected. a sound processing unit for controlling a camera device to be activated when the sound detected by the microphone module corresponds to a wake-up sound, and generating directional information that is information about a location or direction of the wake-up sound;
an input unit receiving image data from the camera device;
an object recognition unit that recognizes an object by parsing the image data; and
An execution control unit for controlling execution of a specific command according to an event generated by the object,
The object recognition unit,
Processing for object recognition is performed using n unit images in which the image data is divided into n (n is a natural number of 2 or more) regions, and among the n unit images, the unit image corresponding to the directional information is firstly targeted. An interfacing device characterized in that it performs processing for object recognition with. A computer-readable recording medium on which a program for performing the method according to claim 1 or 2 is recorded.

Images