KR20230020310A - Electronic apparatus and Method for recognizing a user gesture thereof - Google Patents

Abstract

Provided are an electronic device and a method for recognizing a gesture thereof. The method for recognizing the gesture of the electronic device comprises: a step of outputting a radar signal; a step of acquiring a bit signal based on the radar signal reflected by a gesture of a user; a step of converting the bit signal into an image; a step of compressing the converted image using singularity decomposition; and a step of inputting the compressed image into a learned neural network model to identify a user gesture. Therefore, the present invention is capable of having an effect of minimizing a sensor cost.

Description

Electronic apparatus and method for recognizing a user's gesture

The present disclosure relates to an electronic device and a gesture recognition method thereof, and more particularly, to an electronic device that recognizes a user gesture based on a radar signal obtained by a user gesture and a gesture recognition method thereof.

Radar is widely applied not only for military purposes but also for civilian purposes. Recently, active safety devices for self-driving vehicles, bio-signal sensing devices, and gesture recognition are being used.

In particular, since the Doppler radar can track the movement of a target using continuous radar signals, it has a high sensitivity to a moving target and thus has attracted great attention for recognizing human gestures.

In particular, among various gesture recognition technologies, a foot motion gesture recognition technology is a useful technology capable of controlling an electronic device in an environment in which hands cannot be used. However, since the existing radar-based foot motion gesture recognition technology uses a high-resolution radar signal, there is a disadvantage in that a large amount of memory is required.

The present disclosure has been made to solve the above problems, and an object of the present disclosure is to convert a radar signal obtained by a user's gesture into an image, compress the converted image using singular value decomposition, To provide an electronic device capable of recognizing a gesture by inputting a compressed image to a neural network model and a gesture recognition method thereof.

According to an embodiment of the present disclosure, a gesture recognition method of an electronic device may include outputting a radar signal; acquiring a beat signal based on the radar signal reflected by a user's gesture; converting the bit signal into an image; compressing the transformed image using singular value decomposition; and identifying the user gesture by inputting the compressed image into a trained neural network model.

The converting may include converting the bit signal into a digital signal using an analog-to-digital converter (ADC); obtaining an STFT spectrogram from the bit signal converted to the digital signal using Short-Time Fourier Transform (STFT); and converting the obtained STFT spectrogram into the image.

Also, in the converting of the acquired STFT spectrogram into the image, the obtained STFT spectrogram may be converted into an RGB image through a color map.

The compressing may include obtaining a plurality of eigenvalues from the RGB image by using the singular value decomposition; removing eigenvalues of some order among the plurality of eigenvalues; and obtaining a compressed RGB image by restoring the RGB image based on the eigenvalues from which the part has been removed.

Also, a compression rate of the RGB image may be determined according to the order of the eigenvalues to be removed.

The user gesture may be a user's foot motion gesture, and the learned neural network model may be a model learned to identify at least one of a kick gesture, a swing gesture, a sliding gesture, and a tapping gesture.

Also, the image may be a gray image.

Meanwhile, an electronic device for recognizing a user gesture according to an embodiment of the present disclosure includes a radar outputting a radar signal and receiving a reflected radar signal; a memory for storing the learned neural network model; and acquiring a beat signal based on a radar signal reflected by a user's gesture, converting the beat signal into an image, compressing the converted image using singular value decomposition, and compressing the compressed image. and a processor for identifying the user gesture by inputting ? to the learned neural network model.

The processor converts the bit signal into a digital signal using an analog-to-digital converter (ADC), and converts the bit signal converted into a digital signal into an STFT spectrum using Short-Time Fourier Transform (STFT). A spectrogram is obtained, and the acquired STFT spectrogram may be converted into the image.

In addition, the processor may convert the acquired STFT spectrogram into an RGB image through a color map.

Further, the processor obtains a plurality of eigenvalues from the RGB image by using the singular value decomposition, removes eigenvalues of some orders from among the plurality of eigenvalues, and eigenvalues from which some of the eigenvalues have been removed. A compressed RGB image may be obtained by restoring the RGB image based on the values.

Also, a compression rate of the RGB image may be determined according to the order of the eigenvalues to be removed.

The user gesture may be a user's foot motion gesture, and the learned neural network model may be a model learned to identify at least one of a kick gesture, a swing gesture, a sliding gesture, and a tapping gesture.

Also, the image may be a gray image.

According to various embodiments of the present disclosure as described above, by compressing an image corresponding to a user gesture using SVD, memory efficiency can be increased and sensor cost can be minimized.

1 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure;
2 is a diagram for explaining a method of identifying a type of foot action gesture according to an embodiment of the present disclosure;
3 is a diagram for explaining a type of foot motion gesture according to an embodiment of the present invention;
4 is diagrams illustrating radar signals according to types of foot motion gestures according to an embodiment of the present disclosure;
5 is diagrams illustrating frequency signals according to types of foot motion gestures according to an embodiment of the present disclosure;
6 is diagrams illustrating images according to types of foot motion gestures according to an embodiment of the present disclosure;
7 is diagrams showing images compressed using SVD according to an embodiment of the present disclosure;
8A is a diagram illustrating a result of recognizing a type of foot motion gesture using an original image according to an embodiment of the present disclosure;
8B is a diagram illustrating a result of recognizing a type of foot motion gesture using a compressed image according to an embodiment of the present disclosure;
9 is a flowchart illustrating a gesture recognition method of an electronic device according to an embodiment of the present disclosure.

Since the present embodiments can apply various transformations and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to the specific embodiments, and should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, like reference numerals may be used for like elements.

In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

In addition, the following embodiments may be modified in many different forms, and the scope of the technical idea of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the spirit of the disclosure to those skilled in the art.

Terms used in this disclosure are only used to describe specific embodiments, and are not intended to limit the scope of rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present disclosure, expressions such as “has,” “can have,” “includes,” or “can include” indicate the presence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this disclosure, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as "first," "second," "first," or "second," used in the present disclosure may modify various elements regardless of order and/or importance, and may refer to one element as It is used only to distinguish it from other components and does not limit the corresponding components.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that an element may be directly connected to another element, or may be connected through another element (eg, a third element).

On the other hand, when an element (eg, a first element) is referred to as being "directly connected" or "directly connected" to another element (eg, a second element), it is referred to as a component different from a component. It may be understood that there are no other components (eg, third components) between the elements.

The expression “configured to (or configured to)” as used in this disclosure means, depending on the situation, for example, “suitable for,” “having the capacity to.” ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware.

Instead, in some contexts, the phrase "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (eg, embedded processor) to perform the operation, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

In an embodiment, a 'module' or 'unit' performs at least one function or operation, and may be implemented with hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' may be integrated into at least one module and implemented by at least one processor, except for 'modules' or 'units' that need to be implemented with specific hardware.

Meanwhile, various elements and regions in the drawings are schematically drawn. Therefore, the technical spirit of the present invention is not limited by the relative size or spacing drawn in the accompanying drawings.

Hereinafter, with reference to the accompanying drawings, an embodiment according to the present disclosure will be described in detail so that those skilled in the art can easily implement it.

Hereinafter, the present disclosure will be described in detail with reference to the drawings. 1 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 1 , the electronic device 100 may include a radar 110 , a memory 120 and a processor 130 . In this case, the electronic device 100 is a device for recognizing a user's gesture, and may be implemented as various electronic devices such as automobiles and smart glasses. In addition, the electronic device 100 may include other components such as a functional unit (for example, components related to a driving function in the case of an automobile) and a communication interface in addition to the components shown in FIG. 3 .

The radar 110 may include a transmitter for transmitting a radar signal and a receiver for receiving a radar signal reflected by an object. In particular, the radar 110 may transmit a radar signal and receive a radar signal reflected by a user gesture. In this case, the user's gesture may be a foot motion gesture, but this is merely an example, and may be other gestures (eg, hand motion gestures, etc.). In particular, when the electronic device 100 is implemented as a vehicle, the radar 110 may be located near a lower rear bumper of the vehicle or near a lower portion of a door of the vehicle.

Also, according to an embodiment of the present invention, the radar 110 may be a Doppler radar capable of measuring the direction or speed of a moving object using the Doppler effect.

The memory 120 may store at least one instruction related to the electronic device 100 . In particular, an operating system (O/S) for driving the electronic device 100 may be stored in the memory 140 . Also, various software programs or applications for operating the electronic device 100 may be stored in the memory 140 according to various embodiments of the present disclosure. Also, the memory 140 may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk. Meanwhile, in the present disclosure, the term memory 140 refers to the memory 140, a ROM (not shown) in the processor 180, a RAM (not shown), or a memory card (not shown) mounted in the electronic device 100 (eg For example, micro SD card, memory stick) may be used as a meaning including.

In particular, the memory 120 may store data capable of performing a function of recognizing a user gesture by converting a radar signal obtained by a user gesture into an image. Also, the memory 120 may store a neural network model for recognizing a user gesture based on an image corresponding to the user gesture.

Meanwhile, the memory 120 may include a non-volatile memory capable of maintaining stored information even if power supply is interrupted, and a volatile memory requiring continuous power supply to maintain stored information. Data for performing various operations by various modules for recognizing a user gesture by converting a radar signal acquired by a user gesture into an image may be stored in a non-volatile memory. In addition, a neural network model for recognizing a user gesture based on an image corresponding to the user gesture may also be stored in the non-volatile memory.

In addition, the memory 120 may include at least one buffer for temporarily storing a radar signal acquired by the radar 110, a frequency signal obtained by converting the radar signal, and an image obtained by converting the frequency signal.

The processor 130 may control overall operations and functions of the electronic device 100 . Specifically, the processor 130 is connected to the configuration of the electronic device 100 including the memory 120, and executes at least one command stored in the memory 120 as described above, so that the electronic device 100 You have full control over the action.

Processor 130 can be implemented in a variety of ways. For example, the processor 130 may include an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), a digital signal processor Processor, DSP) may be implemented as at least one. Meanwhile, in the present disclosure, the term processor 130 may be used to include a Central Processing Unit (CPU), a Graphic Processing Unit (GPU), and a Main Processing Unit (MPU).

In particular, the processor 130 acquires a beat signal based on a radar signal reflected by a user's gesture, converts the beat signal into an image, compresses the converted image using singular value decomposition, A user gesture can be identified by inputting the compressed image to the trained neural network model.

Specifically, the processor 130 may control the radar transmitter to transmit a radar signal. Also, the processor 130 may receive a radar signal reflected by a user gesture through a radar receiving unit. In this case, the user's gesture may be a user's foot motion gesture.

The processor 130 may obtain a beat signal based on the reflected radar signal. Also, the processor 130 may convert the bit signal into a digital signal using an analog-to-digital converter (ADC). Further, the processor 130 obtains a frequency signal (eg, STFT spectrogram) from the bit signal converted into a digital signal using a fast Fourier transform (FFT) (eg, short-time Fourier transform (STFT)). and convert the obtained STFT spectrogram into an image. In this case, the image may be a gray image or an RGB image.

In particular, the processor 130 may convert the STFT spectrogram obtained through the color map into an RGB image.

Then, the processor 130 obtains a plurality of eigenvalues from the RGB image by using singular value decomposition, removes eigenvalues of some order among the plurality of eigenvalues, and determines the eigenvalues from which some have been removed. A compressed RGB image may be obtained by restoring the RGB image based on the above. In this case, the compression rate of the RGB image may be determined according to the order of the eigenvalues to be removed.

The processor 130 may identify the type of user gesture by inputting the compressed image to the trained neural network model. In this case, the learned neural network model may be a model learned to identify at least one of a kick gesture, a swing gesture, a sliding gesture, and a tapping gesture. Also, the processor 130 may train a neural network model using the type of user gesture and the compressed image.

Hereinafter, the present invention will be described in more detail with reference to FIGS. 2 to 8B.

The electronic device 100 may acquire a bit signal for the user's foot motion gesture (210). Specifically, in the Doppler radar, the radar signal Tx transmitted by the transmitter is reflected from the foot gesture. The reflected radar signal Rx may be changed into a beat signal at the receiving unit. At this time, a beat signal of a single frame for the CW radar signal is obtained by mixing the transmitted radar signal and the reflected radar signal. The bit signal x(t) of a single frame is obtained by Equation 1 below:

및

는 발동작 제스처로 인한 사람의 신체 움직임 성분으로 구성된 비트 신호의 I 채널 및 Q 채널 성분이다. 또한,

과

은 각각 진폭 및 마이크로 도플러 주파수 구성 요소를 나타낸다. here,

and

is the I channel and Q channel components of a beat signal composed of human body motion components resulting from a footstep gesture. also,

class

denote the amplitude and micro-Doppler frequency components, respectively.

Meanwhile, foot gestures according to an embodiment of the present disclosure may be classified into a plurality of types. Specifically, as shown in FIG. 3, (a) a foot motion gesture includes a kick gesture in which a user kicks a foot forward, (b) a swing gesture in which a user swings a foot back and forth, and (c) a user foot gesture. It may include a sliding gesture of moving left and right, and (d) a tapping gesture of shaking the front part of the user's foot from side to side. In addition, (e) a running ball may be included in addition to the foot motion gesture.

In particular, as shown in FIG. 4 , the electronic device 100 may obtain bit signals corresponding to the plurality of types of footing gestures described in FIG. 3 .

를 가진 ADC를 이용하여 비트 신호를 디지털 신호로 변환할 수 있다. The electronic device 100 may convert the bit signal into a digital signal using an analog-to-digital converter (ADC) (215). At this time, the electronic device 100 determines the sampling rate

A bit signal can be converted into a digital signal using an ADC with

The electronic device 100 may perform time windowing on the bit signal converted into a digital signal (220).

The electronic device 100 may obtain a signal equal to or greater than a predetermined threshold for the foot motion gesture (225).

The electronic device 100 may obtain an STFT spectrogram, which is a frequency signal, by using Short-Time Fourier Transform (STFT) (230). At this time, the STFT spectrogram can be obtained by Equation 2 below.

는 윈도우 함수를 나타내고,

과

은 프레임 별 샘플 수와 FFT 포인트를 나타낸다.here,

denotes a window function,

class

represents the number of samples and FFT points per frame.

As an embodiment of the present disclosure, as shown in FIG. 5 , STFT spectrograms corresponding to a plurality of types of footing gestures may be obtained.

The electronic device 100 may convert the STFT spectrogram into an RGB image (235). Specifically, in order to construct a radar data set for training a neural network model as easily as possible, visualization of radar data is very useful. That is, in order to select valid data for various foot gestures, instead of storing bit signals (raw data), an RGB image obtained by visualizing the STFT spectrogram may be used. By this, it is possible to check the valid data corresponding to the short footing movement.

As an embodiment of the present disclosure, as shown in FIG. 6 , RGB images corresponding to a plurality of types of foot gestures may be acquired.

Meanwhile, in the above-described embodiment, it has been described that the STFT spectrogram is converted into an RGB image, but this is only an embodiment, and the STFT spectrogram may be converted into a gray image.

The electronic device 100 may map the converted RGB image through the jet color map (240). Specifically, since the STFT spectrogram satisfies the shift-invariance property to provide constant time-resolution analysis in time and frequency domains, for classification of accurate footwork gestures. An RGB image must also satisfy transition invariant characteristics in time and frequency domains. To satisfy this transition-invariant feature, the electronic device 100 may obtain an RGB image having a single array proportional to the size through the jet color map. That is, values of an RGB image expressed as a plurality of (eg, three) arrays may be mapped to a single array of numbers through color mapping. That is, the RGB image obtained by the jet color map may have a 1-dimensional value instead of a 3-dimensional value.

The electronic device 100 may perform singular value decomposition (SVD) on the acquired RGB image (245). Specifically, singular value decomposition is a method used in conjunction with PCA (principal component analysis) to reduce high-dimensional data to low-dimensional data while retaining important information. especially. Since singular value decomposition is based on a single data point, it is possible to effectively extract meaningful features for a single radar signal.

Singular value decomposition for an RGB image can be expressed as Equation 3 below.

는 고유 행열이며, 대각 행렬 연산자이다. 즉, U,V,Σ는 아래의 수학식 4와 같은 행렬 형태로 나타내진다:Here, U and V denote left and right matrices, respectively. The columns of orthogonal matrices U and V are eigenvectors and

is an eigenmatrix and is a diagonal matrix operator. That is, U, V, and Σ are represented in the form of a matrix as in Equation 4 below:

과 이에 대응하는 고유벡터

와

에 의해, RGB 이미지(I)는 아래의 수학식 5와 같이, 재배열될 수 있다. eigenvalue

and the corresponding eigenvector

and

, the RGB image I may be rearranged as shown in Equation 5 below.

는 고유값이며

순으로 내림차순을 가지며,

와

는 고유값에 대응되는 유닛 직교 열 벡터를 의미할 수 있다.here

is the eigenvalue

in descending order,

and

may mean a unit orthogonal column vector corresponding to an eigenvalue.

That is, Equation 5 may mean that the RGB image is composed of a weighted linear combination of different features.

The electronic device 100 may remove some eigenvalues by selecting the order of the eigenvalues (250). Specifically, it is important to determine the eigenvalues and their corresponding eigenvectors to contain sufficient characteristic information for each footing gesture in the RGB image I. By looking at knee points for rapidly changing footwork gestures, a threshold of order of eigenvalues can also be selected. In addition, some eigenvalues of the remaining unselected orders may be removed. In this case, the compression rate of the RGB image may be determined according to the order of the eigenvalues to be removed.

By selecting the order of the eigenvalues, unique and sufficient features for the user's footstep gesture can be extracted, and the smaller the order of the selected eigenvalues, the smaller the size of the image.

The electronic device 100 may restore the RGB image (255). That is, the electronic device 100 may restore the RGB image based on the eigenvalues including the main features and having some orders removed. At this time, the electronic device 100 may restore the RGB image by Equation 6 below.

,

,

를 의미하며, r은 수학식 5의 q에 비해 상대적으로 작을 수 있다.here

,

,

Means, r may be relatively small compared to q in Equation 5.

As described above, by removing eigenvalues of some order, the electronic device 100 may obtain a highly compressed image. As an embodiment of the present disclosure, as described above, by removing eigenvalues of some orders, as shown in FIG. 7 , 99% compressed RGB images corresponding to a plurality of types of foot gestures can be obtained. .

The electronic device 100 may input the restored RGB image to the neural network model to identify the type of footstep gesture (260). That is, the electronic device 100 may input the compressed RGB image as input data to the learned neural network model to identify the type of footstep gesture. At this time, the learned neural network model may be a deep learning model trained to identify at least one of the kick gesture, swing gesture, sliding gesture, and tapping gesture shown in FIG. 3, and the RGB image data set compressed by the method described above can be learned using

8A is a diagram illustrating a result of recognizing a type of foot motion gesture using an original RGB image according to an embodiment of the present disclosure, and FIG. It is a diagram showing the result of recognizing the type of foot motion gesture. As shown in FIGS. 8A and 8B , there is no significant difference between a result of recognizing the type of foot motion gesture using the original RGB image and a result of recognizing the type of foot motion gesture using the compressed RGB image. That is, the electronic device 100 can increase memory efficiency and minimize sensor cost by using a compressed image without showing a large difference in recognition results of foot motion gestures.

In addition, the electronic device 100 may determine a compression rate of the RGB image by comparing a result of recognizing the type of foot motion gesture using the original RGB image and a result of recognizing the type of foot motion gesture using the compressed RGB image. That is, in the electronic device 100, the difference between the result of recognizing the type of foot motion gesture using the original RGB image and the result of recognizing the type of foot motion gesture using the compressed RGB image is less than or equal to a threshold value, and the smallest capacity is obtained. It is possible to determine the compression rate of the RGB image to have

9 is a flowchart illustrating a gesture recognition method of an electronic device according to an embodiment of the present disclosure.

The electronic device 100 may output a radar signal (S910).

The electronic device 100 may obtain a beat signal based on the radar signal reflected by the user's gesture (S920). In this case, the user's gesture may be a user's foot motion gesture.

The electronic device 100 may convert the bit signal into an image (S930). Specifically, the electronic device 100 converts a bit signal into a digital signal using an analog-to-digital converter (ADC), and converts the bit signal converted into a digital signal using a short-time Fourier transform (STFT) to the STFT. A spectrogram may be acquired, and the obtained STFT spectrogram may be converted into an image. In this case, the electronic device 100 may convert the STFT spectrogram obtained through the color map into an RGB image. Meanwhile, in the above-described embodiment, it has been described that the image is an RGB image, but this is merely an embodiment and may be a gray image.

The electronic device 100 may compress the transformed image using singular value decomposition (S940). Specifically, the electronic device 100 obtains a plurality of eigenvalues from the RGB image by using singular value decomposition, removes eigenvalues of some orders from among the plurality of eigenvalues, and removes some of the eigenvalues. A compressed RGB image may be obtained by restoring the RGB image based on the . In this case, the compression rate of the RGB image may be determined according to the order of the eigenvalues to be removed.

The electronic device 100 may identify the user's gesture by inputting the compressed image to the learned neural network model (S950). In this case, the learned neural network model may be a model learned to identify at least one of a kick gesture, a swing gesture, a sliding gesture, and a tapping gesture, and may be a model learned using a compressed image data set.

Meanwhile, functions related to the neural network model as described above may be performed through a memory and a processor. A processor may consist of one or a plurality of processors. At this time, one or a plurality of processors are CPUs, general-purpose processors such as APs, GPUs. It may be a graphics-only processor, such as a VPU, or an artificial intelligence-only processor, such as an NPU. One or more processors control the input data to be processed according to predefined operating rules or artificial intelligence models stored in the non-volatile memory and the volatile memory. A predefined action rule or artificial intelligence model is characterized in that it is created through learning.

Here, being created through learning means that a predefined operation rule or an artificial intelligence model having desired characteristics is created by applying a learning algorithm to a plurality of learning data. Such learning may be performed in the device itself in which artificial intelligence according to the present disclosure is performed, or may be performed through a separate server/system.

An artificial intelligence model may be composed of a plurality of neural network layers. Each layer has a plurality of weight values, and the layer operation is performed through the operation result of the previous layer and the plurality of weight values. Examples of neural networks include Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), and GAN. (Generative Adversarial Networks) and deep Q-networks (Deep Q-Networks), and the neural network in the present disclosure is not limited to the above-described examples except for the cases specified.

A learning algorithm is a method of training a predetermined target device (eg, a robot) using a plurality of learning data so that the predetermined target device can make a decision or make a prediction by itself. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, and the learning algorithm in the present disclosure is specified Except for, it is not limited to the above example.

The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary storage medium' only means that it is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term refers to the case where data is stored semi-permanently in the storage medium and temporary It does not discriminate if it is saved as . For example, a 'non-temporary storage medium' may include a buffer in which data is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play Store™) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smartphones. In the case of online distribution, at least a part of a computer program product (eg, a downloadable app) is stored on a device-readable storage medium such as a memory of a manufacturer's server, an application store server, or a relay server. It can be temporarily stored or created temporarily.

Each of the components (eg, modules or programs) according to various embodiments of the present disclosure as described above may be composed of a single object or a plurality of entities, and some of the sub-components described above are omitted. or other sub-elements may be further included in various embodiments. Alternatively or additionally, some components (eg, modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration.

According to various embodiments, operations performed by modules, programs, or other components may be executed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be executed in a different order, may be omitted, or other operations may be added. can

On the other hand, the term "unit" or "module" used in the present disclosure includes units composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, parts, or circuits, for example. can A “unit” or “module” may be an integrated component or a minimum unit or part thereof that performs one or more functions. For example, the module may be composed of an application-specific integrated circuit (ASIC).

Various embodiments of the present disclosure may be implemented as software including commands stored in a storage medium readable by a machine (eg, a computer). The device calls the stored commands from the storage medium. And, as a device capable of operating according to the called command, it may include an electronic device (eg, the electronic device 100) according to the disclosed embodiments.

When the command is executed by a processor, the processor may directly or use other elements under the control of the processor to perform a function corresponding to the command. An instruction may include code generated or executed by a compiler or interpreter.

Although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and is common in the art to which the disclosure belongs without departing from the gist of the present disclosure claimed in the claims. Of course, various modifications are possible by those with knowledge of, and these modifications should not be individually understood from the technical spirit or perspective of the present disclosure.

110: radar 120: memory
130: processor

Claims (14)

In the gesture recognition method of an electronic device,
outputting a radar signal;
acquiring a beat signal based on the radar signal reflected by a user's gesture;
converting the bit signal into an image;
compressing the transformed image using singular value decomposition; and
and identifying the user gesture by inputting the compressed image into a trained neural network model. According to claim 1,
The conversion step is
converting the bit signal into a digital signal using an analog-to-digital converter (ADC);
obtaining an STFT spectrogram from the bit signal converted to the digital signal using Short-Time Fourier Transform (STFT);
Gesture recognition method comprising: converting the acquired STFT spectrogram into the image. According to claim 2,
Converting the acquired STFT spectrogram into the image,
Gesture recognition method characterized by converting the obtained STFT spectrogram into an RGB image through a color map. According to claim 3,
The compression step is
obtaining a plurality of eigenvalues from the RGB image using the singular value decomposition;
removing eigenvalues of some order among the plurality of eigenvalues; and
and obtaining a compressed RGB image by restoring an RGB image based on the eigenvalues from which the part has been removed. According to claim 4,
The gesture recognition method of determining a compression rate of the RGB image according to the order of the removed eigenvalue. According to claim 1,
The user gesture is a user's foot motion gesture,
The trained neural network model,
A gesture recognition method that is a model trained to identify at least one of a kick gesture, a swing gesture, a sliding gesture, and a tapping gesture. According to claim 1,
The gesture recognition method, characterized in that the image is a gray image. An electronic device for recognizing a gesture,
a radar outputting a radar signal and receiving a reflected radar signal;
a memory for storing the learned neural network model; and
Obtaining a beat signal based on a radar signal reflected by a user's gesture;
converting the bit signal into an image;
Compressing the transformed image using singular value decomposition;
and a processor inputting the compressed image to the learned neural network model to identify the user gesture. According to claim 8,
the processor,
Converting the bit signal into a digital signal using an analog-to-digital converter (ADC);
Obtaining an STFT spectrogram from the bit signal converted to the digital signal using Short-Time Fourier Transform (STFT);
An electronic device that converts the acquired STFT spectrogram into the image. According to claim 9,
the processor,
An electronic device characterized in that for converting the acquired STFT spectrogram into an RGB image through a color map. According to claim 10,
the processor,
Obtaining a plurality of eigenvalues from the RGB image using the singular value decomposition,
Remove some order eigenvalues from among the plurality of eigenvalues;
An electronic device for obtaining a compressed RGB image by restoring an RGB image based on the eigenvalues from which the part has been removed. According to claim 11,
An electronic device in which a compression rate of the RGB image is determined according to the order of the eigenvalues to be removed. According to claim 8,
The user gesture is a user's foot motion gesture,
The trained neural network model,
An electronic device that is a model trained to identify at least one of a kick gesture, a swing gesture, a sliding gesture, and a tapping gesture. According to claim 8,
The electronic device, characterized in that the image is a gray image.

Images