WO2023096134A1 - Procédé d'apprentissage pour améliorer les performances de reconnaissance de geste dans un dispositif électronique - Google Patents

Procédé d'apprentissage pour améliorer les performances de reconnaissance de geste dans un dispositif électronique Download PDF

Info

Publication number
WO2023096134A1
WO2023096134A1 PCT/KR2022/014556 KR2022014556W WO2023096134A1 WO 2023096134 A1 WO2023096134 A1 WO 2023096134A1 KR 2022014556 W KR2022014556 W KR 2022014556W WO 2023096134 A1 WO2023096134 A1 WO 2023096134A1
Authority
WO
WIPO (PCT)
Prior art keywords
classifier
electronic device
target gesture
sample
update
Prior art date
Application number
PCT/KR2022/014556
Other languages
English (en)
Korean (ko)
Inventor
정원석
조원준
김태윤
박성진
김철오
안진엽
유병욱
임채만
조용상
Original Assignee
삼성전자주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020210188584A external-priority patent/KR20230077575A/ko
Application filed by 삼성전자주식회사 filed Critical 삼성전자주식회사
Publication of WO2023096134A1 publication Critical patent/WO2023096134A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G04—HOROLOGY
    • G04G—ELECTRONIC TIME-PIECES
    • G04G21/00—Input or output devices integrated in time-pieces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Definitions

  • Various embodiments of the present disclosure relate to a learning method for improving gesture recognition performance in an electronic device.
  • Gesture recognition technology among user interface technologies is largely a technology that recognizes a gesture through an image by using an image sensor, and a sensor other than an image sensor (for example, an acceleration sensor or a gyro sensor, an inertial measurement unit, It can be classified as a technology for recognizing gestures using IMU)).
  • a technology for recognizing a gesture using a sensor other than an image sensor can be used anytime, anywhere, and thus has a higher degree of freedom than a technology for recognizing a gesture using an image sensor.
  • an electronic device may collect at least one gesture sample from a user, and use the collected at least one gesture sample to update a classifier to be suitable for the user.
  • the electronic device may update the classifier to be suitable for the user by inputting the gesture samples collected from the user to the previously generated classifier, identifying leaves corresponding to the gesture sample, and increasing a reward value of the identified leaves.
  • An electronic device includes a memory in which computer-executable instructions are stored, and a processor accessing the memory to execute the instructions, wherein the processor comprises a decision tree network
  • One target gesture sample is input to a (decision tree network) based classifier, first leaves selected for each tree are identified corresponding to the one target gesture sample, and a reward value of each of the identified first leaves is updated. Updating of the classifier may be performed by incrementing by a unit value.
  • a method performed by an electronic device includes an operation of inputting one target gesture sample to a classifier based on a decision tree network, and a first step selected for each tree corresponding to the one target gesture sample. It may include an operation of identifying leaves, and an operation of performing an upward update of the classifier by increasing a compensation value of each of the identified first leaves by an update unit value.
  • FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.
  • FIGS. 2A and 2B are perspective views of an electronic device according to an exemplary embodiment.
  • FIG 3 is an exploded perspective view of an electronic device according to an exemplary embodiment.
  • FIG. 4 is a diagram illustrating a process of performing gesture recognition in an electronic device.
  • FIG. 5 is a diagram illustrating a process of forming a network capable of classifying gestures through training in a classifier.
  • FIG. 6 is a flowchart schematically illustrating an operation of updating a classifier by an electronic device according to an exemplary embodiment.
  • FIG. 7 is a diagram illustrating a process of updating a classifier by an electronic device according to an exemplary embodiment.
  • FIG. 8 is a diagram illustrating the structure of a classifier based on a decision tree network.
  • FIG. 9 is a diagram illustrating a process of updating a classifier by inputting a data sample to a classifier based on a decision tree network by an electronic device according to an embodiment.
  • FIG. 10 is a diagram illustrating an overall process of updating a classifier by an electronic device according to an exemplary embodiment.
  • 11 is a table illustratively illustrating performance evaluation results for each updated classifier when the electronic device updates the classifier for each user according to an embodiment.
  • Figure 13 is and It is a graph showing the relationship with
  • FIG. 14 is a diagram illustrating a process of collecting target gesture samples by an electronic device in a passive method according to an embodiment.
  • 15 is a diagram illustrating a process of collecting target gesture samples by an active method by an electronic device according to an exemplary embodiment.
  • 16 is a flowchart illustrating a process of manually collecting target gesture samples and updating a classifier by an electronic device according to an embodiment.
  • 17 is a diagram for explaining a process of adjusting a threshold probability, which is a standard for collecting target gesture samples, by an electronic device according to an embodiment.
  • 18 is a flowchart illustrating a process in which an electronic device collects target gesture samples using an active method and updates a classifier according to an exemplary embodiment.
  • 19 is a flowchart illustrating a process of actively collecting target gesture samples by an electronic device according to an embodiment.
  • 20 is a diagram for explaining a process of collecting target gesture samples in an active method by an electronic device according to an embodiment.
  • 21 is a flowchart illustrating an operation after an electronic device collects a target gesture sample, according to an exemplary embodiment.
  • FIG. 22 is a diagram illustrating that an electronic device displays a performance evaluation result of a classifier on a screen according to an exemplary embodiment.
  • FIG. 1 is a block diagram of an electronic device 101 within a network environment 100, according to various embodiments.
  • an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or through a second network 199. It is possible to communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to one embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 .
  • a first network 198 eg, a short-range wireless communication network
  • the server 108 e.g, a long-distance wireless communication network
  • the electronic device 101 includes a processor 120, a memory 130, an input module 150, an audio output module 155, a display module 160, an audio module 170, a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or the antenna module 197 may be included.
  • at least one of these components eg, the connection terminal 178) may be omitted or one or more other components may be added.
  • some of these components eg, sensor module 176, camera module 180, or antenna module 197) are integrated into a single component (eg, display module 160). It can be.
  • the processor 120 for example, executes software (eg, the program 140) to cause at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or calculations. According to one embodiment, as at least part of data processing or operation, the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 .
  • software eg, the program 140
  • the processor 120 transfers instructions or data received from other components (e.g., sensor module 176 or communication module 190) to volatile memory 132. , processing commands or data stored in the volatile memory 132 , and storing resultant data in the non-volatile memory 134 .
  • the processor 120 may include a main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor).
  • a main processor 121 eg, a central processing unit or an application processor
  • a secondary processor 123 eg, a graphic processing unit, a neural network processing unit ( NPU: neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor.
  • NPU neural network processing unit
  • the secondary processor 123 may be implemented separately from or as part of the main processor 121 .
  • the secondary processor 123 may, for example, take the place of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, running an application). ) state, together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states.
  • the auxiliary processor 123 eg, image signal processor or communication processor
  • the auxiliary processor 123 may include a hardware structure specialized for processing an artificial intelligence model.
  • AI models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself where artificial intelligence is performed, or may be performed through a separate server (eg, the server 108).
  • the learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning or reinforcement learning, but in the above example Not limited.
  • the artificial intelligence model may include a plurality of artificial neural network layers.
  • Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the foregoing, but is not limited to the foregoing examples.
  • the artificial intelligence model may include, in addition or alternatively, software structures in addition to hardware structures.
  • the memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176) of the electronic device 101 .
  • the data may include, for example, input data or output data for software (eg, program 140) and commands related thereto.
  • the memory 130 may include volatile memory 132 or non-volatile memory 134 .
  • the program 140 may be stored as software in the memory 130 and may include, for example, an operating system 142 , middleware 144 , or an application 146 .
  • the input module 150 may receive a command or data to be used by a component (eg, the processor 120) of the electronic device 101 from the outside of the electronic device 101 (eg, a user).
  • the input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).
  • the sound output module 155 may output sound signals to the outside of the electronic device 101 .
  • the sound output module 155 may include, for example, a speaker or a receiver.
  • the speaker can be used for general purposes such as multimedia playback or recording playback.
  • a receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from the speaker or as part of it.
  • the display module 160 may visually provide information to the outside of the electronic device 101 (eg, a user).
  • the display module 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device.
  • the display module 160 may include a touch sensor set to detect a touch or a pressure sensor set to measure the intensity of force generated by the touch.
  • the audio module 170 may convert sound into an electrical signal or vice versa. According to one embodiment, the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).
  • the audio module 170 acquires sound through the input module 150, the sound output module 155, or an external electronic device connected directly or wirelessly to the electronic device 101 (eg: Sound may be output through the electronic device 102 (eg, a speaker or a headphone).
  • the sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state. can do.
  • the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a bio sensor, It may include a temperature sensor, humidity sensor, or light sensor.
  • the interface 177 may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device 101 to an external electronic device (eg, the electronic device 102).
  • the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
  • HDMI high definition multimedia interface
  • USB universal serial bus
  • SD card interface Secure Digital Card interface
  • audio interface audio interface
  • connection terminal 178 may include a connector through which the electronic device 101 may be physically connected to an external electronic device (eg, the electronic device 102).
  • the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).
  • the haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or motion) or electrical stimuli that a user may perceive through tactile or kinesthetic senses.
  • the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
  • the camera module 180 may capture still images and moving images. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.
  • the power management module 188 may manage power supplied to the electronic device 101 .
  • the power management module 188 may be implemented as at least part of a power management integrated circuit (PMIC), for example.
  • PMIC power management integrated circuit
  • the battery 189 may supply power to at least one component of the electronic device 101 .
  • the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.
  • the communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). Establishment and communication through the established communication channel may be supported.
  • the communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication.
  • the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : a local area network (LAN) communication module or a power line communication module).
  • a wireless communication module 192 eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module
  • GNSS global navigation satellite system
  • wired communication module 194 eg, : a local area network (LAN) communication module or a power line communication module.
  • a corresponding communication module is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a telecommunications network such as a computer network (eg, a LAN or a WAN).
  • a telecommunications network such as a computer network (eg, a LAN or a WAN).
  • These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips).
  • the wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199.
  • subscriber information eg, International Mobile Subscriber Identifier (IMSI)
  • IMSI International Mobile Subscriber Identifier
  • the electronic device 101 may be identified or authenticated.
  • the wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, NR access technology (new radio access technology).
  • NR access technologies include high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and access of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low latency (URLLC)).
  • eMBB enhanced mobile broadband
  • mMTC massive machine type communications
  • URLLC ultra-reliable and low latency
  • -latency communications can be supported.
  • the wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example.
  • the wireless communication module 192 uses various technologies for securing performance in a high frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), and full-dimensional multiplexing. Technologies such as input/output (FD-MIMO: full dimensional MIMO), array antenna, analog beam-forming, or large scale antenna may be supported.
  • the wireless communication module 192 may support various requirements defined for the electronic device 101, an external electronic device (eg, the electronic device 104), or a network system (eg, the second network 199).
  • the wireless communication module 192 is a peak data rate for eMBB realization (eg, 20 Gbps or more), a loss coverage for mMTC realization (eg, 164 dB or less), or a U-plane latency for URLLC realization (eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less) may be supported.
  • eMBB peak data rate for eMBB realization
  • a loss coverage for mMTC realization eg, 164 dB or less
  • U-plane latency for URLLC realization eg, Example: downlink (DL) and uplink (UL) each of 0.5 ms or less, or round trip 1 ms or less
  • the antenna module 197 may transmit or receive signals or power to the outside (eg, an external electronic device).
  • the antenna module 197 may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (eg, PCB).
  • the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is selected from the plurality of antennas by the communication module 190, for example. can be chosen A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna.
  • other components eg, a radio frequency integrated circuit (RFIC) may be additionally formed as a part of the antenna module 197 in addition to the radiator.
  • RFIC radio frequency integrated circuit
  • the antenna module 197 may form a mmWave antenna module.
  • the mmWave antenna module includes a printed circuit board, an RFIC disposed on or adjacent to a first surface (eg, a lower surface) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, array antennas) disposed on or adjacent to a second surface (eg, a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.
  • peripheral devices eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)
  • signal e.g. commands or data
  • commands or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 .
  • Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 .
  • all or part of operations executed in the electronic device 101 may be executed in one or more external electronic devices among the external electronic devices 102 , 104 , or 108 .
  • the electronic device 101 when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead of executing the function or service by itself.
  • one or more external electronic devices may be requested to perform the function or at least part of the service.
  • One or more external electronic devices receiving the request may execute at least a part of the requested function or service or an additional function or service related to the request, and deliver the execution result to the electronic device 101 .
  • the electronic device 101 may provide the result as at least part of a response to the request as it is or additionally processed.
  • cloud computing distributed computing, mobile edge computing (MEC), or client-server computing technology may be used.
  • the electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing.
  • the external electronic device 104 may include an internet of things (IoT) device.
  • Server 108 may be an intelligent server using machine learning and/or neural networks. According to one embodiment, the external electronic device 104 or server 108 may be included in the second network 199 .
  • the electronic device 101 may be applied to intelligent services (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.
  • an electronic device 200 (eg, the electronic device 101 of FIG. 1 ) according to an embodiment has a first side (or front side) 210A and a second side (or back side). 210B, and a housing 210 including a side surface 210C surrounding a space between the first surface 210A and the second surface 210B, and connected to at least a part of the housing 210, and the electronic
  • the apparatus 200 may include attachment members 250 and 260 configured to detachably attach the device 200 to a part of the user's body (eg, a wrist or an ankle).
  • the housing may refer to a structure that forms part of the first face 210A, the second face 210B, and the side face 210C of FIGS. 2A and 2B .
  • the first surface 210A may be formed by a front plate 201 (eg, a glass plate or a polymer plate including various coating layers) that is substantially transparent at least in part.
  • the second face 210B may be formed by the substantially opaque back plate 207 .
  • the rear plate 207 is formed, for example, of coated or tinted glass, ceramic, polymer, metal (eg, aluminum, stainless steel (STS), or magnesium), or a combination of at least two of the foregoing. It can be.
  • the side surface 210C is coupled to the front plate 201 and the rear plate 207 and may be formed by a side bezel structure (or “side member”) 206 including metal and/or polymer.
  • the back plate 207 and the side bezel structure 206 may be integrally formed and include the same material (eg, a metal material such as aluminum).
  • the binding members 250 and 260 may be formed of various materials and shapes. Integral and plurality of unit links may be formed to flow with each other by woven material, leather, rubber, urethane, metal, ceramic, or a combination of at least two of the above materials.
  • the electronic device 200 includes a display 220 (see FIG. 3), audio modules 205 and 208, sensor modules 211, key input devices 202, 203 and 204, and connector holes ( 209) may include at least one or more. In some embodiments, the electronic device 200 omits at least one of the components (eg, the key input devices 202, 203, 204, the connector hole 209, or the sensor module 211) or has other components. Additional elements may be included.
  • the display 220 may be visually exposed, for example, through a substantial portion of the front plate 201 .
  • the shape of the display 220 may be a shape corresponding to the shape of the front plate 201, and may have various shapes such as a circular shape, an oval shape, or a polygonal shape.
  • the display 220 may be coupled to or disposed adjacent to a touch sensing circuit, a pressure sensor capable of measuring the strength (pressure) of a touch, and/or a fingerprint sensor.
  • the audio modules 205 and 208 may include a microphone hole 205 and a speaker hole 208 .
  • a microphone for acquiring external sound may be disposed inside the microphone hole 205, and in some embodiments, a plurality of microphones may be disposed to detect the direction of sound.
  • the speaker hole 208 can be used as an external speaker and a receiver for a call.
  • the speaker hole 208 and the microphone hole 205 may be implemented as one hole, or a speaker may be included without the speaker hole 208 (eg, a piezo speaker).
  • the sensor module 211 may generate an electrical signal or data value corresponding to an internal operating state of the electronic device 200 or an external environmental state.
  • the sensor module 211 may include, for example, a biometric sensor module 211 (eg, an HRM sensor) disposed on the second surface 210B of the housing 210 .
  • the electronic device 200 includes a sensor module (not shown), for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a color sensor, an IR (infrared) sensor, a bio sensor, a temperature sensor, At least one of a humidity sensor and an illuminance sensor may be further included.
  • the sensor module 211 may include electrode regions 213 and 214 forming a part of the surface of the electronic device 200 and a biosignal detection circuit (not shown) electrically connected to the electrode regions 213 and 214. there is.
  • the electrode regions 213 and 214 may include a first electrode region 213 and a second electrode region 214 disposed on the second surface 210B of the housing 210 .
  • the sensor module 211 may be configured such that the electrode areas 213 and 214 obtain an electrical signal from a part of the user's body, and the biosignal detection circuit detects the user's biometric information based on the electrical signal.
  • the key input devices 202, 203, and 204 include a wheel key 202 disposed on a first surface 210A of the housing 210 and rotatable in at least one direction, and/or a side surface 210C of the housing 210. ) may include side key buttons 203 and 204 disposed on.
  • the wheel key may have a shape corresponding to the shape of the front plate 201 .
  • the electronic device 200 may not include some or all of the above-mentioned key input devices 202, 203, and 204, and the key input devices 202, 203, and 204 that are not included may display 220 may be implemented in other forms such as soft keys.
  • the connector hole 209 may accommodate a connector (eg, a USB connector) for transmitting and receiving power and/or data to and from an external electronic device and a connector for transmitting and receiving an audio signal to and from an external electronic device.
  • a connector eg, a USB connector
  • Other connector holes may be included.
  • the electronic device 200 may further include, for example, a connector cover (not shown) that covers at least a portion of the connector hole 209 and blocks external foreign substances from entering the connector hole.
  • the binding members 250 and 260 may be detachably attached to at least a partial region of the housing 210 using the locking members 251 and 261 .
  • the fastening members 250 and 260 may include one or more of a fixing member 252 , a fixing member fastening hole 253 , a band guide member 254 , and a band fixing ring 255 .
  • the fixing member 252 may be configured to fix the housing 210 and the fastening members 250 and 260 to a part of the user's body (eg, wrist, ankle, etc.).
  • the fixing member fastening hole 253 corresponds to the fixing member 252 to fix the housing 210 and the fastening members 250 and 260 to a part of the user's body.
  • the band guide member 254 is configured to limit the movement range of the fixing member 252 when the fixing member 252 is fastened to the fixing member fastening hole 253, so that the fastening members 250 and 260 are attached to a part of the user's body. It can be tightly bonded.
  • the band fixing ring 255 may limit the movement range of the fastening members 250 and 260 in a state in which the fixing member 252 and the fixing member fastening hole 253 are fastened.
  • an electronic device 300 (eg, the electronic device 101 of FIG. 1 or the electronic device 200 of FIG. 2 ) includes a side bezel structure 310, a wheel key 320, and a front plate 201 ), display 220, first antenna 350, second antenna 355, support member 360 (eg bracket), battery 370, printed circuit board 380, sealing member 390, A rear plate 393 and coupling members 395 and 397 may be included.
  • At least one of the components of the electronic device 300 may be the same as or similar to at least one of the components of the electronic device 101 of FIG. 1 or the electronic device 200 of FIG. 2 , and overlapping descriptions may be made. is omitted below.
  • the support member 360 may be disposed inside the electronic device 300 and connected to the side bezel structure 310 or integrally formed with the side bezel structure 310 .
  • the support member 360 may be formed of, for example, a metal material and/or a non-metal (eg, polymer) material.
  • the support member 360 may have the display 220 coupled to one surface and the printed circuit board 380 coupled to the other surface.
  • a processor, memory, and/or interface may be mounted on the printed circuit board 380 .
  • the processor may include, for example, one or more of a central processing unit, a graphic processing unit (GPU), an application processor, a sensor processor, or a communication processor.
  • Memory may include, for example, volatile memory or non-volatile memory.
  • the interface may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, and/or an audio interface.
  • HDMI high definition multimedia interface
  • USB universal serial bus
  • the interface may electrically or physically connect the electronic device 300 to an external electronic device, and may include a USB connector, an SD card/MMC connector, or an audio connector.
  • the battery 370 is a device for supplying power to at least one component of the electronic device 300, and may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell. there is. At least a portion of the battery 370 may be disposed on substantially the same plane as the printed circuit board 380 , for example.
  • the battery 370 may be integrally disposed inside the electronic device 200 or may be disposed detachably from the electronic device 200 .
  • the first antenna 350 may be disposed between the display 220 and the support member 360 .
  • the first antenna 350 may include, for example, a near field communication (NFC) antenna, a wireless charging antenna, and/or a magnetic secure transmission (MST) antenna.
  • the first antenna 350 may, for example, perform short-range communication with an external device, wirelessly transmit/receive power required for charging, and transmit a short-range communication signal or a magnetic-based signal including payment data.
  • an antenna structure may be formed by a part of the side bezel structure 310 and/or the support member 360 or a combination thereof.
  • the second antenna 355 may be disposed between the printed circuit board 380 and the rear plate 393 .
  • the second antenna 355 may include, for example, a near field communication (NFC) antenna, a wireless charging antenna, and/or a magnetic secure transmission (MST) antenna.
  • the second antenna 355 may, for example, perform short-range communication with an external device, wirelessly transmit/receive power required for charging, and transmit a short-range communication signal or a magnetic-based signal including payment data.
  • an antenna structure may be formed by a part of the side bezel structure 310 and/or the rear plate 393 or a combination thereof.
  • the sealing member 390 may be positioned between the side bezel structure 310 and the rear plate 393 .
  • the sealing member 390 may be configured to block moisture and foreign substances from entering into the space surrounded by the side bezel structure 310 and the back plate 393 from the outside.
  • Electronic devices may be devices of various types.
  • the electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance.
  • a portable communication device eg, a smart phone
  • a computer device e.g., a smart phone
  • a portable multimedia device e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a portable medical device
  • a camera e.g., a camera
  • a wearable device e.g., a smart bracelet
  • first, second, or first or secondary may simply be used to distinguish a given component from other corresponding components, and may be used to refer to a given component in another aspect (eg, importance or order) is not limited.
  • a (e.g., first) component is said to be “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicatively.”
  • the certain component may be connected to the other component directly (eg by wire), wirelessly, or through a third component.
  • module used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logical blocks, parts, or circuits.
  • a module may be an integrally constructed component or a minimal unit of components or a portion thereof that performs one or more functions.
  • the module may be implemented in the form of an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • a storage medium eg, internal memory 136 or external memory 138
  • a machine eg, electronic device 101
  • a processor eg, the processor 120
  • a device eg, the electronic device 101
  • the one or more instructions may include code generated by a compiler or code executable by an interpreter.
  • the device-readable storage medium may be provided in the form of a non-transitory storage medium.
  • the storage medium is a tangible device and does not contain a signal (e.g. electromagnetic wave), and this term refers to the case where data is stored semi-permanently in the storage medium. It does not discriminate when it is temporarily stored.
  • a signal e.g. electromagnetic wave
  • the method according to various embodiments disclosed in this document may be included and provided in a computer program product.
  • Computer program products may be traded between sellers and buyers as commodities.
  • a computer program product is distributed in the form of a device-readable storage medium (e.g. compact disc read only memory (CD-ROM)), or through an application store (e.g. Play StoreTM) or on two user devices (e.g. It can be distributed (eg downloaded or uploaded) online, directly between smart phones.
  • a device-readable storage medium e.g. compact disc read only memory (CD-ROM)
  • an application store e.g. Play StoreTM
  • two user devices e.g. It can be distributed (eg downloaded or uploaded) online, directly between smart phones.
  • at least part of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium such as a manufacturer's server, an application store server, or a relay server's memory.
  • each component (eg, module or program) of the above-described components may include a single object or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is.
  • one or more components or operations among the aforementioned corresponding components may be omitted, or one or more other components or operations may be added.
  • a plurality of components eg modules or programs
  • the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by a corresponding component of the plurality of components prior to the integration. .
  • the actions performed by a module, program, or other component are executed sequentially, in parallel, iteratively, or heuristically, or one or more of the actions are executed in a different order, or omitted. or one or more other actions may be added.
  • FIG. 4 is a diagram illustrating a process of performing gesture recognition in an electronic device.
  • gesture recognition is used in electronic devices.
  • the user may perform an open-clench-open (OCO) gesture to receive the call, and the electronic device may receive the incoming call by recognizing the user's OCO gesture.
  • OCO open-clench-open
  • the electronic device must observe the signal of the sensor and perform a segmentation operation to crop the part where the signal bounces, and to extract various features from the signal part obtained by performing the segmentation operation. (classification) operation must be performed.
  • classifier capable of determining the type of gesture represented by input data.
  • the electronic device may collect training data samples for each of the gestures to be trained, input the collected training data samples to the classifier, and repeatedly train the classifier to generate a classifier.
  • FIG. 5 is a diagram illustrating a process of forming a network capable of classifying gestures through training in a classifier.
  • a network 510 capable of classifying gestures may be formed through training in the classifier 501 .
  • the network 510 may include a multi-layer perceptron network 511 and a fully connected network 512 .
  • the multilayer perceptron network 511 may include a plurality of nodes, and each of the plurality of nodes may transmit information determined to be active or inactive based on segmented information of input data to another node.
  • the multi-layer perceptron network 511 may classify gesture characteristics of input data in detail.
  • the fully connected network 512 may calculate a score corresponding to the input data based on the characteristics of the gesture of the input data finely classified from the multilayer perceptron network 511 .
  • the multilayer perceptron network 511 is composed of nodes for features that can subdivide the two categories in detail. .
  • the multi-layer perceptron network 511 may transmit information about finely classified characteristics of the input data to the fully connected network 512 when any input data is input, and the fully connected network 512 may transmit information about the input data.
  • a score for an 'OCO gesture' and a score for each of a 'non-OCO gesture' may be calculated by integrating the finely classified features.
  • the network 510 may determine which gesture the input signal represents based on the score calculated for each category. For example, when the score for 'OCO gesture' is calculated as 90 and the score for 'non-OCO gesture' is calculated as 10, the network 510 determines that the input data represents the 'OCO gesture'. can
  • the training data set 511 should consist of only accurate data samples, and the data samples included in the training data set should be able to represent data samples for gestures. That is, in order to generate a training data set, a sufficient number of data samples must be collected to be regarded as representing data samples for gestures. To this end, conventionally, various people (eg gender, age, skin tone), various situations (eg standing situation, sitting situation, walking situation), various wearing conditions (eg loose wearing condition, Tight wearing condition), etc., were collected, and a training data set was created by inspecting the collected data samples.
  • various people eg gender, age, skin tone
  • various situations eg standing situation, sitting situation, walking situation
  • various wearing conditions eg loose wearing condition, Tight wearing condition
  • FIG. 6 is a flowchart schematically illustrating an operation of updating a classifier by an electronic device according to an exemplary embodiment.
  • the electronic device may include a universal classifier trained with a training data set.
  • the user can use the universal classifier included in the electronic device as a basis.
  • the user may request an update of the universal classifier (hereinafter referred to as 'classifier') if he or she wants to adapt the universal classifier to himself or if it is determined that the universal classifier is not suitable for the user.
  • the electronic device may update the classifier by collecting target gesture samples from the user.
  • the electronic device may collect target gesture samples.
  • a user may perform an operation for a target gesture.
  • the electronic device may collect and store a data sample (hereinafter referred to as 'target gesture sample') for the target gesture by recognizing an operation of the user's target gesture with a sensor.
  • 'target gesture sample' a data sample for the target gesture by recognizing an operation of the user's target gesture with a sensor.
  • the electronic device may identify first leaves corresponding to the target gesture sample by inputting the collected target gesture sample to a classifier.
  • the electronic device may store information about the identified first leaves. For example, the electronic device may store an index number of each of the identified first leaves.
  • the first leaves corresponding to the target gesture sample may represent leaves finally reached for each tree of the multilayer perceptron network when the target gesture sample is input as input data to the classifier.
  • the index number of a leaf is described as a leaf index.
  • the electronic device may update a compensation value (also referred to as a return value) of each of the identified first leaves so that the identified first leaves have more weight.
  • the electronic device may update the classifier to suit the user by applying and updating the compensation value of each of the identified first leaves.
  • the electronic device may load the compensation value of each of the identified first leaves from the registry corresponding to the stored leaf index, and update the classifier by increasing the compensation value of each of the identified first leaves by an update unit value. .
  • FIG. 7 is a diagram illustrating a process of updating a classifier by an electronic device according to an exemplary embodiment.
  • the electronic device may update the classifier by updating compensation values of leaves.
  • the electronic device may update the compensation values of the leaves 710 arranged in a step of transitioning from the multi-perceptron network 511 to the fully connected network 512 .
  • the leaves 710 may represent nodes placed just before an operation to calculate a score for each classification in the fully connected network 512 is performed. Updating the compensation values of the leaves may represent weighting a network path reaching the leaf when input data is input to the network.
  • the multilayer perceptron network 511 is a decision tree network is mainly described, but the multilayer perceptron network is not necessarily limited thereto.
  • FIG. 8 is a diagram illustrating the structure of a classifier 810 based on a decision tree network.
  • the classifier 810 may include a plurality of trees 811, 812, ..., 813, and within each tree, a branch to the right or left from one node based on a binary decision (binary decision) branch to reach the next node, and branches may finally terminate at a leaf.
  • a reward value of the reached leaf may be returned.
  • the electronic device receives the leaf from the registry A 0 corresponding to the leaf index of the leaf 811-0.
  • the compensation value of (811-0) can be loaded.
  • the classifier can be optimized by a training algorithm, and as the classifier is trained with the training data set, gesture recognition performance for the entire group that can be represented by the training data set can be maximized. However, even if the classifier maximizes the gesture recognition performance for the entire group, it cannot be said that the gesture recognition performance for the individual is maximized. In other words, since the classifier is not optimized for gesture recognition performance for each individual, gesture recognition performance for each individual is not maximized.
  • An electronic device may maximize gesture recognition performance of a classifier for an individual by adjusting compensation values of leaves of the classifier.
  • FIG. 9 is a diagram illustrating a process of updating the classifier by inputting a data sample to the decision tree network-based classifier 810 by an electronic device according to an embodiment.
  • the electronic device may input input data to the classifier 810 based on the decision tree network.
  • the input data may be the target gesture sample 911.
  • each of the trees 811, 812, ..., 813 of the classifier 810 performs branch selection and node selection based on a binary decision.
  • the input data 911 is input to the first tree 811
  • the highest node 821 of the first tree 811 may be selected.
  • either the right branch or the left branch may be selected according to a condition of the node 821.
  • the node 831 which is a left child node of the node 821, may be selected. can be selected.
  • another branch may be selected, and child nodes of the node 831 may be selected by the selected branch.
  • the leaf 811-1 within the tree 811 can be finally selected, and the reward value of the selected leaf 811-1 is set as the target gesture.
  • the sample 911 is a score obtained from the tree 811 .
  • the electronic device may calculate scores for each of the trees 812, ..., 813 by inputting the target gesture sample 911 to each of the remaining trees 812, ..., 813.
  • the electronic device may sum all the scores calculated for each of the trees 811, 812, ..., 813, and determine the summed score as the final score obtained from the classifier 810 for the target gesture sample. .
  • the electronic device may update the classifier 810 according to the user by inputting the target gesture sample 911 to the classifier 810 .
  • the electronic device may input the target gesture sample 911 for each of the trees 811, 812, ..., 813, and may identify first leaves selected for each tree in correspondence with the target gesture sample 911.
  • the first leaf may indicate a leaf selected to correspond to the target gesture sample.
  • the electronic device sets the compensation value of each of the identified leaves as an update unit value ( ) can be increased by
  • the electronic device inputs the target gesture sample 911 to the classifier 810 and selects first leaves 811-1, 812-15, ..., for each tree 811, 812, ..., 813. 813-2)
  • Each leaf index may be stored.
  • the electronic device may increase the compensation value of each of the first leaves 811-1, 812-15, ..., 813-2 loaded from the registry corresponding to each of the stored leaf indexes.
  • a process of inputting a target gesture sample to the classifier and increasing a reward value of each of the first leaves selected for each tree corresponding to the target gesture sample will be described as an upward update operation of the classifier.
  • the total number of trees included in the classifier 810 may be N.
  • N may represent a natural number of 1 or greater.
  • An update unit value ( ) to update the classifier 810 if the same target gesture sample is input to the classifier 810, the final score calculated for the target gesture sample is the target gesture sample in the classifier before the upward update operation is performed. than the final score calculated for will increase as much as
  • the electronic device may update the classifier 810 according to the user by inputting false alarm data samples to the classifier 810 .
  • the false alarm data sample is not a data sample representing the target gesture, but may represent a data sample that the classifier 810 incorrectly determines as the target gesture.
  • the classifier 810 When the reward value of the first leaves is increased by inputting only the target gesture sample to the classifier 810, the classifier 810 generates a false alarm because the sum of reward values of all leaves in the classifier 810 continues to increase. There are side effects that may increase the probability of an alarm).
  • a false alarm may indicate that when a data sample that does not represent a target gesture is input to the classifier 810, the classifier 810 determines that the corresponding data sample represents the target gesture.
  • the classifier is updated using only target gesture samples.
  • a data sample related to a gesture other than the target gesture is input to the updated classifier, and leaves with a greatly increased reward value corresponding to the input data sample are selected, the final score calculated for the data sample is obtained from the classifier before the update.
  • the classifier has a high probability of erroneously determining that the corresponding data sample represents the target gesture.
  • the operation of the opposite benefit needs to be performed together so that the sum of compensation values of all leaves does not increase.
  • the electronic device may input false alarm data samples to the classifier 810 to identify second leaves selected for each of the trees 811, 812, ..., 813.
  • the second leaf may indicate a leaf selected in response to the false alarm data sample.
  • the electronic device may decrease a compensation value of each of the identified second leaves.
  • the electronic device sets the compensation value of each of the identified second leaves to an update unit value ( ) can be reduced by
  • a process of inputting a false alarm data sample to the classifier and reducing a compensation value of each of the second leaves selected for each tree corresponding to the false alarm data sample will be described as a downward update operation of the classifier.
  • the total number of trees included in the classifier 810 is N, and the compensation value of each of the second leaves corresponding to the false alarm data sample is an update unit value ( ) to update the classifier 810, when the same false alarm data sample is input to the classifier 810, the final score calculated for the false alarm data sample corresponds to the corresponding false alarm data sample in the classifier before the downward update operation is performed. than the final score calculated for will decrease as much as
  • the final score calculated by the classifier for the false alarm data sample is The probability of determining that the corresponding false alarm data sample is data representing the target gesture may be reduced by decreasing the number of times.
  • the electronic device downwardly updates the classifier using one false alarm data sample, the sum of compensation values of all leaves in the classifier Since it decreases by , it is possible to maintain the balance of the sum of reward values together with the upward update operation of the classifier.
  • the electronic device initially provides a universal classifier to the user, and updates the universal classifier to a classifier suitable for the user by updating the classifier using the target gesture sample and the false alarm data sample.
  • the electronic device may induce the user to perform a target gesture operation by displaying a user interface (UI) on the screen of the electronic device, and may collect target gesture samples by recognizing the user's target gesture operation.
  • UI user interface
  • the electronic device may provide a user with a clear guide for target gesture operations and may calculate a probability of indicating a target gesture for each data sample acquired from the user, valid target gesture samples may be selected from the user.
  • the electronic device may obtain false alarm data samples from a training data set used for training the universal classifier instead of acquiring false alarm data samples from the user.
  • the electronic device may initially provide false alarm data samples extracted from a training data set together with a universal classifier to a user.
  • the electronic device updates the classifier using false alarm data samples obtained from the training data set used to train the universal classifier, but it is not necessarily limited thereto, and the electronic device updates the classifier in the process of updating the classifier.
  • a false alarm data sample may be obtained through a data sample obtained from a user.
  • the electronic device may load updated compensation values of leaves in the classifier 810 in various ways.
  • the electronic device may generate registries corresponding to each of all leaves in the classifier 810 and load the updated compensation value of the leaf by storing the updated compensation value of the leaf in the created registry.
  • the electronic device provides registries (eg, A 0 , A 1 , A 2 , . . . ) can be created.
  • the electronic device may store the updated compensation value of the leaf in the registry corresponding to the leaf index of the leaf.
  • the updated compensation value of the leaf may represent a value obtained by adding the updated compensation value of the leaf, which is a value representing the degree of increase or decrease of the existing compensation value, to the existing compensation value of the leaf.
  • the electronic device may maintain the existing compensation value of the leaf stored in the registry corresponding to the leaf index of the leaf.
  • the method of storing the updated compensation value of the leaf in the registry corresponding to the leaf has an advantage that the electronic device can quickly load the updated compensation value of the leaf from the registry corresponding to the leaf.
  • the electronic device instead of creating registries corresponding to each of all the leaves in the classifier 810 as shown in FIG. 8 or 9, the electronic device corresponds to each of the leaves whose compensation value is updated according to the update of the classifier.
  • the updated compensation value of the leaf can be loaded by creating registries that are More specifically, when updating the classifier, the electronic device may create registries corresponding to leaves whose compensation values are updated.
  • the electronic device may store a leaf index of a specific leaf for which a compensation value is updated and an updated value of the specific leaf in a registry corresponding to the specific leaf. Thereafter, the electronic device may load the leaf index and the updated value stored in the registry and calculate the updated compensation value of the leaf by adding the updated value to the compensation value of the leaf corresponding to the leaf index.
  • the electronic device may calculate final compensation values for each of the leaves in the classifier 810 by loading a leaf index and an updated value stored in each of the generated registries.
  • the method of loading the updated compensation value of a leaf by creating registries corresponding to each of the leaves whose compensation value is updated uses storage space efficiently because it is sufficient to create registries corresponding to the leaves whose compensation value is updated.
  • FIG. 10 is a diagram illustrating an overall process of updating a classifier by an electronic device according to an exemplary embodiment.
  • the electronic device may extract false alarm data samples 1030 from the training data set 1010 used to train the classifier 1020.
  • the electronic device may obtain the false alarm data samples 1030 from the training data set 1010 used for training the classifier 1020.
  • a gesture type represented by each of the data samples included in the training data set 1010 may be predetermined.
  • the electronic device may extract data samples that do not represent the target gesture from among data samples included in the training data set 1010 .
  • the electronic device may input each of the extracted data samples to the classifier 1020 .
  • the electronic device may use data samples classified as target gestures as the false alarm data samples 1030 when they are input to the classifier 1020 from among the extracted data samples.
  • the electronic device may perform a downward update operation of the classifier using the false alarm data samples 1030 .
  • the electronic device may map one target gesture sample 1041 and one false alarm data sample 1042 collected from the user into one pair.
  • the electronic device may simultaneously perform upward update of the classifier 1020 based on the target gesture sample 1041 and downward update of the classifier 1020 based on the false alarm data sample 1042 .
  • the electronic device may update the classifier by alternately performing upward updating of the classifier and downward updating of the classifier once.
  • the electronic device may input the target gesture sample 1041 to the classifier 1020 to identify first leaves selected for each tree corresponding to the target gesture sample 1041 .
  • the electronic device may store leaf indices of first leaves selected in correspondence with the target gesture sample 1041 .
  • the electronic device may input the false alarm data sample 1042 to the classifier 1020 to identify second leaves selected for each tree corresponding to the false alarm data sample 1042 .
  • the electronic device may store leaf indices of second leaves selected in response to the false alarm data sample 1042 .
  • the electronic device may update the classifier 1020 by increasing and decreasing reward values of leaves in the classifier 1020.
  • the electronic device sets the compensation value of each of the first leaves identified corresponding to the target gesture sample 1041 to an update unit value ( ), the up-update operation 1051 of the classifier 1020 may be performed.
  • the electronic device sets the compensation value of each of the identified second leaves corresponding to the false alarm data sample 1042 to an update unit value ( ), the down update operation 1052 of the classifier 1020 may be performed.
  • the electronic device may update the classifier 1020 by simultaneously performing an upward update operation of the classifier based on the target gesture sample and a downward update operation of the classifier based on the false alarm data sample mapped to the corresponding target gesture sample. .
  • the electronic device updates the classifier 1020 by inputting the target gesture sample 1041 and the false alarm data sample 1042 as a pair to the classifier 1020, false alarms among leaves matching the target gesture may increase. It may indicate that the compensation value is increased only for the remaining leaves except for the leaves with For example, if a target gesture and a non-target gesture have a common characteristic of moving the wrist left and right once, if the reward value of the leaf that assigns a score to this common characteristic increases, the gesture that is not the target gesture When the data sample for is also input to the classifier, it obtains a high score.
  • the electronic device inputs the target gesture sample 1041 and the false alarm data sample 1042 as a pair to the classifier 1020 to update the classifier, thereby assigning a score to common characteristics of the target gesture and non-target gestures. It is possible to prevent the compensation value of from increasing.
  • the electronic device may update the classifier 1020 according to Equation 1 below.
  • R n,m is the compensation value of the m-th leaf in the n-th tree
  • s k t is a target gesture sample used in the k-th update
  • s k f is a false alarm data sample used in the k-th update
  • the electronic device may update the classifier by alternately performing an upward update operation of the classifier using the target gesture sample and a downward update operation of the classifier using the false alarm data sample once each.
  • the update unit value ( ) is set large, the performance of the classifier may rapidly change when an update is performed on the classifier.
  • the electronic device may update the classifier with a classifier suitable for the user through a small number of updates to the classifier.
  • the small number of updates increases the classifier's dependency on a specific data sample, thereby increasing the probability of generating a false alarm. Therefore, the electronic device needs to update the classifier a large number of times to update the classifier suitable for the user, and the update unit value ( ) should not be too large.
  • An electronic device according to an embodiment has an update unit value (with a constraint condition according to Equation 2 below) ) range can be set.
  • Equation 2 is a predetermined value
  • N is the total number of trees in the classifier
  • Is a statistical function for the compensation value of all leaves may represent the weight of the change value due to the update.
  • 11 is a table illustratively illustrating performance evaluation results for each updated classifier when the electronic device updates the classifier for each user according to an embodiment.
  • the electronic device updates the classifier for each user according to Equation 1 using 10 target gesture samples collected for each user, the update unit value is 0.01, and the target gesture is "Knock Down twice", It is assumed that the classifier is a classifier capable of classifying the target gesture and the remaining gestures excluding the target gesture into two types.
  • the electronic device may calculate target gesture detection probability and false alarm probability for the classifier before being updated. For example, in the case where the classifier before being updated normally detects 989 data samples as target gestures among data samples representing 1074 target gestures, and falsely detects 85 data samples as gestures other than the target gesture The target gesture recognition rate can be measured as 92.1%. If the classifier before updating incorrectly detects 147 data samples among 108055 data samples indicating gestures other than the target gesture as the target gesture, the false alarm probability may be measured as 0.1%. After updating the classifiers having the above performance for each user and then evaluating the performance of each updated classifier, improved performance can be confirmed as shown in the table shown in FIG. 11 .
  • a target gesture recognition rate of a classifier may indicate a probability that a data sample representing a target gesture is recognized by the classifier as the target gesture.
  • the false alarm probability by the classifier before the update and the false alarm probability by the classifier after the update slightly increased when each user saw them individually.
  • the target gesture recognition rate by the classifier after the update is mostly increased for the user whose target gesture recognition rate by the classifier before the update was low.
  • the causes of these side effects can be explained in two ways.
  • the first reason is that compensation values of specific leaves in the classifier are excessively increased when upward updating of the classifier is performed based on the target gesture sample.
  • the second reason is that, when performing downlink update of the classifier, it has a dependency on the false alarm data sample used for downlink update.
  • the first cause of the side effects of the classifier is that when the classifier is adjusted to the user or the classifier is updated to some extent to a state suitable for the user, the electronic device continuously updates the classifier to change the reward value of the leaf, rather than the performance of the classifier. This means that it can cause a decline.
  • two methods capable of preventing side effects of the classifier due to the first cause will be described.
  • the electronic device is an update unit value used for updating the classifier ( ) to prevent side effects.
  • the electronic device has an update unit value ( ) at a constant constant value, the update unit value ( ) can be adjusted in inverse proportion to the gesture recognition rate of the current classifier to minimize side effects.
  • the electronic device may adjust an update unit value in inverse proportion to the target gesture recognition rate of the current classifier.
  • the update unit value ( ) can be kept at a constant constant value.
  • the electronic device may update the classifier according to Equation 3 below.
  • Equation 3 R n,m is the compensation value of the m-th leaf in the n-th tree, s k t is a target gesture sample used in the k-th update, s k f is a false alarm data sample used in the k-th update, Is , Is can represent
  • c represents a constant constant value
  • Is a weight added to a constant constant value (c) in the k-th update.
  • can be proportional to may represent a gesture recognition rate derived by the classifier.
  • Figure 12 and It is a graph showing the relationship with
  • the electronic device can prevent side effects by reducing the compensation value of each of the leaves of the classifier.
  • the electronic device when performing upward update and downward update of the classifier once, the electronic device may decrease the compensation value of each of all leaves of the classifier by a compensation adjustment value (b k ).
  • the electronic device may adjust the compensation adjustment value (b k ) in proportion to the target gesture recognition rate of the current classifier after performing up-update of the classifier and down-update of the classifier once.
  • the electronic device may lower the sum of reward values of all leaves of the classifier by reducing all of the reward values of all leaves of the classifier. This is a method that focuses on lowering the false alarm probability of the classifier by lowering the reward value of all leaves using a part of the increased reward value because the reward value of the leaves that assign scores to the characteristics of the target gesture is sufficiently high by updating the classifier. am.
  • the electronic device may update the classifier according to Equation 4 below.
  • the electronic device can adjust the update unit value of the classifier to be inversely proportional to the target gesture recognition rate of the current classifier, while reducing the reward value of each leaf in proportion to the target gesture recognition rate of the classifier. there is.
  • Equation 4 b k is the compensation adjustment value at the kth update.
  • the second cause of the side effects of the classifier means that when the electronic device performs a down-update operation to balance the up-up update operation of the classifier, the classifier may have a dependency on the false alarm data sample used in the down-up update operation. do. For example, it is assumed that a total of 30 false alarm data samples exist. In this case, when the electronic device performs downward update for the classifier 30 times, all false alarm data samples are used and the downward update is performed, so that the compensation value reduction of the leaves can be generally applied. However, when the electronic device performs only 3 downlink updates on the classifier, the downlink update is performed using only 3 false alarm data samples, so the classifier has a high dependency on the 3 false alarm data samples. Occurs. Accordingly, when performing downward update of the classifier, the electronic device may minimize side effects by identifying leaves corresponding to each of all false alarm data samples and additionally reducing a compensation value of the identified leaves.
  • the electronic device when performing a downward update of the classifier, identifies leaves corresponding to each of all false alarm data samples, and for each of the identified leaves, the corresponding leaf corresponds to all false alarm data samples. A statistical value indicating the number of times selected is calculated, and the compensation value of the corresponding leaf may be further reduced to a value proportional to the statistical value calculated for the corresponding leaves.
  • the electronic device may store leaf indexes of identified leaves corresponding to each of all false alarm data samples, and calculate statistical values for each leaf by counting the number of times the leaf index of each of the identified leaves appears. .
  • the electronic device may reduce the compensation value to a large value for a leaf with a high statistical value, and may decrease the compensation value to a small value for a leaf with a low statistical value.
  • the electronic device may update the classifier according to Equation 5 below.
  • Equation 5 may represent the ratio of the application amount of the false alarm data sample used when performing the downward update of the classifier and the application amount of all false alarm data samples. Is a statistical function, and may represent, for example, an average function or a mean square function. may indicate that the compensation value of the leaf is deducted by the individual false alarm data sample used when performing the downward update, may indicate that the compensation value of the leaf is deducted by the statistical value of the leaves identified corresponding to all false alarm data samples during downlink update.
  • the electronic device adjusts the update unit value in inverse proportion to the target gesture recognition rate of the current classifier, reduces the reward value of each leaf in proportion to the target gesture recognition rate of the classifier, and generates all false alarms. Compensation values of leaves identified corresponding to each of the data samples may be additionally reduced according to statistical values of the identified leaves.
  • identifying leaves corresponding to all false alarm data samples and calculating a statistical value for each of the identified leaves may impose a large load on the electronic device.
  • the electronic device By increasing , the load that the electronic device must bear can be reduced by identifying leaves corresponding to all false alarm data samples and calculating statistical values of leaf nops whenever the classifier is updated.
  • the classifier's target gesture recognition rate rises quickly because the upward update has a large effect during the initial update of the classifier, and updates are accumulated for the classifier to improve the classifier's performance.
  • the target gesture recognition rate increases, the effect of the downward update increases, so that the classifier may be updated in a direction of reducing the probability of false alarms.
  • the electronic device may collect a target gesture sample by recognizing a gesture action performed by a user.
  • the electronic device may collect at least one target gesture sample having a target gesture probability greater than or equal to a threshold probability by cropping a signal of a sensor generated by recognizing a user's target gesture motion, and collect at least one sample of the target gesture.
  • One target gesture sample can be used to update the classifier.
  • the electronic device may collect target gesture samples in two ways. The electronic device may collect target gesture samples using a passive method and an active method.
  • the electronic device passively collects the target gesture sample means that the electronic device starts and ends a sample collection timer (hereinafter referred to as 'timer') to collect the target gesture sample, and is performed during the duration of the timer. It may indicate that a target gesture sample is collected by recognizing a target gesture motion of a user who has been selected.
  • 'timer' a sample collection timer
  • the electronic device can assume that the user performed the target gesture operation during the duration of the timer.
  • the electronic device passively collects target gesture samples there is a disadvantage in that the user must actively perform the target gesture operation at the timing of performing the target gesture operation.
  • Collecting the target gesture sample by the electronic device in an active way may indicate that the electronic device actively recognizes the user's target gesture operation and collects the target gesture sample when the user performs the target gesture operation at an arbitrary point in time. .
  • an electronic device actively collects target gesture samples there is an advantage that a user can perform a target gesture operation at a desired time without time constraints.
  • the electronic device collects target gesture samples in an active method since the user cannot know when to perform the target gesture operation, the electronic device continuously crops the signal of the sensor and performs the segmentation operation.
  • it is necessary to monitor whether the corresponding signal part is a signal corresponding to the target gesture while performing a classification operation for extracting various features from the signal part obtained by performing .
  • the electronic device actively collects target gesture samples the electronic device has a disadvantage in that it cannot specify a section in which the user's target gesture operation was performed.
  • FIG. 14 is a diagram illustrating a process of collecting target gesture samples by an electronic device in a passive method according to an embodiment.
  • the user may activate the object 1411 for passively collecting the target gesture sample displayed on the screen 1410 of the electronic device through user manipulation.
  • Activation of an object may include, for example, a user's manipulation of touching the object.
  • the object 1411 may represent an object for which the electronic device is configured to collect target gesture samples in a passive manner.
  • the electronic device may start a first timer in response to the object 1411 being activated.
  • the electronic device may advance the first timer for the first time. When a predetermined time elapses from the start of the first timer, the electronic device may generate a vibration to notify the user of the desired target gesture operation timing.
  • the electronic device may display the number 1421 (eg, 1) of collected target gesture samples on the screen.
  • a user may activate an object 1412 (eg, an “Apply updates” button) for updating a classifier displayed on the screen through a user manipulation.
  • the object 1412 may represent an object that is configured to update the classifier by inputting at least one target gesture sample stored in the electronic device to the classifier.
  • the electronic device may update the classifier by inputting at least one stored target gesture sample into the classifier.
  • the electronic device may perform up-update and down-update operations of the classifier using at least one stored target gesture sample and a false alarm data sample extracted from a training data set.
  • the electronic device may display the number 1422 (eg, 1) of target gesture samples used for updating the classifier up to now on the screen. Then, the user may activate the object 1411 for passively collecting the target gesture samples again, and the electronic device may start the first timer to passively collect the target gesture samples.
  • the number 1422 eg, 1
  • 15 is a diagram illustrating a process of collecting target gesture samples by an active method by an electronic device according to an exemplary embodiment.
  • the user may activate the object 1511 for actively collecting target gesture samples displayed on the screen 1510 of the electronic device through user manipulation.
  • the object 1511 may represent an object for which the electronic device is configured to collect target gesture samples in an active manner.
  • the electronic device may enter a sample collection mode in response to the object 1511 being activated.
  • the electronic device may collect target gesture samples from the user until the sample collection mode is released after entering the sample collection mode.
  • the electronic device may display a message (eg, 'Try Gesture') 1512 requesting the user to perform a target gesture operation on the screen.
  • the user may perform the target gesture operation at any time until the sample collection mode of the electronic device is released.
  • the electronic device may obtain a data sample having a target gesture probability greater than or equal to a threshold probability based on a signal from a sensor.
  • the target gesture probability may indicate a probability that a data sample indicates a target gesture.
  • the electronic device advances a second timer for a second time from the start of the reference data sample to search for a data sample having a higher target gesture probability during the progress period of the second timer. can do. While searching for data samples, the electronic device may display a message 1513 (eg, 'wait catching') 1513 on the screen.
  • the electronic device may store, as the target gesture sample, a data sample having the highest target gesture probability among the reference data samples and data samples having a target gesture probability greater than or equal to a threshold probability retrieved during the second time. Thereafter, the electronic device may display a message (eg, 'Try Gesture') 1514 requesting the user to perform the target gesture operation again on the screen, and may collect target gesture samples from the user according to the above-described method.
  • a user may activate an object 1515 (eg, an “Apply updates” button) for updating a classifier displayed on the screen through a user manipulation.
  • the object 1515 may represent an object that is set to update the classifier by inputting at least one target gesture sample stored in the electronic device to the classifier.
  • the electronic device may update the classifier by inputting at least one stored target gesture sample into the classifier.
  • the electronic device may collect target gesture samples by recognizing the user's target gesture operation until the sample collection mode is released.
  • 16 is a flowchart illustrating a process of manually collecting target gesture samples and updating a classifier by an electronic device according to an embodiment.
  • the electronic device may start a first timer when passively collecting target gesture samples is initiated.
  • the electronic device may start the first timer in response to activation of an object (eg, the object 1411 of FIG. 14 ) for passively collecting target gesture samples by the user.
  • the user may perform the target gesture operation during the progress period from the start time of the first timer to the end time of the first timer.
  • the electronic device may extract data samples having a target gesture probability greater than or equal to a threshold probability among data samples acquired from a signal from a sensor during the duration of the first timer. there is.
  • the electronic device may select a data sample having the highest target gesture probability among data samples having a target gesture probability greater than or equal to a threshold probability, and may store the selected data sample as a target gesture sample.
  • the electronic device may determine whether to update the classifier using at least one stored target gesture sample. When the electronic device does not update the classifier, the electronic device may return to operation 1610 to recognize the user's target gesture motion and additionally store another target gesture sample.
  • the electronic device may update the classifier using at least one stored target gesture sample. For example, the electronic device responds to activation of an object (eg, object 1412 of FIG. 14 or object 1515 of FIG. 15 ) set to update the classifier by the user, and at least one stored target gesture sample. can be used to update the classifier.
  • an object eg, object 1412 of FIG. 14 or object 1515 of FIG. 15
  • the electronic device responds to activation of an object (eg, object 1412 of FIG. 14 or object 1515 of FIG. 15 ) set to update the classifier by the user, and at least one stored target gesture sample. can be used to update the classifier.
  • 17 is a diagram for explaining a process of adjusting a threshold probability, which is a standard for collecting target gesture samples, by an electronic device according to an embodiment.
  • the electronic device may not use, among data samples obtained by recognizing a target gesture operation performed by a user, data samples having a target gesture probability less than a threshold probability as target gesture samples.
  • the criterion for collecting target gesture samples may vary according to the size of the threshold probability.
  • the electronic device may set the threshold probability low before updating the classifier.
  • the electronic device may adjust the threshold probability to increase as the number of target gesture samples used to update the classifier increases.
  • the electronic device may increase the threshold probability based on the number of target gesture samples used to update the classifier up to now. For example, the electronic device responds to a case where an object (eg, the object 1413 of FIG. 14 ) for classifier update is activated by the user, based on the number of target gesture samples used to update the classifier up to now.
  • the threshold probability can be increased. Referring to FIG. 17 , the electronic device may adjust the threshold probability to increase at time points 1701 , 1702 , and 1703 of updating the classifier when an object for updating the classifier is activated by the user.
  • the electronic device may collect more accurate target gesture samples by adjusting the threshold probability.
  • 18 is a flowchart illustrating a process in which an electronic device collects target gesture samples using an active method and updates a classifier according to an exemplary embodiment.
  • the electronic device may enter a sample collection mode.
  • the electronic device may enter a sample collection mode in response to activation of an object (eg, the object 1511 of FIG. 15 ) for actively collecting target gesture samples by the user.
  • an object eg, the object 1511 of FIG. 15
  • the electronic device may acquire one data sample by cropping the signal of the sensor.
  • the electronic device may determine whether the target gesture probability of the obtained data sample is greater than or equal to a threshold probability. When the target gesture probability of the obtained data sample is less than the threshold probability, the electronic device may return to operation 1820 to acquire another data sample by cropping the signal of the sensor.
  • the electronic device may collect the target gesture sample from the sensor signal by using the obtained data sample as a reference data sample.
  • a process of collecting a target gesture sample from a signal of a sensor based on a data sample having a target gesture probability greater than or equal to a threshold probability will be described in more detail with reference to FIG. 19 .
  • the electronic device may store the collected target gesture samples.
  • the electronic device may determine whether to update the classifier using at least one stored target gesture sample. When the electronic device does not update the classifier, it returns to operation 1820 to additionally store another target gesture sample.
  • the electronic device may update the classifier using the stored at least one target gesture sample. For example, the electronic device updates the classifier using at least one stored target gesture sample in response to activation of an object set to update the classifier (eg, the object 1515 of FIG. 15) by the user. can do. For another example, when a predetermined number of target gesture samples not used for updating the classifier are stored, the electronic device may update the classifier using the stored predetermined number of target gesture samples. The predetermined number may be, for example, three. The electronic device may adjust the threshold probability to increase at the time of updating the classifier.
  • the electronic device may continuously collect target gesture samples and update a classifier based on the stored target gesture samples until the sample collection mode is released.
  • 19 is a flowchart illustrating a process of actively collecting target gesture samples by an electronic device according to an embodiment.
  • the electronic device may store the obtained data sample as a reference data sample and start a second timer based on a start point of the reference data sample.
  • the electronic device may initially store the reference data sample.
  • the electronic device may advance a second timer for a second time period, and search for another data sample having a target gesture probability greater than or equal to a threshold probability by cropping a sensor signal during a second time period from the start of the reference data sample. can do.
  • the electronic device may collect a reference data sample, which is a data sample currently stored in the electronic device, as the target data sample.
  • the electronic device may determine whether a target gesture probability of a searched data sample is higher than a target gesture probability of a stored data sample. For example, since the electronic device initially stores the reference data sample, it may determine whether the target gesture probability of another data sample is higher than the target gesture probability of the reference data sample. When the target gesture probability of another data sample is lower than the target gesture probability of the stored data sample, the electronic device returns to operation 1920 to search for a next data sample having a target gesture probability greater than or equal to the threshold probability.
  • the electronic device may discard the stored data sample and store the other data sample as a new data sample.
  • the electronic device may determine whether the second timer has expired. When the second timer does not expire, the electronic device may return to operation 1930 and search for a next data sample having a target gesture probability greater than or equal to a threshold probability.
  • the electronic device may collect a data sample currently stored in the electronic device as a target data sample.
  • the electronic device may collect data samples having the highest target gesture probability as the target data samples during the interval between the start time and end time of the second timer.
  • 20 is a diagram for explaining a process of collecting target gesture samples in an active method by an electronic device according to an embodiment.
  • An electronic device may enter a sample collection mode.
  • the electronic device may obtain data samples by cropping the signal of the sensor.
  • the electronic device may not use data samples 2001 and 2002 whose target gesture probability is less than the threshold probability among the obtained data samples as target gesture samples.
  • the electronic device may collect a target gesture sample by using a data sample 2010 having a target gesture probability greater than or equal to a threshold probability among acquired data samples as a reference data sample. Based on the start time of the data sample 2010, the electronic device may advance the second timer for a second time.
  • the electronic device may store the data sample 2010 and may search for another data sample 2020 having a target gesture probability greater than or equal to a threshold probability during the progress period 2030 of the second timer.
  • the electronic device may discard the previously stored data sample 2010 and newly store another data sample 2020. There is.
  • the electronic device may collect the data sample 2020 currently stored in the electronic device as the target data sample.
  • 21 is a flowchart illustrating an operation after an electronic device collects a target gesture sample, according to an exemplary embodiment.
  • the electronic device may collect target gesture samples in an active method or a passive method.
  • the electronic device may store the collected target gesture samples.
  • the electronic device may perform one of a plurality of operations based on a user's input.
  • the electronic device may update a classifier using at least one stored target gesture sample.
  • the electronic device responds to activation of an object (eg, object 1412 of FIG. 14 or object 1515 of FIG. 15 ) set to update the classifier by the user, and at least one stored target gesture sample. can be used to update the classifier.
  • an object eg, object 1412 of FIG. 14 or object 1515 of FIG. 15
  • the electronic device responds to activation of an object (eg, object 1412 of FIG. 14 or object 1515 of FIG. 15 ) set to update the classifier by the user, and at least one stored target gesture sample. can be used to update the classifier.
  • the electronic device may discard the collected target gesture samples. For example, the electronic device may discard the collected target gesture samples in response to activating an object (eg, a 'Discard sample' button) set to discard the collected target gesture samples by the user. According to an embodiment, the electronic device may discard the most recently collected target gesture sample. For example, when discarding target gesture samples, the electronic device may deduct the number of target gesture samples not used for update by 1 and display the number of target gesture samples on the screen.
  • an object eg, a 'Discard sample' button
  • the electronic device may reset the classifier.
  • the electronic device may reset the classifier in response to activation of an object set by the user to reset the classifier (eg, a 'Reset classifier' button).
  • the electronic device may initialize the classifier before updating. In other words, the electronic device may initialize compensation values of all leaves of the classifier. For example, the electronic device may initialize only the classifier before update while maintaining the stored target gesture sample.
  • FIG. 22 is a diagram illustrating that an electronic device displays a performance evaluation result of a classifier on a screen according to an exemplary embodiment.
  • An electronic device may acquire a data sample by recognizing a user's motion.
  • the electronic device may display the target gesture probability 2211 of the acquired data sample on the screen.
  • the electronic device may calculate a target gesture probability by inputting the obtained data sample to the current classifier.
  • the electronic device may display a message indicating that the target gesture probability is low (eg, 'Low Probability') 2212 .
  • the electronic device may induce the user to perform a more accurate target gesture operation by displaying the message 2212 .
  • the electronic device may collect a target gesture sample based on the acquired data sample when the target gesture probability of the obtained data sample is greater than or equal to a threshold probability.
  • the electronic device may display a message 2224 indicating collection of the target gesture sample (eg, 'Catch done').
  • the electronic device may display the target gesture probability 2221 of the target gesture sample calculated when the target gesture sample is input to the current classifier on the screen.
  • the electronic device may display the target gesture probability 2222 of the target gesture sample calculated by inputting the target gesture sample to the updated classifier using at least one stored target gesture sample on the screen.
  • the electronic device may display the number 2223 of target gesture samples currently stored in the electronic device that have not been used for updating the classifier and the number of target gesture samples that have been used for updating the classifier up to now.
  • the electronic device may display an object 2230 for updating the classifier. When the object 2230 is activated through a user manipulation, the electronic device may update the classifier by using at least one target gesture sample stored in the electronic device and unused for updating the classifier.
  • a message 2242 indicating that the collected target gesture samples are sufficient may be displayed on the screen.
  • the electronic device can notify the user of when to stop performing the target gesture operation by displaying the message 2242, and the user can stop performing the target gesture operation by checking the message 2242.
  • An electronic device includes a memory in which computer-executable instructions are stored, and a processor that accesses the memory to execute the instructions, and the processor includes a decision tree network. network) based classifier by inputting one target gesture sample, identifying first leaves selected for each tree corresponding to one target gesture sample, and increasing the reward value of each of the identified first leaves by an update unit value Updating the classifier can be performed.
  • the processor inputs one false alarm data sample to the classifier, identifies second leaves selected for each tree corresponding to one false alarm data sample, and reduces a compensation value of each of the identified second leaves by an update unit value.
  • a downward update of the classifier can be performed.
  • the processor extracts data samples that do not represent the target gesture from among data samples included in the training data set used for training the classifier, and from among the extracted data samples, data samples classified as target gestures when input to the classifier can be used as false alarm data samples.
  • the processor may update the classifier by alternately performing upward updating of the classifier and downward updating of the classifier once.
  • the processor may adjust the update unit value to be in inverse proportion to the target gesture recognition rate of the classifier.
  • the processor may decrease the compensation value of each of all leaves of the classifier by the compensation adjustment value when the classifier is updated once and the classifier is updated once.
  • the processor may adjust the compensation adjustment value in proportion to the target gesture recognition rate of the classifier.
  • the processor When the processor performs upward updating of the classifier and downward updating of the classifier once, the processor identifies leaves corresponding to each of all false alarm data samples, and for each of the identified leaves, the corresponding leaf corresponds to all false alarm data samples. A statistical value indicating the number of times selected is calculated, and the compensation value of the corresponding leaf can be further reduced to a value proportional to the statistical value calculated for the corresponding leaf.
  • the processor collects at least one target gesture sample having a target gesture probability greater than or equal to a threshold probability by cropping a signal of a sensor generated by recognizing a user's target gesture motion, and using the collected at least one target gesture sample You can perform an update of the classifier.
  • the processor advances a first timer for a first time to select data samples having a target gesture probability greater than or equal to a threshold probability among data samples acquired during the duration of the first timer. and a data sample having the highest target gesture probability among the extracted data samples may be collected as the target gesture sample.
  • the processor crops a signal of a sensor to extract candidate data samples having a target gesture probability greater than or equal to a threshold probability, and sets a second timer at a start point of the extracted candidate data samples. , and data samples having the highest target gesture probability among data samples obtained during the duration of the second timer may be collected as target gesture samples.
  • the processor may adjust the threshold probability in proportion to the number of target gesture samples used for updating the classifier at the time of updating the classifier.
  • a method performed by an electronic device includes an operation of inputting one target gesture sample to a classifier based on a decision tree network, and a first leaf selected for each tree corresponding to one target gesture sample. and performing an upward update of the classifier by increasing a compensation value of each of the identified first leaves by an update unit value.
  • a method performed by an electronic device includes an operation of inputting one false alarm data sample to a classifier, an operation of identifying second leaves selected for each tree corresponding to one false alarm data sample, and an operation of identifying the second leaves corresponding to the one false alarm data sample.
  • An operation of performing downward update of the classifier by reducing the compensation value of each of the 2 leaves by an update unit value may be included.
  • a method performed by an electronic device includes an operation of extracting data samples that do not represent a target gesture from among data samples included in a training data set used for training a classifier, and extracting data samples from among the extracted data samples.
  • An operation of using data samples classified as target gestures as false alarm data samples when input to the classifier may be included.
  • the method performed by the electronic device may include updating the classifier by alternately performing upward updating of the classifier and downward updating of the classifier once each.
  • the method performed by the electronic device may further include an operation of adjusting an update unit value to be in inverse proportion to a target gesture recognition rate of the classifier when upwardly updating the classifier.
  • a method performed by an electronic device may include an operation of reducing a compensation value of each of all leaves of the classifier by a compensation adjustment value when up-updating the classifier and down-updating the classifier are performed once.
  • a method performed by an electronic device includes an operation of collecting at least one target gesture sample having a target gesture probability greater than or equal to a threshold probability by cropping a signal of a sensor generated by recognizing a user's target gesture operation. , and an operation of performing an update of the classifier using the collected at least one target gesture sample.
  • Collecting at least one target gesture sample may include, when the electronic device collects the target gesture sample in a passive manner, advancing a first timer for a first time, among data samples obtained during the advancing period of the first timer. An operation of extracting data samples having a gesture probability greater than or equal to a threshold probability and collecting data samples having the highest target gesture probability among the extracted data samples as target gesture samples.
  • Collecting at least one target gesture sample includes, when the electronic device collects the target gesture sample in an active way, cropping a sensor signal to extract a candidate data sample having a target gesture probability greater than or equal to a threshold probability, and a second timer. It may include an operation of proceeding for a second time from the starting point of the extracted candidate data sample to collect a data sample having the highest target gesture probability among data samples obtained during the duration of the second timer as the target gesture sample. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Human Computer Interaction (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Quality & Reliability (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Un dispositif électronique selon un mode de réalisation comprend : une mémoire pour stocker des instructions exécutables par ordinateur ; et un processeur pour exécuter les instructions en accédant à la mémoire, le processeur pouvant entrer un échantillon de geste cible dans un classificateur basé sur un réseau d'arbre de décision, identifier des premières feuilles correspondant à un échantillon de geste cible de façon à être sélectionnées par arbre, et augmenter, jusqu'à une valeur d'unité de mise à jour, la valeur de compensation de chacune des premières feuilles identifiées de façon à effectuer une mise à jour de liaison montante du classificateur.
PCT/KR2022/014556 2021-11-25 2022-09-28 Procédé d'apprentissage pour améliorer les performances de reconnaissance de geste dans un dispositif électronique WO2023096134A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2021-0164379 2021-11-25
KR20210164379 2021-11-25
KR1020210188584A KR20230077575A (ko) 2021-11-25 2021-12-27 전자 장치 내에서 제스처 인식 성능 개선을 위한 러닝 방법
KR10-2021-0188584 2021-12-27

Publications (1)

Publication Number Publication Date
WO2023096134A1 true WO2023096134A1 (fr) 2023-06-01

Family

ID=86539758

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/014556 WO2023096134A1 (fr) 2021-11-25 2022-09-28 Procédé d'apprentissage pour améliorer les performances de reconnaissance de geste dans un dispositif électronique

Country Status (1)

Country Link
WO (1) WO2023096134A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140064693A (ko) * 2012-11-20 2014-05-28 삼성전자주식회사 디바이스의 움직임을 포함하는, 착용식 전자 디바이스로의 사용자 제스처 입력
KR20140134803A (ko) * 2013-05-14 2014-11-25 중앙대학교 산학협력단 다중 클래스 svm과 트리 분류를 이용한 제스처 인식 장치 및 방법
US20160018872A1 (en) * 2014-07-18 2016-01-21 Apple Inc. Raise gesture detection in a device
WO2019133334A1 (fr) * 2017-12-28 2019-07-04 Dropbox, Inc. Gestion efficace de mises à jour de synchronisation client
KR20190094133A (ko) * 2019-04-16 2019-08-12 엘지전자 주식회사 객체를 인식하는 인공 지능 장치 및 그 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140064693A (ko) * 2012-11-20 2014-05-28 삼성전자주식회사 디바이스의 움직임을 포함하는, 착용식 전자 디바이스로의 사용자 제스처 입력
KR20140134803A (ko) * 2013-05-14 2014-11-25 중앙대학교 산학협력단 다중 클래스 svm과 트리 분류를 이용한 제스처 인식 장치 및 방법
US20160018872A1 (en) * 2014-07-18 2016-01-21 Apple Inc. Raise gesture detection in a device
WO2019133334A1 (fr) * 2017-12-28 2019-07-04 Dropbox, Inc. Gestion efficace de mises à jour de synchronisation client
KR20190094133A (ko) * 2019-04-16 2019-08-12 엘지전자 주식회사 객체를 인식하는 인공 지능 장치 및 그 방법

Similar Documents

Publication Publication Date Title
WO2020050595A1 (fr) Serveur pour fournir un service de reconnaissance vocale
WO2020130691A1 (fr) Dispositif électronique et procédé pour fournir des informations sur celui-ci
WO2021230485A1 (fr) Procédé et appareil de fourniture d'image
WO2020085784A1 (fr) Dispositif électronique et système qui fournissent un service sur la base d'une reconnaissance vocale
WO2020130260A1 (fr) Terminal mobile et son procédé de fonctionnement
WO2020130301A1 (fr) Dispositif électronique permettant de suivre l'activité d'un utilisateur et son procédé de fonctionnement
WO2022154379A1 (fr) Dispositif électronique et procédé de réglage de luminosité
WO2022158692A1 (fr) Dispositif électronique permettant d'identifier une force tactile et son procédé de fonctionnement
WO2021118229A1 (fr) Procédé de fourniture d'informations et dispositif électronique prenant en charge ce procédé
WO2023096134A1 (fr) Procédé d'apprentissage pour améliorer les performances de reconnaissance de geste dans un dispositif électronique
WO2023132459A1 (fr) Dispositif électronique permettant d'afficher un objet de réalité augmentée et procédé associé
WO2022203330A1 (fr) Dispositif électronique et son procédé de reconnaissance d'empreinte digitale
WO2020022545A1 (fr) Système de reconnaissance vocale
WO2022025444A1 (fr) Procédé et appareil d'affichage d'écran
WO2024096565A1 (fr) Système et procédé de génération d'une routine d'exercice
WO2024090964A1 (fr) Dispositif électronique servant à commander un dispositif d'affichage et procédé de fonctionnement associé
WO2023158088A1 (fr) Dispositif électronique destiné à fournir des informations concernant le taux d'alcoolémie d'un utilisateur et son procédé de fonctionnement
WO2023063638A1 (fr) Dispositif électronique pour fournir un encadrement et procédé de fonctionnement associé
WO2023128208A1 (fr) Dispositif électronique pouvant être monté sur la tête d'un utilisateur, et procédé pour fournir une fonction à l'aide d'informations biométriques dans le même dispositif électronique
WO2022255785A1 (fr) Procédé de fourniture de données de séance d'entraînement au moyen d'une pluralité de dispositifs électroniques, et dispositif électronique associé
WO2022075603A1 (fr) Dispositif électronique pour identifier des informations relatives à la marche, et procédé de fonctionnement du dispositif électronique pour identifier des informations relatives à la marche
WO2023008743A1 (fr) Dispositif électronique de mesure de la biométrie et son procédé de fonctionnement
WO2022119132A1 (fr) Procédé de détection du comportement de lavage des mains, et dispositif électronique associé
WO2023106591A1 (fr) Dispositif électronique et procédé de fourniture de contenu sur la base de l'émotion d'utilisateur
WO2024075935A1 (fr) Dispositif électronique et procédé de réalisation d'une détection de chute

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22898817

Country of ref document: EP

Kind code of ref document: A1