WO2022099588A1 - 字符输入方法、装置、电子设备及存储介质 - Google Patents

字符输入方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2022099588A1
WO2022099588A1 PCT/CN2020/128586 CN2020128586W WO2022099588A1 WO 2022099588 A1 WO2022099588 A1 WO 2022099588A1 CN 2020128586 W CN2020128586 W CN 2020128586W WO 2022099588 A1 WO2022099588 A1 WO 2022099588A1
Authority
WO
WIPO (PCT)
Prior art keywords
vibration signal
gesture
target
signal
initial
Prior art date
Application number
PCT/CN2020/128586
Other languages
English (en)
French (fr)
Inventor
陈文强
陈林
斯坦科维奇·约翰
Original Assignee
深圳振科智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳振科智能科技有限公司 filed Critical 深圳振科智能科技有限公司
Priority to PCT/CN2020/128586 priority Critical patent/WO2022099588A1/zh
Publication of WO2022099588A1 publication Critical patent/WO2022099588A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing

Definitions

  • the present application relates to the technical field of human-computer interaction, for example, to a character input method, device, electronic device and storage medium.
  • the input of characters written in the air can be applied to the remote input operation of various smart devices, such as smart glasses, smart TVs, and so on.
  • various smart devices such as smart glasses, smart TVs, and so on.
  • the camera-based volley handwriting cannot achieve precise finger writing recognition, and the method based on gesture movement trajectory tracking also requires the device to move to track, resulting in the user having to make large movements to write characters. Therefore, it is very necessary to realize how to finely realize the volley writing characters.
  • the present application provides a character input method, device, electronic device and storage medium, so as to realize precise volley writing and character input without the need for customized equipment.
  • an embodiment of the present application provides a character input method, which is applied to an electronic device, and the method includes:
  • the gesture type when writing characters in the air is determined, so as to perform a character input operation based on the gesture type.
  • the embodiments of the present application also provide a character input device, which is configured in an electronic device, and the device includes:
  • the initial signal determination module is used to determine the initial gesture vibration signal generated in the process of performing volley writing characters by gesture
  • a target signal extraction module used for extracting a target gesture vibration signal for effectively characterizing the volley writing character information from the initial gesture vibration signal
  • the gesture character input module is configured to determine the gesture type when writing characters in the air by performing vibration feature recognition on the target gesture vibration signal, so as to perform a character input operation based on the gesture type.
  • the embodiments of the present application also provide an electronic device, including:
  • processors one or more processors
  • a storage device for storing one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the character input method as provided in any embodiment of the present application.
  • an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the character input method provided in any embodiment of the present application.
  • the embodiment of the present application provides a character input method.
  • the electronic device can determine the initial gesture vibration signal generated in the process of the user performing volley writing characters through gestures, and use the electronic device to determine the initial gesture vibration signal. From the initial gesture vibration signal, extract the target gesture vibration signal used to effectively represent the information of the characters written in the air, and then determine the gesture type when the characters are written in the air by identifying the vibration characteristics of the target gesture vibration signal, so as to realize the character input based on the gesture type. operate.
  • the user can accurately recognize the character information written by the user's gesture without having to move a large amount to write characters by gesture, simplifying the complexity of remote input and improving the convenience of input, and at the same time, it does not need to customize special equipment to achieve Recognition of fine finger volley writing, reducing equipment development costs and implementation costs, realizing refined finger volley writing, and enhancing human-computer interaction experience.
  • FIG. 2 is a schematic diagram of a gesture vibration signal in the process of writing characters in the air provided in an embodiment of the present application
  • FIG. 4 is a process diagram of detecting a gesture signal provided in an embodiment of the present application.
  • FIG. 5 is a partial schematic diagram of a frame energy-based endpoint detection provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a video feature after digital gravity removal provided in an embodiment of the present application.
  • FIG. 8 is a structural block diagram of a character input device provided in an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 1 is a flowchart of a character input method provided in an embodiment of the present application.
  • the embodiments of the present application are applicable to the case of remote input to the intelligent electronic device.
  • the method can be performed by a character input device, which can be implemented in software and/or hardware, and the device can be configured in an electronic device with a network communication function; for example, it can be an electronic watch, smart glasses, smart TV As well as terminal equipment such as mobile phones, it can also be server equipment.
  • the character input method provided in the embodiment of the present application may include the following steps:
  • FIG. 2 is a schematic diagram of a gesture vibration signal in the process of writing characters in the air according to an embodiment of the present application.
  • the user can write characters in the air through gestures, and a certain vibration signal is usually generated in the process of writing characters, so that the vibration signal generated by the vibration of the written characters can be collected, which is recorded as the initial gesture vibration. Signal.
  • determining the initial gesture vibration signal generated in the process of executing volley writing characters by gestures may include: using a preset vibration signal sensor to collect the initial gesture vibration signals generated in the process of executing volley writing characters by gestures.
  • the vibration signal sensor includes an accelerometer and a gyroscope, and the vibration signal sensor and the electronic device are integrally arranged or separately arranged.
  • the vibration signal sensor can capture the vibration signal generated when the user writes characters in the air through gestures, but generally does not screen and capture the vibration signal to obtain an initial gesture vibration signal.
  • the vibration signal sensor can be directly integrated in terminal devices such as electronic watches and electronic bracelets. Since such devices are usually worn on the wrist and close to the fingers, the corresponding initial gestures can be easily captured when writing characters with the fingers.
  • the vibration signal sensor can also be installed separately from electronic devices such as smart glasses, smart TVs and mobile phones, because the above electronic devices are usually not always worn on the wrist and are far away from the fingers, so they are not directly integrated in the
  • the above electronic device is connected to the electronic device wirelessly, and the vibration signal sensor is separately arranged near the finger, and the gesture vibration signal is collected and wirelessly sent to the electronic device that is set separately from it.
  • the acquired initial gesture vibration signals not only include valid vibration signals caused by writing characters , and also includes interfering vibration signals due to other factors. For this reason, after obtaining the initial gesture vibration signal, it is necessary to extract the vibration signal that can reflect the characters written by the user's gesture, and use the extracted vibration signal as the target gesture vibration signal, so as to avoid using the initial gesture vibration signal to identify the gesture type directly.
  • the interference signal included in the gesture vibration signal causes the wrong gesture type recognition.
  • different characters have different writing trajectories when writing characters, so the vibration characteristics of gesture vibration signals generated by writing different characters are different. Based on this principle, it is possible to identify the specific gesture type when writing characters by identifying the vibration characteristics of each target gesture vibration signal.
  • multiple gestures for writing characters in the air may be acquired in advance to form a gesture set, for example, gestures for writing ten Arabic numerals and twenty-six English letters respectively are acquired to form a number gesture set and an alphabet gesture set.
  • the vibration feature corresponding to the gesture can be associated, so that when the vibration feature of the vibration signal of the target gesture is recognized, the corresponding gesture type associated with the vibration feature can be known, and then the written characters can be known. type.
  • the user can accurately recognize the character information written by the user's gesture without the need for a large movement to write characters by gesture, simplify the complexity of remote input, improve the convenience of input, and also improve the input convenience.
  • FIG. 3 is a flowchart of another character input method provided in an embodiment of the present application.
  • the technical solutions of the embodiments of the present application are described on the basis of the foregoing embodiments, and the embodiments of the present application may be combined with each optional solution in one or more of the foregoing embodiments.
  • the character input method provided in the embodiment of the present application may include the following steps:
  • S310 Determine an initial gesture vibration signal generated during the process of performing volley writing of characters through gestures.
  • S320 Perform endpoint detection according to the signal strength of the initial gesture vibration signal, and determine the target start endpoint and target end point of the gesture vibration signal that can effectively represent the volley-written character information in the initial gesture vibration signal.
  • FIG. 4 is a process diagram of detecting a gesture signal according to an embodiment of the present application.
  • the initial gesture vibration signal includes an effective gesture vibration signal and an invalid interference vibration signal, wherein the invalid vibration signal can include the vibration signal that occurs when the character is being written and the vibration signal that appears before and after writing a character, so it can be By detecting the endpoints of the signal, identify the starting endpoint and the ending endpoint of the gesture vibration signal that can effectively characterize the volley-written character information in the initial gesture vibration signal, so that the vibration signal between the two ends can be used as a valid signal from the initial gesture vibration signal. It can be cut out as a later recognition object, so as to avoid the vibration signals before and after writing characters being mixed into it, which will affect the recognition process of subsequent vibration characteristics.
  • carry out endpoint detection according to the signal strength of the initial gesture vibration signal, determine the target start endpoint and the target termination endpoint of the gesture vibration signal that can effectively characterize the volley writing character information in the initial gesture vibration signal may include the following steps A1-A3:
  • Step A1 Detect the signal strength of the initial gesture vibration signal according to a preset time interval.
  • Step A2 if it is detected that the signal strength of the initial gesture vibration signal begins to be greater than the first intensity threshold, then the detection point that begins to be greater than the first intensity threshold is used as the target starting endpoint; And, after detecting a target starting endpoint, continue The signal strength of the initial gesture vibration signal is detected.
  • Step A3 If it is detected that the signal strength of the initial gesture vibration signal starts to be less than the second strength threshold during the continuous detection process, the detection point when it starts to be less than the second strength threshold is used as the target termination endpoint.
  • the values of the first intensity threshold and the second intensity threshold are the same or the difference between the thresholds is within a preset range.
  • Endpoint detection can be performed on the initial gesture vibration signal based on a set signal strength threshold, for example, a first strength threshold for detecting the start endpoint of the gesture vibration signal and a second strength threshold for detecting the end point of the gesture vibration signal are set. .
  • the value of the signal strength of the initial gesture vibration signal can be detected according to the preset time interval, and when the signal strength begins to be greater than the first strength threshold during the detection process, the detection point is considered to be an effective start point of the gesture vibration signal; After detecting the starting endpoint of the gesture vibration signal, continue to detect the initial gesture vibration signal until the detected signal strength is lower than the second strength threshold, and the detection point is considered to be the valid endpoint of the gesture vibration signal.
  • the gesture vibration signal from the point to the termination endpoint is the required vibration signal.
  • two first intensity thresholds one large and one small, can be set for the detection of the starting endpoint.
  • the initial gesture vibration signal must first pass the detection of the smaller first intensity threshold. After passing the detection of the smaller first intensity threshold, the initial gesture vibration signal needs to be detected using the larger first intensity threshold.
  • the initial gesture vibration is detected Only when the signal strength of the signal is greater than the smaller first strength threshold and the larger first strength threshold successively is determined as the starting end point of the detected signal.
  • the use of double thresholds can remove the influence of outliers, and since two thresholds, one large and one small, can be used, the signal can be detected more accurately to the starting end point.
  • carry out endpoint detection according to the signal strength of the initial gesture vibration signal, and determine the target start endpoint and target termination endpoint of the gesture vibration signal that can effectively characterize the volley writing character information in the initial gesture vibration signal may include the following steps B1-B3:
  • Step B1 Extract a gesture vibration signal with a preset frame length from the initial gesture vibration signal according to a preset frame shift step length, and calculate the signal energy of the gesture vibration signal with the preset frame length.
  • Step B2 if detecting that the signal energy of the extracted preset frame length gesture vibration signal begins to be greater than the first energy threshold, then the extraction point when starting to be greater than the first energy threshold is used as the target starting endpoint; And, when a target is detected After starting the endpoint, continue to extract the gesture vibration signal of the preset frame length and calculate the signal energy value.
  • Step B3 If it is detected that the signal energy of the extracted preset frame length gesture vibration signal starts to be smaller than the second energy threshold during the continuous extraction process, the extraction point when it starts to be smaller than the second energy threshold is used as the target termination endpoint.
  • the values of the first energy threshold and the second energy threshold are the same or the difference between the thresholds is within a preset range.
  • the threshold-based endpoint detection scheme in order to avoid a sudden signal strength at a certain moment higher than a given strength threshold due to the error of the vibration sensor or the influence of the environment, it is mistakenly identified as the starting endpoint of the gesture vibration signal. Therefore, in order to detect the effective gesture vibration signal when the user writes characters as accurately as possible, it is necessary to reduce the amount of calculation, and no longer detect the signal strength of a single time point, but to detect the data of a time period (for example, a frame of data) to calculate the energy in the frame, and perform endpoint detection through the signal energy of a period of time. By considering the signal of a time segment, the impact of outliers on endpoint detection can also be avoided.
  • a first energy threshold for detecting the start endpoint of the gesture vibration signal and a second energy threshold for detecting the end point of the gesture vibration signal are set.
  • a gesture vibration signal of a preset frame length can be sequentially extracted from the initial gesture vibration signal according to a preset frame shift step, and the signal energy of the gesture vibration signal of the preset frame length can be calculated.
  • the detection point when the frame energy begins to be greater than the first energy threshold, the detection point is considered to be the starting point of the effective gesture vibration signal; after detecting the start point of the gesture vibration signal, continue to detect the initial gesture vibration signal , until it is detected that the frame energy is lower than the second energy threshold, the detection point is considered to be the termination endpoint of an effective gesture vibration signal, and the gesture vibration signal from the start endpoint to the termination endpoint is the required vibration signal.
  • FIG. 5 is a partial schematic diagram of a frame energy-based endpoint detection provided in an embodiment of the present application.
  • the frame length set here is 0.2s, and the frame Shift to 0.01s.
  • the threshold is set to 0.03
  • the current position is e (signal end position).
  • FIG. 5 shows the actual collection of ten digital gesture signal segments from 0 to 9. It can be seen that the system can accurately segment the gesture signal.
  • the frame energy of the signal after calculating the frame energy of the signal, it can not only have a better suppression effect on the noise signal, but also have a relatively obvious amplification effect on the effective signal, and can also realize the frame in a constant number of operations on a real-time system.
  • the calculation of energy avoids the performance burden caused by the excessive amount of calculation.
  • intercepting the effective target gesture vibration signal from the initial gesture vibration signal according to the target start end point and the target end point may include the following operations:
  • the noise signal is usually short, such as when a finger moves occasionally, or in most cases is long, while the length of the gesture signal is usually within a time frame.
  • a length requirement is added to the valid gesture signal, and the signal length l needs to satisfy 0.6s ⁇ l ⁇ 2.0s to be considered as a valid gesture vibration signal, which can reduce the probability that the noise signal is mistakenly identified as a gesture vibration signal.
  • intercepting the effective target gesture vibration signal from the initial gesture vibration signal can include the following operations:
  • the gesture vibration signal from the preset frame length before the target start endpoint to the preset length after the target termination endpoint is determined as the target gesture vibration signal.
  • the target gesture vibration signal after judging the validity of the gesture vibration signal, can be extracted according to the target starting endpoint and the target ending endpoint obtained by endpoint detection.
  • the preprocessing of the signal in the subsequent feature extraction step requires that a section of buffer data be reserved before and after the signal, in addition to the signal detected at the interception endpoint in the signal interception step, it is necessary to intercept an extra section before the signal and after the signal, here Set the buffer length to 0.2s.
  • the length of the gesture signal is fixed to 2.0s.
  • the fixed-length interception method is to move forward 0.2 s from the starting point position as the feature extraction signal preprocessing buffer data according to the starting point position of the signal position, and then intercept the signal with a frame length of 2.0 s backward from this position.
  • the fused initial gesture vibration signal is obtained by fusing the six-axis initial gesture vibration signals of the three-axis accelerometer and the three-axis gyroscope in the vibration sensor.
  • the effective target gesture vibration signal is intercepted from the initial gesture vibration signal generated by the user through the process of writing characters through gestures, so as to avoid the subsequent vibration feature recognition results of the invalid vibration signal image, thereby causing wrong Character input, and the user does not need to move a lot to write characters by gestures, the character information written by the user's gestures can be accurately recognized, the complexity of remote input is simplified, and the input convenience is improved, and there is no need to customize special equipment to realize recognition. Fine finger volley writing reduces equipment development costs and implementation costs, realizes refined finger volley writing, and enhances the human-computer interaction experience.
  • the initial gesture vibration signal is filtered to obtain the filtered initial gesture vibration signal.
  • the high-pass filtering can remove the gravity component contained in the collected accelerometer signal acquisition, reducing the influence of gravity on the signal detection; and reducing the influence of noise (such as the user's arm shaking slightly during the signal acquisition process).
  • high-pass 5HZ processing can be selected for the initial gesture vibration signal, and the filter selection uses Butter-Worth as a prototype.
  • FIG. 6 is a flowchart of another character input method provided in an embodiment of the present application.
  • the technical solutions of the embodiments of the present application are described on the basis of the foregoing embodiments, and the embodiments of the present application may be combined with each optional solution in one or more of the foregoing embodiments.
  • the character input method provided in the embodiment of the present application may include the following steps:
  • S610 Determine an initial gesture vibration signal generated during the process of performing volley writing of characters through gestures.
  • FIG. 7 is a schematic diagram of a video feature after digital gravity removal provided in the embodiment of the present application.
  • the target gesture vibration signal can be intercepted from the original initial gesture vibration signal, and then time domain features and frequency domain features can be extracted from it; The signal is filtered to obtain the filtered initial gesture vibration signal, and then the time domain feature and frequency domain feature are extracted from it.
  • the hand gesture signals are mainly distributed below 25 Hz, and also distributed below 5 Hz, so The low-frequency information needs to be preserved, and the change of gravity is also preserved without filtering. This part of the information is also helpful for distinguishing gestures: the transformation of the gravity of the watch under different gestures is also inconsistent. Considering that different gestures also have differences in frequency domain, frequency domain information is added to the features.
  • the frequency domain information of the target gesture vibration signal can be represented by the following Fourier transform formula (1).
  • the frequency domain information can be extracted from the time domain information by an FFT (Fast Fourier Transform) algorithm, and the finally selected vibration signal fusion features include both time domain feature information and frequency domain features information.
  • FFT Fast Fourier Transform
  • the endpoint detection of different gesture vibration signals will inevitably have a certain offset, and the classifier uses time domain information as a feature, even a slight offset between different signals is inevitable. It will increase the classification difficulty of the classifier and reduce the classification accuracy of the classifier. It is not difficult to know in the calculation of distance, even if there are two identical signals, if one of the signals is offset and then the Euclidean distance or Manhattan distance is calculated for the two signals, a relatively large value will be calculated, even if the two signals are The signal is the same signal with different time delays. Therefore, the offset between the signals will undoubtedly have a certain impact on the classification algorithm based on the distance information between the signals. In order to solve this problem, GCC (Generalized Cross-Correlation, generalized cross-correlation) can be used to calculate the time delay between the two signals to realize the alignment of the gesture signals.
  • GCC Generalized Cross-Correlation, generalized cross-correlation
  • the method may further include: performing feature normalization processing on the vibration signal fusion feature of the target gesture vibration signal.
  • feature normalization is to normalize the dimensions of features of different dimensions. By normalizing the features, it is possible to eliminate the problems caused by the inconsistency of dimensions between different features in the classification algorithm that uses distance as a measure of similarity. The degree of optimization of different features is different, which is conducive to improving the classification accuracy of the model; at the same time, for neural networks, normalized features can make the local optimal coefficients of different features in the same order of magnitude, which speeds up the use of gradient descent algorithm for target optimization. convergence speed.
  • min-max normalization will normalize the value of a set of data to the range of [0,1], the specific calculation is shown in formula (2), in formula (2), y(t) is the normalized time t data, x(t) is the data at time t.
  • the processing method of z-score is to subtract the mean of a set of data and divide by the standard deviation, so that this set of data is converted into data with a mean of 0 and a variance of 1.
  • both min-max normalization and z-score normalization are essentially scaling and shifting the data, and the formulas for both can be expressed by formula 4.
  • min-max only considers the minimum and maximum values of the data during zooming and panning, and scales the data to a fixed range [0,1].
  • z-score needs to calculate the mean and variance of the data during the zooming and panning process, using all the data, and the scale of the data is not fixed.
  • the data collected by the gesture vibration signal is the acceleration sensor data and the gyroscope sensor data, and the fluctuation of the data value is relatively unfixed, which is not suitable for scaling the data to a fixed range. Therefore, z- The score method is more suitable for normalizing the data.
  • the user can accurately recognize the character information written by the user's gesture without the need for a large movement to write characters by gesture, simplify the complexity of remote input, improve the convenience of input, and also improve the input convenience.
  • FIG. 8 is a structural block diagram of a character input device provided in an embodiment of the present application.
  • the embodiments of the present application are applicable to the case of remote input to the intelligent electronic device.
  • the apparatus can be implemented in software and/or hardware, and the apparatus can be configured in an electronic device with a network communication function; for example, it can be terminal devices such as electronic watches, smart glasses, smart TVs, and mobile phones.
  • the character input device provided in this embodiment of the present application may include the following: an initial signal determination module 810 , a target signal extraction module 820 , and a gesture character input module 830 . in:
  • the initial signal determination module 810 is used to determine the initial gesture vibration signal generated in the process of performing volley writing characters through gestures; the target signal extraction module 820 is used to extract information for effectively characterizing volley writing characters from the initial gesture vibration signal The gesture character input module 830 is used to identify the gesture type when writing characters in the air by performing vibration feature recognition on the target gesture vibration signal, so as to perform character input operation based on the gesture type.
  • the initial signal determination module 810 includes:
  • a preset vibration signal sensor is used to collect the initial gesture vibration signal generated during the process of writing characters in the air through gestures; wherein, the vibration signal sensor includes an accelerometer and a gyroscope, and the vibration signal sensor is integrated with the electronic device Configurable settings or separate settings.
  • the target signal extraction module 820 includes:
  • endpoint detection is performed according to the signal strength of the initial gesture vibration signal to determine the target starting endpoint of the gesture vibration signal that can effectively characterize the volley-written character information in the initial gesture vibration signal.
  • target termination endpoints including:
  • the signal strength of the initial gesture vibration signal is detected; if it is detected that the signal strength of the initial gesture vibration signal starts to be greater than the first strength threshold, the detection point that starts to be greater than the first strength threshold is used as the target. Start point; And, after detecting a target start point, continue to detect the signal strength of the initial gesture vibration signal; If the signal strength of the initial gesture vibration signal is detected to be smaller than the second If the intensity threshold is lower than the second intensity threshold, the detection point is initially smaller than the second intensity threshold as the target termination endpoint; wherein the first intensity threshold and the second intensity threshold have the same value or the threshold difference is within a preset range.
  • endpoint detection is performed according to the signal strength of the initial gesture vibration signal to determine the target starting endpoint of the gesture vibration signal that can effectively characterize the volley-written character information in the initial gesture vibration signal.
  • target termination endpoints including:
  • the gesture vibration signal of the preset frame length is extracted from the initial gesture vibration signal, and the signal energy of the gesture vibration signal of the preset frame length is calculated; if the extracted gesture vibration signal of the preset frame length is detected The signal energy of the signal begins to be greater than the first energy threshold, then the extraction point when it begins to be greater than the first energy threshold is used as the target starting endpoint; And, after detecting a target starting endpoint, continue to extract the preset frame length gesture vibration signal And calculate the signal energy value; if the signal energy of the extracted preset frame length gesture vibration signal is detected to be smaller than the second energy threshold in the continuous extraction process, then the extraction point when it is smaller than the second energy threshold is used as the target termination endpoint; The values of the first energy threshold and the second energy threshold are the same or the threshold difference is within a preset range.
  • intercepting a valid target gesture vibration signal from the initial gesture vibration signal including:
  • the target start end point and the target end end point If it is determined according to the target start end point and the target end end point that the length between the end points between the two end points is within the preset end point length threshold range, then according to the target start end point and the target end point, from the initial gesture
  • the effective target gesture vibration signal is intercepted from the vibration signal.
  • intercepting a valid target gesture vibration signal from the initial gesture vibration signal including:
  • the method further includes:
  • the initial gesture vibration signal is filtered to obtain the filtered initial gesture vibration signal.
  • the method further includes:
  • the fused initial gesture vibration signal is obtained by fusing the six-axis initial gesture vibration signals of the three-axis accelerometer and the three-axis gyroscope in the vibration sensor.
  • the gesture character input module 830 includes:
  • the method before inputting the vibration signal fusion feature of the target gesture vibration signal into the pre-trained gesture classification model, the method further includes:
  • Feature normalization processing is performed on the vibration signal fusion feature of the target gesture vibration signal.
  • the character input device provided in the embodiment of the present application can execute the character input method provided in any of the above embodiments of the present application, and has the corresponding functions and beneficial effects of executing the character input method, and the technical details not described in detail in the above embodiments , please refer to the character input method provided in any embodiment of this application.
  • FIG. 9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • the electronic device provided in this embodiment of the present application includes: one or more processors 910 and a storage device 920 ; the number of processors 910 in the electronic device may be one or more.
  • the processor 910 is taken as an example; the storage device 920 is used to store one or more programs; the one or more programs are executed by the one or more processors 910, so that the one or more processors 910 implement the The character input method described in any one of the application embodiments.
  • the electronic device may further include: an input device 930 and an output device 940 .
  • the processor 910 , the storage device 920 , the input device 930 and the output device 940 in the electronic device may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 9 .
  • the storage device 920 in the electronic device can be used to store one or more programs, and the programs can be software programs, computer-executable programs, and modules, as provided in the embodiments of the present application.
  • the program instruction/module corresponding to the character input method.
  • the processor 910 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the storage device 920, ie, implements the application control method in the above method embodiments.
  • the storage device 920 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the electronic device, and the like. Additionally, storage device 920 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, storage device 920 may further include memory located remotely from processor 910, which may be connected to the device through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the input device 930 may be used to receive input numerical or character information, and generate key signal input related to user setting and function control of the electronic device.
  • the output device 940 may include a display device such as a display screen.
  • An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, is used to execute a character input method, and the method includes:
  • the program when executed by the processor, it can also be used to execute the character input method provided in any embodiment of the present application.
  • the computer storage medium of the embodiments of the present application may adopt any combination of one or more computer-readable media.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above.
  • Computer readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (Read Only Memory, ROM), Erasable Programmable Read Only Memory (EPROM), flash memory, optical fiber, portable CD-ROM, optical storage device, magnetic storage device, or any suitable combination of the above .
  • a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • suitable medium including but not limited to: wireless, wire, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • Computer program code for carrying out the operations of the present application may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider to via Internet connection).
  • LAN local area network
  • WAN wide area network

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本文公开了一种字符输入方法、装置、电子设备及存储介质。所述方法包括：确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。

Description

字符输入方法、装置、电子设备及存储介质 技术领域
本申请涉及人机交互技术领域，例如涉及一种字符输入方法、装置、电子设备及存储介质。
背景技术
随着技术的不断发展，手凌空书写字符进行输入可以被应用于各种智能设备的远程输入操作，例如智能眼镜、智能电视等。但是，基于摄像头的凌空手写无法做到精细地手指写字识别，而基于手势移动轨迹追踪的方式也要求设备进行移动才能进行追踪，导致用户必须大动作书写字符。因此，如何精细化的实现凌空书写字符是十分有必要的。
发明内容
本申请提供了一种字符输入方法、装置、电子设备及存储介质，以实现不需要定制化设备就可精细化进行凌空写字，进行字符输入。
第一方面，本申请实施例中提供了一种字符输入方法，应用于电子设备，所述方法包括：
确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；
从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；
通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
第二方面，本申请实施例中还提供了一种字符输入装置，配置于电子设备，所述装置包括：
初始信号确定模块，用于确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；
目标信号提取模块，用于从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；
手势字符输入模块，用于通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
第三方面，本申请实施例中还提供了一种电子设备，包括：
一个或多个处理器；
存储装置，用于存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本申请任意实施例中提供的字符输入方法。
第四方面，本申请实施例中还提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如本申请任意实施例中提供的字符输入方法。
本申请实施例中提供了一种字符输入方法，在用户需要针对各种智能电子设备进行远程输入时，可以通过电子设备确定用户通过手势执行凌空书写字符过程中产生的初始手势振动信号，并从初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号，进而通过识别目标手势振动信号的振动特征，来确定凌空书写字符时的手势类型，实现基于手势类型执行字符输入的操作。采用本申请方案，用户不需要大幅度移动进行手势书写字符，就能准确地识别用户手势书写的字符信息，简化远程输入的复杂度，提高其输入便捷性，同时也不需要定制特制设备来实现识别精细的手指凌空写字，降低设备研制成本和实现成本，实现精细化地手指凌空写字，增强人机交互体验。
附图说明
图1是本申请实施例中提供的一种字符输入方法的流程图；
图2是本申请实施例中提供的一种凌空书写字符过程中的手势震动信号的示意图；
图3是本申请实施例中提供的另一种字符输入方法的流程图；
图4是本申请实施例中提供一种对手势信号进行检测的过程图；
图5是本申请实施例中提供的一种基于帧能量的端点检测的局部示意图；
图6是本申请实施例中提供的又一种字符输入方法的流程图；
图7是本申请实施例中提供的一种数字去除重力后的视频特征的示意图；
图8是本申请实施例中提供的一种字符输入装置的结构框图；
图9是本申请实施例中提供的一种电子设备的结构示意图。
具体实施方式
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅用于解释本公开，而非对本申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
在更加详细地讨论示例性实施例之前，应当提到的是，一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各项操作(或步骤)描述成顺序的处理，但是其中的许多操作(或步骤)可以被并行地、并发地或者同时实施。此外，各项操作的顺序可以被重新安排。当其操作完成时所述处理可以被终止，但是还可以具有未包括在附图中的附加步骤。所述处理可以对应于方法、函数、规程、子例程、子程序等等。
图1是本申请实施例中提供的一种字符输入方法的流程图。本申请实施例可适用于对智能电子设备进行远程输入的情况。该方法可以由字符输入装置来执行，该装置可以采用软件和/或硬件的方式来实现，该装置可以配置于具有网络通信功能的电子设备中；例如，可为电子手表、智能眼镜、智能电视以及手机等终端设备，也可为服务器设备。如图1所示，本申请实施例中提供的字符输入方法，可包括以下步骤：
S110、确定通过手势执行凌空书写字符过程中产生的初始手势振动信号。
在本实施例中，用户通过手势凌空写字可被应用于各种智能电子设备，但是基于手势的凌空写字均是通过对手势进行追踪，来实现书写字符的字符识别，这就造成各种局限，比如，需要用户大幅度移动来进行书写，否则无法精确地追踪到用户的手势路径痕迹。图2是本申请实施例中提供的一种凌空书写字符过程中的手势震动信号的示意图。参见图2，用户可以通过手势在空中进行书写字符，而在书写字符过程中通常会产生一定的振动信号，这样就可以采集到由于书写字符的振动所产生的振动信号，这里记为初始手势振动信号。
在本实施例的一种可选方案中，可以与上述一个或者多个实施例中各个可选方案结合。其中，确定通过手势执行凌空书写字符过程中产生的初始手势振动信号，可包括：采用预设的振动信号传感器，采集通过手势执行凌空书写字符过程中产生的初始手势振动信号。
其中，振动信号传感器包括加速度计和陀螺仪，且振动信号传感器与电子设备一体化设置或者分离设置。
在本实施例中，振动信号传感器可以捕捉用户通过手势凌空书写字符时所产生的振动信号，但通常不会对振动信号进行筛选捕捉，得到一个初始手势振动信号。同时，振动信号传感器可以直接集成在电子手表、电子手环等终端设备中，由于这类设备通常佩戴在手腕处距离手指比较近，因此在通过手指书写 字符时能够很容易捕捉到对应的初始手势振动信号；当然，振动信号传感器还可与智能眼镜、智能电视以及手机等电子设备分离设置，因为上述电子设备通常不会一直佩戴在手腕处，距离手指较远，因此其并不会直接集成在上述电子设备内部，而是通过无线方式与电子设备连接，单独将振动信号传感器配置在手指附近，采集到手势振动信号后通过无线发送给与其分离设置的电子设备。
S120、从初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号。
在本实施例中，在通过手势执行凌空书写字符过程中，不仅书写字符会引起振动，同样其他操作也会引起振动，因此获取的初始手势振动信号中不仅包括由于书写字符而引起的有效振动信号，同时也会包括由于其他因素产生的干扰型振动信号。为此，在得到初始手势振动信号后，需要从中提取能够反映用户手势书写字符的振动信号，并将提取的振动信号作为目标手势振动信号，避免直接使用初始手势振动信号识别手势类型，导致由于初始手势振动信号中包括的干扰信号，而造成手势类型识别错误。
S130、通过对目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
在本实施例中，在书写字符时不同的字符具有不同的书写轨迹，因此书写不同字符产生的手势振动信号的振动特征有所差异。基于这个原理，可以通过识别各个目标手势振动信号的振动特征，来识别书写字符时的具体手势类型是哪个。可选地，可预先获取多个凌空书写字符的手势组成手势集，例如获取分别书写十个阿拉伯数字和二十六个英文字母的手势，组成数字手势集和字母手势集。对于手势集中的每一个手势，均可关联与该手势对应的振动特征，这样就可以在识别到目标手势振动信号的振动特征，就可获知对应振动特征关联匹配的手势类型，进而获知书写的字符类型。
根据本申请实施例中提供的字符输入方法，用户不需要大幅度移动进行手势书写字符，就能准确地识别用户手势书写的字符信息，简化远程输入的复杂度，提高其输入便捷性，同时也不需要定制特制设备来实现识别精细的手指凌空写字，降低设备研制成本和实现成本，实现精细化地手指凌空写字，增强人机交互体验。
图3是本申请实施例中提供的另一种字符输入方法的流程图。本申请实施例的技术方案在上述实施例的基础上进行说明，本申请实施例可以与上述一个或者多个实施例中各个可选方案结合。如图3所示，本申请实施例中提供的字符输入方法，可包括以下步骤：
S310、确定通过手势执行凌空书写字符过程中产生的初始手势振动信号。
S320、依据初始手势振动信号的信号强度进行端点检测，确定初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点。
在本实施例中，图4是本申请实施例中提供一种对手势信号进行检测的过程图。参见图4，初始手势振动信号中包括有效的手势振动信号和无效的干扰振动信号，其中无效振动信号可以包括在正在书写字符时所出现的振动信号和书写一个字符前后出现的振动信号，因此可以通过对信号进行端点检测，识别出初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的起始端点和终止端点，以便将两端点之间的振动信号作为有效信号从初始手势振动信号中截取出来作为后面的识别对象，避免书写字符前后的振动信号掺杂在其中，影响后续振动特征的识别过程。
在本实施例的一种可选方案中，可以与上述一个或者多个实施例中各个可选方案结合。其中，依据初始手势振动信号的信号强度进行端点检测，确定初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点，可包括以下步骤A1-A3：
步骤A1、按照预设时间间隔，对初始手势振动信号的信号强度进行检测。
步骤A2、若检测到初始手势振动信号的信号强度开始大于第一强度阈值，则将开始大于第一强度阈值的检测点作为目标起始端点；以及，在检测到一个目标起始端点后，继续对初始手势振动信号的信号强度进行检测。
步骤A3、若在继续检测过程中检测到初始手势振动信号的信号强度开始小于第二强度阈值，则将开始小于第二强度阈值时的检测点作为目标终止端点。
在本实施例中，第一强度阈值与第二强度阈值的取值相同或者阈值差在预设范围之内。可以基于设定的信号强度阈值来对初始手势振动信号进行端点检测，比如，设置一个用于检测手势振动信号起始端点的第一强度阈值和用于检测手势振动信号终止端点的第二强度阈值。这样，就可以按照预设时间间隔检测初始手势振动信号的信号强度取值，在检测过程中当信号强度开始大于第一强度阈值时，认为该检测点是有效的手势振动信号的起始端点；当检测到手势振动信号的起始端点后，继续对初始手势振动信号进行检测，直到检测到信号强度低于第二强度阈值，认为该检测点是有效的手势振动信号的终止端点，从起始端点到终止端点的手势振动信号即为需要的振动信号。
在本实施例中，在起始端点的单阈值端点检测基础上，可以针对起始端点的检测设置一大一小两个第一强度阈值，在对初始手势振动信号进行起始端点 检测时，初始手势振动信号得先通过较小第一强度阈值的检测，在通过较小第一强度阈值的检测后还需要使用较大第一强度阈值对初始手势振动信号进行检测，当检测到初始手势振动信号的信号强度先后大于较小第一强度阈值和较大第一强度阈值才认定为检测到信号的起始端点。上述方案中，使用双阈值能去除异常值影响，并且由于使用了一大一小两个阈值，可以更为精确的检测信号到起始端点。
在本实施例的另一种可选方案中，可以与上述一个或者多个实施例中各个可选方案结合。其中，依据初始手势振动信号的信号强度进行端点检测，确定初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点，可包括以下步骤B1-B3：
步骤B1、按照预设的帧移步长，从初始手势振动信号提取预设帧长度的手势振动信号，并计算预设帧长度手势振动信号的信号能量。
步骤B2、若检测到提取的预设帧长度手势振动信号的信号能量开始大于第一能量阈值，则将开始大于第一能量阈值时的提取点作为目标起始端点；以及，在检测到一个目标起始端点后，继续提取预设帧长度手势振动信号并计算信号能量值。
步骤B3、若在继续提取过程中检测到提取的预设帧长度手势振动信号的信号能量开始小于第二能量阈值，则将开始小于第二能量阈值时的提取点作为目标终止端点。
在本实施例中，第一能量阈值与第二能量阈值的取值相同或者阈值差在预设范围之内。在基于阈值的端点检测方案之上，为了避免由于振动传感器的误差或者环境的影响导致的突然某一时刻的信号强度高于给定强度阈值而误识别为手势振动信号的起始端点。因此，为了尽可能准确的检测到用户书写字符时有效的手势振动信号，同时需要减少运算量，不再对单个时间点的信号强度进行检测，而是对一个时间段的数据(例如一帧的数据)计算帧内能量，通过一段时间的信号能量进行端点检测，通过考虑一个时间片段的信号，也可以避免异常值对端点检测的影响。
在本实施例中，设置一个用于检测手势振动信号起始端点的第一能量阈值和用于检测手势振动信号终止端点的第二能量阈值。这样，可按照预设的帧移步长，依次从初始手势振动信号提取预设帧长度的手势振动信号，并计算预设帧长度手势振动信号的信号能量。在检测过程中当帧能量开始大于第一能量阈值时，认为该检测点是有效的手势振动信号的起始端点；在检测到手势振动信号的起始端点后，继续对初始手势振动信号进行检测，直到检测到帧能量低于第二能量阈值，认为该检测点是有效的手势振动信号的终止端点，从起始端点 到终止端点的手势振动信号即为需要的振动信号。
在本实施例中，图5是本申请实施例中提供的一种基于帧能量的端点检测的局部示意图。参见图5，通过合理的设置帧长度及帧移，在基于帧能量对初始手势振动信号进行端点检测时，可以有效的避免对噪音信号的误检测，例如这里设置的帧长度为0.2s，帧移为0.01s。使用帧能量计算得到的数据，当检测到帧能量大于阈值(根据实际信号情况，这里设置阈值为0.03)时，记录当前位置为b(信号起始位置)，并继续检测帧能量。当帧能量小于阈值时，记录当前位置为e(信号结束位置)。图5示出了实际采集的0-9十个数字手势信号切段，可以看出系统能够准确的对手势信号进行切段。
采用上述可选方式，将信号通过计算帧能量后，不仅对噪音信号能够有更好的抑制效果，对有效信号也有比较明显的放大效果，而且在实时系统上也能够在常数次运算内实现帧能量的计算，避免运算量过多造成性能负担。
S330、依据目标起始端点和目标终止端点，从初始手势振动信号中截取有效的目标手势振动信号。
在本实施例的一种可选方案中，可以与上述一个或者多个实施例中各个可选方案结合。其中，依据目标起始端点和目标终止端点，从初始手势振动信号中截取有效的目标手势振动信号，可包括以下操作：
若依据目标起始端点和目标终止端点，确定两端点之间的端点间长度在预设端点长度阈值范围之内，则按照目标起始端点和目标终止端点，从初始手势振动信号中截取有效的目标手势振动信号。
在本实施例中，在端点检测阶段，对细微的震动也认为是书写字符产生的手势振动信号，这可以有效的避免手势振动信号检测的遗漏，但同时会增加对噪音信号的误检测。因此尽管信号通过了对端点检测，还需要进一步对得到的信号进行有效性判断，降低噪音误检测的概率。通过判断目标起始端点与目标终止端点之间的端点间长度是否在预设端点长度阈值范围之内，来确定目标起始端点与目标终止端点是否有效。例如，根据信号的长度l＝e-b(e为目标终止端点，b为目标起始端点)，判断截取的信号是否为合法手势振动信号。因为噪音信号通常较短，例如当手指偶尔的活动，或者多数情况下会较长，而手势信号的长度通常在一个时间范围内。对有效手势信号增加一个长度要求，需要信号长度l需满足0.6s<l<2.0s，才认为是有效的手势振动信号，这样可以降低噪音信号误识别为手势振动信号的概率。
在本实施例的一种可选方案中，可以与上述一个或者多个实施例中各个可选方案结合。其中，按照目标起始端点和目标终止端点，从初始手势振动信号 中截取有效的目标手势振动信号，可包括以下操作：
将初始手势振动信号中从位于目标起始端点之前的预设帧长度处到位于目标终止端点之后的预设长度处的手势振动信号，确定为目标手势振动信号。
在本实施例中，在通过手势振动信号有效性判断后，根据端点检测得到的目标起始端点和目标终止端点即可提取出目标手势振动信号。但是由于后续的特征提取步骤中对信号的预处理需要信号前后均保留一段缓冲区数据，因此在信号截取这一步在截取端点检测到的信号外，需要对信号前和信号后多截取一段，这里将缓冲区长度设置为0.2s。截取手势信号时将手势信号长度固定为2.0s。固定长度的截取方法为，根据信号位置的起始点位置，从起始点位置向前移动0.2s作为特征提取信号预处理缓冲区数据，然后从该位置向后截取帧长2.0s的信号。
在本实施例的一种可选方案中，可以与上述一个或者多个实施例中各个可选方案结合。其中，在确定初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点之前，还可包括以下操作：
将通过振动传感器中三轴加速度计和三轴陀螺仪的六轴初始手势振动信号进行融合得到融合后的初始手势振动信号。
在本实施例中，参见图5，为了能更灵敏的检测到手势振动信号的细微变化，帧能量计算前需将三轴加速度计和三轴陀螺仪共六轴的数据进行融合，通过将六轴数据能量相加能有效避免对手势振动信号检测的遗漏。
S340、通过对目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
根据本申请实施例中提供的字符输入方法，从用户通过手势书写字符过程产生的初始手势振动信号中截取有效的目标手势振动信号，避免无效振动信号影像后续的振动特征识别结果，从而导致错误的字符输入，且用户不需要大幅度移动进行手势书写字符，就能准确地识别用户手势书写的字符信息，简化远程输入的复杂度，提高其输入便捷性，同时也不需要定制特制设备来实现识别精细的手指凌空写字，降低设备研制成本和实现成本，实现精细化地手指凌空写字，增强人机交互体验。
在上述实施例的基础上，可以与上述一个或者多个实施例中各个可选方案结合。其中，在确定初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点之前，还可包括以下操作：
采用预设的高通滤波器，对初始手势振动信号进行滤波处理，得到滤波处理后的初始手势振动信号。
在本实施例中，在对初始手势振动信号进行端点检测前，需要先对初始手势振动信号进行高通滤波处理。其中，通过高通滤波可以去除由于采集的加速度计信号采集所包含的重力分量，减少重力对信号检测的影响；并且，减少噪音(如用户手臂在信号采集过程中轻微抖动)的影响。可选地，通过分析手势振动信号的频域分布特性，可选择对初始手势振动信号进行高通5HZ处理，滤波器选择使用Butter-Worth作为原型。
图6是本申请实施例中提供的又一种字符输入方法的流程图。本申请实施例的技术方案在上述实施例的基础上进行说明，本申请实施例可以与上述一个或者多个实施例中各个可选方案结合。如图6所示，本申请实施例中提供的字符输入方法，可包括以下步骤：
S610、确定通过手势执行凌空书写字符过程中产生的初始手势振动信号。
S620、从初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号。
S630、提取目标手势振动信号的时域特征和频域特征，得到融合时域特征和频域特征的振动信号融合特征。
在本实施例中，图7是本申请实施例中提供的一种数字去除重力后的视频特征的示意图。参见图7，在最终的时域特征提取是，可以先从原始的初始手势振动信号中截取到目标手势振动信号，进而从中提取时域特征和频域特征；也可先对原始的初始手势振动信号进行滤波处理得到滤波后的初始手势振动信号，进而从中提时域特征和频域特征。
在本实施例中，参见图7，示出了通过对连续采集0-9共十个数字的手势信号进行时频分析可以看到，手势信号主要分布在25Hz以下，且5Hz以下也有分布，因此需要保留低频信息，不滤波则同时也保留了重力的变化情况，这部分信息同样有利于区分手势：不同手势下手表重力的变换同样不一致。考虑到不同手势在频域上也存在差异，因此在特征上加入了频域信息。目标手势振动信号的频域信息可由以下傅里叶变换公式(1)表示。
Figure PCTCN2020128586-appb-000001
在本实施例中，频域信息可以通过FFT(Fast Fourier Transformation，快速傅里叶变换)算法从时域信息中提取，最终选择的振动信号融合特征中同时包含了时域特征信息和频域特征信息。
S640、将目标手势振动信号的振动信号融合特征输入到预训练的手势分类模型中，输出凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
在本实施例中，在信号检测阶段，对不同手势振动信号进行端点检测难免会有一定的偏移，而分类器使用了时域信息作为特征，不同信号间的即使是细微的偏移也必然会增加分类器的分类难度，降低分类器的分类精度。在距离的计算上不难知道，即使是两个一样的信号，将其中一个信号进行偏移后再对这两个信号计算欧式距离或者曼哈顿距离都会计算出一个比较大的值，即使这两个信号是不同时间延迟下的同一个信号。因此，信号间的偏移无疑会对基于信号间的距离信息的分类算法造成一定的影响。为了解决该问题，可利用GCC(Generalized Cross-Correlation，广义互相关)计算出两个信号间时间延迟，实现对手势信号的对齐。
在将目标手势振动信号的振动信号融合特征输入到预先训练的手势分类模型中之前，还可包括：对目标手势振动信号的振动信号融合特征进行特征归一化处理。
在本实施例中，特征归一化是将不同维度特征的量纲归一化，通过归一化特征可以消除不同特征间由于量纲不一致导致的在使用距离作为度量相似性的分类算法上造成对不同特征优化程度不一样，有利于提升模型的分类精度；同时对于神经网络来说，归一化特征能让不同特征的局部最优系数处于同一个数量级，加快了使用梯度下降算法进行目标优化的收敛速度。
在本实施例中，数据归一化算法选择比较多，常用的有1)min-max归一化和z-score零均值标准化。min-max归一化会将一组数据的值归一化到[0,1]范围内，具体计算如公式(2)所示，在公式(2)中y(t)为t时刻标准化后的数据，x(t)为t时刻的数据。z-score的处理方法是将一组数据减去均值，除以标准差，使这组数据转换为均值为0，方差为1的数据，具体计算如公(3)所示，在公式(3)中y(t)为t时刻标准化后的数据，x(t)为t时刻的数据，μ为数据x的均值，σ为数据x的标准差。
Figure PCTCN2020128586-appb-000002
Figure PCTCN2020128586-appb-000003
在本实施例中，min-max归一化和z-score标准化本质上都是对数据进行缩放和平移，二者公式均可使用公式4表示。min-max在缩放和平移过程中只考虑了数据的最小值和最大值，并将数据缩放到固定的范围[0,1]中。相比于min-max，z-score在缩放和平移过程中需要计算数据的均值和方差，使用了所有的数据，在数据缩放后的范围上非固定。
在本实施例中，可选地，手势振动信号采集的数据为加速度传感器数据和陀螺仪传感器数据，数据值的大小波动上相对不固定，不适合将数据缩放到固定范围内，因此选用z-score方法对数据进行标准化更为合适。
根据本申请实施例中提供的字符输入方法，用户不需要大幅度移动进行手势书写字符，就能准确地识别用户手势书写的字符信息，简化远程输入的复杂度，提高其输入便捷性，同时也不需要定制特制设备来实现识别精细的手指凌空写字，降低设备研制成本和实现成本，实现精细化地手指凌空写字，增强人机交互体验。
图8是本申请实施例中提供的一种字符输入装置的结构框图。本申请实施例可适用于对智能电子设备进行远程输入的情况。该装置可以采用软件和/或硬件的方式来实现，该装置可以配置于具有网络通信功能的电子设备中；例如可为电子手表、智能眼镜、智能电视以及手机等终端设备。如图8所示，本申请实施例中提供的字符输入装置，可包括以下：初始信号确定模块810、目标信号提取模块820和手势字符输入模块830。其中：
初始信号确定模块810，用于确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；目标信号提取模块820，用于从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；手势字符输入模块830，用于通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
在上述实施例的基础上，可选地，初始信号确定模块810包括：
采用预设的振动信号传感器，采集通过手势执行凌空书写字符过程中产生的初始手势振动信号；其中，所述振动信号传感器包括加速度计和陀螺仪，且所述振动信号传感器与所述电子设备一体化设置或者分离设置。
在上述实施例的基础上，可选地，目标信号提取模块820包括：
依据所述初始手势振动信号的信号强度进行端点检测，确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点；依据所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号。
在上述实施例的基础上，可选地，依据所述初始手势振动信号的信号强度进行端点检测，确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点，包括：
按照预设时间间隔，对所述初始手势振动信号的信号强度进行检测；若检测到初始手势振动信号的信号强度开始大于第一强度阈值，则将开始大于第一 强度阈值的检测点作为目标起始端点；以及，在检测到一个目标起始端点后，继续对所述初始手势振动信号的信号强度进行检测；若在继续检测过程中检测到所述初始手势振动信号的信号强度开始小于第二强度阈值，则将开始小于第二强度阈值时的检测点作为目标终止端点；其中，所述第一强度阈值与所述第二强度阈值的取值相同或者阈值差在预设范围之内。
在上述实施例的基础上，可选地，依据所述初始手势振动信号的信号强度进行端点检测，确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点，包括：
按照预设的帧移步长，从所述初始手势振动信号提取预设帧长度的手势振动信号，并计算预设帧长度手势振动信号的信号能量；若检测到提取的预设帧长度手势振动信号的信号能量开始大于第一能量阈值，则将开始大于第一能量阈值时的提取点作为目标起始端点；以及，在检测到一个目标起始端点后，继续提取预设帧长度手势振动信号并计算信号能量值；若在继续提取过程中检测到提取的预设帧长度手势振动信号的信号能量开始小于第二能量阈值，则将开始小于第二能量阈值时的提取点作为目标终止端点；其中，所述第一能量阈值与所述第二能量阈值的取值相同或者阈值差在预设范围之内。
在上述实施例的基础上，可选地，依据所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号，包括：
若依据所述目标起始端点和目标终止端点，确定两端点之间的端点间长度在预设端点长度阈值范围之内，则按照所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号。
在上述实施例的基础上，可选地，按照所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号，包括：
将所述初始手势振动信号中从位于所述目标起始端点之前的预设帧长度处到位于目标终止端点之后的预设长度处的手势振动信号，确定为所述目标手势振动信号。
在上述实施例的基础上，可选地，在确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点之前，还包括：
采用预设的高通滤波器，对所述初始手势振动信号进行滤波处理，得到滤波处理后的初始手势振动信号。
在上述实施例的基础上，可选地，在确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点之前， 还包括：
将通过振动传感器中三轴加速度计和三轴陀螺仪的六轴初始手势振动信号进行融合得到融合后的初始手势振动信号。
在上述实施例的基础上，可选地，手势字符输入模块830包括：
提取所述目标手势振动信号的时域特征和频域特征，得到融合时域特征和频域特征的振动信号融合特征；将所述目标手势振动信号的振动信号融合特征输入到预先训练的手势分类模型中，输出凌空书写字符时的手势类型。
在上述实施例的基础上，可选地，在将所述目标手势振动信号的振动信号融合特征输入到预先训练的手势分类模型中之前，还包括：
对所述目标手势振动信号的振动信号融合特征进行特征归一化处理。
本申请实施例中所提供的字符输入装置可执行上述本申请任意实施例中所提供的字符输入方法，具备执行字符输入方法相应的功能和有益效果，未在上述实施例中详尽描述的技术细节，可参见本申请任意实施例中所提供的字符输入方法。
图9是本申请实施例中提供的一种电子设备的结构示意图。如图9所示结构，本申请实施例中提供的电子设备包括：一个或多个处理器910和存储装置920；该电子设备中的处理器910可以是一个或多个，图9中以一个处理器910为例；存储装置920用于存储一个或多个程序；所述一个或多个程序被所述一个或多个处理器910执行，使得所述一个或多个处理器910实现如本申请实施例中任一项所述的字符输入方法。
该电子设备还可以包括：输入装置930和输出装置940。
该电子设备中的处理器910、存储装置920、输入装置930和输出装置940可以通过总线或其他方式连接，图9中以通过总线连接为例。
该电子设备中的存储装置920作为一种计算机可读存储介质，可用于存储一个或多个程序，所述程序可以是软件程序、计算机可执行程序以及模块，如本申请实施例中所提供的字符输入方法对应的程序指令/模块。处理器910通过运行存储在存储装置920中的软件程序、指令以及模块，从而执行电子设备的各种功能应用以及数据处理，即实现上述方法实施例中应用控制方法。
存储装置920可包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据电子设备的使用所创建的数据等。此外，存储装置920可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非 易失性固态存储器件。在一些实例中，存储装置920可进一步包括相对于处理器910远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置930可用于接收输入的数字或字符信息，以及产生与电子设备的用户设置以及功能控制有关的键信号输入。输出装置940可包括显示屏等显示设备。
并且，当上述电子设备所包括一个或者多个程序被所述一个或者多个处理器910执行时，程序进行如下操作：
确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
当然，本领域技术人员可以理解，当上述电子设备所包括一个或者多个程序被所述一个或者多个处理器910执行时，程序还可以进行本申请任意实施例中所提供的字符输入方法中的相关操作。
本申请的一个实施例中提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时用于执行字符输入方法，该方法包括：
确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
可选的，该程序被处理器执行时还可以用于执行本申请任意实施例中所提供的字符输入方法。
本申请实施例的计算机存储介质，可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(Random Access Memory，RAM)、只读存储器(Read Only Memory，ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、闪存、光纤、便携式CD-ROM、光存储器件、磁存储器件、或者上述的任意合适的组合。计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或 者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于：电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、无线电频率(Radio Frequency，RF)等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。

Claims (10)

  1. 一种字符输入方法，应用于电子设备，包括：
    确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；
    从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；
    通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
  2. 根据权利要求1所述的方法，其中，从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号，包括：
    依据所述初始手势振动信号的信号强度进行端点检测，确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点；
    依据所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号。
  3. 根据权利要求2所述的方法，其中，依据所述初始手势振动信号的信号强度进行端点检测，确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点，包括：
    按照预设时间间隔，对所述初始手势振动信号的信号强度进行检测；
    若检测到初始手势振动信号的信号强度开始大于第一强度阈值，则将开始大于第一强度阈值的检测点作为目标起始端点；以及，在检测到一个目标起始端点后，继续对所述初始手势振动信号的信号强度进行检测；
    若在继续检测过程中检测到所述初始手势振动信号的信号强度开始小于第二强度阈值，则将开始小于第二强度阈值时的检测点作为目标终止端点；
    其中，所述第一强度阈值与所述第二强度阈值的取值相同或者阈值差在预设范围之内。
  4. 根据权利要求2所述的方法，其中，依据所述初始手势振动信号的信号强度进行端点检测，确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点，包括：
    按照预设的帧移步长，从所述初始手势振动信号提取预设帧长度的手势振动信号，并计算预设帧长度手势振动信号的信号能量；
    若检测到提取的预设帧长度手势振动信号的信号能量开始大于第一能量阈值，则将开始大于第一能量阈值时的提取点作为目标起始端点；以及，在检测 到一个目标起始端点后，继续提取预设帧长度手势振动信号并计算信号能量值；
    若在继续提取过程中检测到提取的预设帧长度手势振动信号的信号能量开始小于第二能量阈值，则将开始小于第二能量阈值时的提取点作为目标终止端点；
    其中，所述第一能量阈值与所述第二能量阈值的取值相同或者阈值差在预设范围之内。
  5. 根据权利要求2所述的方法，其中，依据所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号，包括：
    若依据所述目标起始端点和目标终止端点，确定两端点之间的端点间长度在预设端点长度阈值范围之内，则按照所述目标起始端点和目标终止端点，从所述初始手势振动信号中截取有效的目标手势振动信号。
  6. 根据权利要求2所述的方法，其中，在确定所述初始手势振动信号中能够有效表征凌空书写字符信息的手势振动信号的目标起始端点和目标终止端点之前，还包括：
    采用预设的高通滤波器，对所述初始手势振动信号进行滤波处理，得到滤波处理后的初始手势振动信号。
  7. 根据权利要求1所述的方法，其中，通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，包括：
    提取所述目标手势振动信号的时域特征和频域特征，得到融合时域特征和频域特征的振动信号融合特征；
    将所述目标手势振动信号的振动信号融合特征输入到预先训练的手势分类模型中，输出凌空书写字符时的手势类型。
  8. 一种字符输入装置，配置于电子设备，包括：
    初始信号确定模块，用于确定通过手势执行凌空书写字符过程中产生的初始手势振动信号；
    目标信号提取模块，用于从所述初始手势振动信号中，提取用于有效表征凌空书写字符信息的目标手势振动信号；
    手势字符输入模块，用于通过对所述目标手势振动信号进行振动特征识别，确定凌空书写字符时的手势类型，以基于手势类型执行字符输入操作。
  9. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现权利要求1-7中任一项所述的字符输入方法。
  10. 一种计算机可读存储介质，存储有计算机程序，其中，该程序被处理器执行时实现权利要求1-7中任一项所述的字符输入方法。
PCT/CN2020/128586 2020-11-13 2020-11-13 字符输入方法、装置、电子设备及存储介质 WO2022099588A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/128586 WO2022099588A1 (zh) 2020-11-13 2020-11-13 字符输入方法、装置、电子设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/128586 WO2022099588A1 (zh) 2020-11-13 2020-11-13 字符输入方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022099588A1 true WO2022099588A1 (zh) 2022-05-19

Family

ID=81601996

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/128586 WO2022099588A1 (zh) 2020-11-13 2020-11-13 字符输入方法、装置、电子设备及存储介质

Country Status (1)

Country Link
WO (1) WO2022099588A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140085311A1 (en) * 2012-09-24 2014-03-27 Co-Operwrite Limited Method and system for providing animated font for character and command input to a computer
CN104866099A (zh) * 2015-05-27 2015-08-26 东南大学 基于运动传感器提高智能设备手势识别精度的误差补偿方法
CN105224066A (zh) * 2014-06-03 2016-01-06 北京创思博德科技有限公司 一种基于云端处理的手势识别方法
CN107316067A (zh) * 2017-05-27 2017-11-03 华南理工大学 一种基于惯性传感器的空中手写字符识别方法
CN107643840A (zh) * 2017-09-21 2018-01-30 上海电机学院 基于惯性器件的手写笔文字自动识别方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140085311A1 (en) * 2012-09-24 2014-03-27 Co-Operwrite Limited Method and system for providing animated font for character and command input to a computer
CN105224066A (zh) * 2014-06-03 2016-01-06 北京创思博德科技有限公司 一种基于云端处理的手势识别方法
CN104866099A (zh) * 2015-05-27 2015-08-26 东南大学 基于运动传感器提高智能设备手势识别精度的误差补偿方法
CN107316067A (zh) * 2017-05-27 2017-11-03 华南理工大学 一种基于惯性传感器的空中手写字符识别方法
CN107643840A (zh) * 2017-09-21 2018-01-30 上海电机学院 基于惯性器件的手写笔文字自动识别方法及系统

Similar Documents

Publication Publication Date Title
EP3523754B1 (en) Face liveness detection method and apparatus, and electronic device
US11762474B2 (en) Systems, methods and devices for gesture recognition
WO2018090538A1 (zh) 网球拍动作识别方法及装置
WO2020078017A1 (zh) 用于识别空中手写的方法、装置、设备以及计算机可读存储介质
US10126825B2 (en) Method for recognizing handwriting on a physical surface
US20140168057A1 (en) Gyro aided tap gesture detection
CN107688790B (zh) 人体行为识别方法、装置、存储介质及电子设备
CN108847941B (zh) 身份认证方法、装置、终端及存储介质
WO2020124994A1 (zh) 活体检测方法、装置、电子设备及存储介质
US20150254575A1 (en) Learn-by-example systems and methos
CN111752388A (zh) 应用控制方法、装置、设备及存储介质
CN109032345A (zh) 设备控制方法、装置、设备、服务端和存储介质
WO2022095674A1 (zh) 用于操作移动设备的方法和装置
KR20160133305A (ko) 제스쳐 인식 방법, 컴퓨팅 장치 및 제어 장치
WO2022001791A1 (zh) 一种基于ppg信息的智能设备交互方法
WO2022099588A1 (zh) 字符输入方法、装置、电子设备及存储介质
CN113342170A (zh) 手势控制方法、装置、终端和存储介质
CN112363622A (zh) 字符输入方法、装置、电子设备及存储介质
CN201903889U (zh) 手指静脉识别终端
CN111766941B (zh) 一种基于智能戒指的手势识别方法及系统
CN112216283B (zh) 一种语音识别方法、装置、设备及存储介质
CN114397964B (zh) 一种有效注视点检测方法、装置、电子设备及存储介质
US20220269958A1 (en) Device-invariant, frequency-domain signal processing with machine learning
CN114253397B (zh) 一种基于耳戴式惯性传感器的智能设备交互系统
WO2022061500A1 (zh) 一种基于电容触摸屏的人机交互系统和交互方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20961145

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.10.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20961145

Country of ref document: EP

Kind code of ref document: A1