WO2020211467A1 - 一种电子设备、控制方法及设备系统 - Google Patents

一种电子设备、控制方法及设备系统 Download PDF

Info

Publication number
WO2020211467A1
WO2020211467A1 PCT/CN2019/130593 CN2019130593W WO2020211467A1 WO 2020211467 A1 WO2020211467 A1 WO 2020211467A1 CN 2019130593 W CN2019130593 W CN 2019130593W WO 2020211467 A1 WO2020211467 A1 WO 2020211467A1
Authority
WO
WIPO (PCT)
Prior art keywords
action
electronic device
information
event information
user
Prior art date
Application number
PCT/CN2019/130593
Other languages
English (en)
French (fr)
Inventor
王劲力
崔磊
张瞳
吴玉锦
周舒然
Original Assignee
出门问问信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN201910319222.2A external-priority patent/CN110110616A/zh
Priority claimed from CN201910391236.5A external-priority patent/CN110245328A/zh
Priority claimed from CN201910391230.8A external-priority patent/CN110262654A/zh
Application filed by 出门问问信息科技有限公司 filed Critical 出门问问信息科技有限公司
Publication of WO2020211467A1 publication Critical patent/WO2020211467A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the embodiments of the present invention relate to the field of data processing technology, and more specifically, to an electronic device, a control method, and a device system.
  • smart electronic devices continue to introduce new ones, bringing more and more traversal to people's lives.
  • smart earphones can remove environmental noise, so that the sound from the speaker of the smart earphone remains authentic.
  • embodiments of the present invention provide an electronic device, a control method, and a device system to expand the functions of the electronic device.
  • an embodiment of the present invention provides an electronic device, and the electronic device includes:
  • a sensor group configured to collect user action parameters, the sensor group including an acceleration sensor;
  • the noise reduction device is configured to perform noise reduction processing on an input signal to obtain data after the noise reduction processing, the input signal is determined according to the action parameter;
  • the processor is configured to extract a state recognition feature vector based on the data after the noise reduction processing, and determine event information based on the state recognition feature vector.
  • the input signal is the action parameter.
  • the event information is a user's action or user's voice information
  • the processor is further configured to determine an operation instruction corresponding to the user's action.
  • the processor is further configured to control the electronic device to perform an operation corresponding to the operation instruction;
  • the processor is configured to control a transmitter of the electronic device to send the operation instruction to a corresponding terminal device, so as to control the terminal device to perform an operation corresponding to the operation instruction.
  • the processor is further configured to periodically count event information corresponding to the user's action in response to the user's action being a motion posture.
  • the transmitter in the electronic device is controlled to synchronize the motion data to the corresponding terminal device.
  • the processor is configured to use a windowed feature extraction method to extract the state recognition feature vector from the noise-reduction processed data.
  • the sensor group further includes a microphone, and the microphone is configured to collect sound signals emitted from the outside;
  • the electronic device further includes:
  • the sound activity detection device is configured to wake up sensors other than the preset sensor in the sensor group when a voice signal transmitted by a preset sensor in the sensor group is received;
  • the acceleration sensor or the microphone is set as a preset sensor.
  • the electronic equipment further includes a sound enhancement processing device, a sound fusion device, and an echo cancellation device;
  • the echo cancellation device is configured to receive a sound signal sent by the microphone, and perform echo cancellation processing on the sound signal;
  • the sound enhancement processing device is configured to perform noise reduction processing on the sound signal after echo cancellation processing to enhance the sound signal;
  • the sound fusion device is configured to use an adaptive filtering method to perform fusion processing on the enhanced sound signal and the action parameters collected by the sensor group to obtain the input signal.
  • the processor is further configured to wake up the electronic device according to the event information and a local hot word database, the local hot word database including at least one hot word, and the at least one hot word including at least one voice Information and/or at least one user action, the electronic device is awakened after recognizing the corresponding hot word.
  • the processor is further configured to determine a corresponding hot word according to the event information and a local hot word database, and perform secondary verification on the corresponding hot word to wake up the electronic device.
  • the processor determines the user action according to the state recognition feature vector, a preset action type, and a corresponding action type feature vector.
  • the processor is configured to monitor the event information according to time information for generating the event information, and generate voice information in response to not generating the event information again within a predetermined time threshold.
  • the processor is further configured to generate voice information in response to the event information being not generated within a predetermined time threshold after the initial action parameters are collected.
  • the processor is further configured to generate voice information in response to the operation instruction being not determined within a predetermined time threshold after the initial action parameter is collected.
  • an embodiment of the present invention provides a method for controlling an electronic device, the method including:
  • the event information is determined according to the state recognition feature vector.
  • the event information is voice information of user actions
  • the method also includes:
  • the method further includes:
  • a transmitter that controls the electronic device sends the operation instruction to a corresponding terminal device, so as to control the terminal device to perform an operation corresponding to the operation instruction.
  • the method further includes:
  • an embodiment of the present invention provides an equipment system, wherein the equipment system includes:
  • At least one electronic device as described in the first aspect of the embodiment of the present invention is provided.
  • each of the electronic devices is set at a different target part, and is configured to determine event information of the corresponding target part;
  • the terminal device is configured to optimize the quantity statistics information of the same type of event information for each target part.
  • the terminal device generates action completion information corresponding to a corresponding comprehensive action type based on at least one event completion information of each target part;
  • the comprehensive action type includes at least one action type corresponding to at least two target parts respectively.
  • the user's action parameters are collected through the sensor group, and after the noise reduction is performed by the noise reduction device, the processor extracts the state recognition feature vector from the data after the noise reduction processing, and determines the corresponding event information according to the state recognition feature vector , After the electronic device recognizes the user’s event information, it executes the corresponding operation function, thus, the function of the electronic device can be expanded, and the communication connection can also be established through other terminal devices, which is inconvenient for the user to directly control other terminal devices Next, collect some control instructions of the user to control other terminal equipment to provide better service to the user.
  • Fig. 1 is a schematic diagram of an electronic device according to a first embodiment of the present invention
  • Fig. 2 is a schematic diagram of an electronic device according to a second embodiment of the present invention.
  • Fig. 3 is a schematic diagram of an electronic device according to a third embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an electronic device according to a fourth embodiment of the present invention.
  • FIG. 5 is a schematic diagram of the equipment system of the fifth embodiment of the present invention.
  • Fig. 6 is a flowchart of a control method of an electronic device according to a sixth embodiment of the present invention.
  • FIG. 7 is a flowchart of an action recognition method of an electronic device according to a seventh embodiment of the present invention.
  • FIG. 8 is a flowchart of an action recognition method of an electronic device according to an eighth embodiment of the present invention.
  • FIG. 9 is a flowchart of an action recognition method of an electronic device according to a ninth embodiment of the present invention.
  • the embodiment of the present invention provides an electronic device.
  • the electronic device of this embodiment is mainly a wearable device, which itself has certain functions.
  • a smart headset has its own function to play music or transmit voice signals when the user is on a call, including the voice signal sent by the user or the voice signal transmitted by the phone peer; in the case of a watch, its own functions include viewing time, calendar, Set alarms, etc.
  • some other practical functions are added to the terminal device, and these functions are used to bring certain convenience to the user.
  • Fig. 1 is a schematic diagram of an electronic device according to a first embodiment of the present invention.
  • the electronic device 1 includes a sensor group 10, a noise reduction device 20 and a processor 30.
  • the sensor group 10 includes at least an acceleration sensor.
  • the sensor group 10 may include a bone conduction sensor or a motion sensor.
  • the bone conduction sensor can detect the motion information of the moving part (for example, the head) according to the vibration frequency of the moving part, or obtain the voice signal according to the vibration frequency of the ear bone, skull, jaw and other parts. It is easy to understand that the bone conduction sensor is essentially an acceleration sensor.
  • the motion sensor is also called IMU (Inertial Measurement Unit).
  • the motion sensor is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object.
  • an IMU includes acceleration sensors and gyroscopes for each axis to detect acceleration information and angular velocity information of an object (for example, an electronic device fixed to a certain part) in a three-dimensional space, so that the movement posture of the object can be obtained.
  • the sensor group 10 is configured to collect user action parameters.
  • the action parameters can include the action parameters generated when a part (such as the head or hands) performs an action (such as nodding, shaking the head, arm movements, etc.), and can also include the ear bones, skull, jaw, etc. when the user speaks.
  • the operation parameters of vibration, etc. are not limited in this embodiment.
  • the noise reduction device 20 is configured to perform noise reduction processing on the input signal and obtain data after the noise reduction processing. Among them, the input signal is determined by the collected action parameters. In an optional implementation manner, the input signal is an action parameter collected by the sensor group.
  • the processor 30 is configured to extract a state recognition feature vector based on the data after the noise reduction process, and determine event information based on the state recognition feature vector.
  • the action parameter is an action parameter generated when a part performs an action
  • determining the event information may be determining the user's action.
  • the motion parameter is an action parameter that generates vibrations such as ear bones, skull, jaw bones, etc. when the user speaks
  • the determined event information may be voice information that determines the user.
  • the user's actions may include head movements and/or hand movements. Among them, the head movement can include nodding, shaking the head, etc., and the hand movement can include arm swinging during walking and running.
  • the processor 30 is further configured to determine an operation instruction corresponding to the above-mentioned event information, and control the corresponding electronic device to perform an operation corresponding to the operation instruction.
  • the event information is a user's action.
  • the determined operation instruction is "Stop playing current music”
  • the processor controls the smart watch to stop playing the current music.
  • the electronic device 1 further includes a transmitter 40.
  • the processor 30 is further configured to determine an operation instruction corresponding to the above event information, and the control transmitter 40 sends the operation instruction to a corresponding terminal device to control the terminal device to perform an operation corresponding to the operation instruction.
  • the event information is a user's action
  • the electronic device 1 is a smart headset as an example
  • the smart headset is connected to the corresponding terminal device via Bluetooth.
  • the processor 30 determines that the user's action is a shaking head
  • the determined operation instruction is "Stop playing current music”
  • the processor 30 controls the transmitter 40 to send the "Stop playing current music” operation instruction to the corresponding terminal via Bluetooth communication.
  • Device to control the terminal device to stop playing the current music according to the operation instruction.
  • the event information is the user's voice information
  • the operation instruction corresponding to the voice information can be determined to control the electronic device or the corresponding terminal device to perform the operation corresponding to the operation instruction.
  • the processor is further configured to wake up the electronic device according to the event information and a local hot word database, the local hot word database including at least one hot word, and the at least one hot word including at least one voice Information and/or at least one user action, the electronic device is awakened after recognizing the corresponding hot word. That is, when the electronic device is in the dormant state, when corresponding voice information or action information is detected, the electronic device is awakened. For example, it is recognized that the input voice is "start" or the electronic device is touched and a user action is formed on the electronic device to wake up the sleeping electronic device.
  • the processor is further configured to determine a corresponding hot word according to the event information and a local hot word database, and perform secondary verification on the corresponding hot word to wake up the electronic device.
  • the first verification of the event information may be preliminary filtering, and the second verification may be accurate verification, so as to avoid the occurrence of similar homophonic words also starting the device, which improves the reliability of the electronic device.
  • the processor 30 is further configured to periodically count the motion data corresponding to the user's action.
  • the motion posture may be a swimming posture
  • the electronic device may include a smart earphone and/or a smart watch, where the smart earphone can detect head movements during swimming, and the smart watch can detect hand movements during swimming.
  • relevant sports information can be obtained, such as the number of strokes of swimming, swimming distance, etc.; or running distance, number of running steps, etc.
  • the processor 30 is further configured to control the transmitter 40 to synchronize the motion data to the corresponding terminal device.
  • data can be synchronized in real time or periodically.
  • the electronic device 1 is not connected to the corresponding terminal device, the next time it is connected, the movement The data is synchronized to the corresponding terminal device, which is not limited in this embodiment.
  • a windowed feature extraction method may be used.
  • the windowed feature extraction is a custom noun, and its actual meaning is: first perform the windowing process on the data after the noise reduction process, and then perform the feature extraction. For example, if the time window is 50 Hz, and the number of points collected by each coordinate axis in the time window is 50 points, then the three axes will collect 50*3 points. These points form a 50*3 point matrix, and Extract the state recognition feature vector from this matrix.
  • the state recognition feature vector may include a series of vectors such as the maximum value, minimum value, variance, or mean value of the data on each coordinate axis. It should be understood that this embodiment does not limit the method for extracting the state recognition feature vector, and other methods such as neural network models can be applied in this embodiment.
  • the obtained state recognition vector is input into a pre-trained state recognition model for classification by the processor 30, so as to predict the aforementioned event information (for example, user action or user voice).
  • the specific process is conventional machine learning technology, so I won't explain it too much here.
  • the processor 30 determines the user action according to the determined state recognition feature vector, the preset action type, and the corresponding action type feature vector. Specifically, the feature vector of the preset action type and the corresponding relationship between the action type and its corresponding feature vector can be obtained in advance, and stored in a predetermined memory. When acquiring the current state recognition feature vector, perform a similarity comparison with feature vectors of feature types in a predetermined memory, and determine the user's action when the corresponding similarity is greater than a predetermined threshold (for example, 80%).
  • a predetermined threshold for example, 80%
  • the bone conduction sensor can collect the user's head motion parameters, that is, the acceleration data when the head is moving. Since the user is moving as a three-dimensional vector, it includes three-axis data such as X, Y and Z. .
  • the noise reduction device 20 respectively performs noise reduction processing on the three-axis data, removes noise such as glitches in the head motion parameters, and obtains data.
  • the processor 30 may extract a state recognition feature vector from the data after the noise reduction process, and obtain event information according to the state recognition feature vector.
  • the operation instruction corresponding to the determined event information is matched from a pre-established database. Among them, the pre-established database includes the corresponding relationship between each event information and operation instructions.
  • the action is shaking the head
  • the operation instruction corresponding to the shaking head is connected to the mobile phone.
  • the electronic device sends the operation instruction to the smart terminal through the transmitter 40, so that the smart terminal hangs up the current call request according to the operation instruction.
  • the recognition action is nodding
  • matching the operation instruction corresponding to the action from the pre-established database is answering the mobile phone call.
  • the electronic device sends a control instruction for answering the mobile phone call to the smart terminal to control the smart terminal to answer the mobile phone call.
  • the processor 30 is also used to match the operation instruction corresponding to the gesture action, and then use the transmitter 40 to send the operation instruction to the smart home, and control the smart home to perform the corresponding operation .
  • the processor 30 can also control the electronic device to execute the operation corresponding to the operation instruction according to the operation instruction. For example, if the electronic device is a smart headset, if you tilt your head to the left, you can operate the smart headset to reduce the volume, or tilt your head to the right to increase the volume of the headset.
  • the motion of nodding and shaking the head can generally be divided into at least two parts. For example, shaking the head can be divided into a leftward offset part and a rightward offset part.
  • the processor 30 is further configured to monitor the event information according to the time information when the event information is generated, and generate voice information in response to not generating the event information again within a predetermined time threshold. Taking head shaking as an example, the processor determines that the event information corresponding to the action parameter collected this time is "shifted to the left by 40°", which is not received within a predetermined time threshold, that is, it has exceeded the predetermined time and has not received the characterization. The event information "shifted to the right" can prompt the user to do the next action through voice to complete the action the user wants to make.
  • the predetermined time threshold may be 10 seconds, 30 seconds, 1 minute, etc., which is not limited in this embodiment.
  • the processor 30 is further configured to generate voice information in response to the event information being not generated within a predetermined time threshold after the initial action parameters are collected. In another optional implementation manner, the processor 30 is further configured to generate voice information in response to the operation instruction being not determined within a predetermined time threshold after the initial action parameters are collected.
  • the initial action parameter is the action parameter of the user putting the smart earphone into the ear. After the user puts the earphone into the ear for a predetermined time threshold, no event information (or operation instruction) is generated. The voice message reminds the user to proceed with the next step.
  • the predetermined time threshold may be 1 minute, 5 minutes, 10 minutes, etc., which is not limited in this embodiment.
  • the electronic device can also implement periodic statistics of user action data and synchronize it to the terminal device.
  • the terminal device can be a smart wearable device or a smart terminal, or even a smart home.
  • the functions performed by the functional modules in the electronic device are as follows:
  • the bone conduction sensor 10 collects the user's action parameters, which are the action parameters corresponding to the motion posture, that is, in this embodiment, the action parameters of different parts can be collected according to different wearing positions of the electronic device.
  • the noise reduction device 20 is used to reduce the noise of the action parameters to obtain the corresponding data.
  • the processor 30 extracts the state recognition feature vector from the data and inputs it into the pre-trained action recognition model to determine the user
  • the action is a sports posture. Sports postures can include running, walking, swimming and so on. It is mainly to periodically count the user’s exercise data, such as the user’s daily running time, calories burned, or the number of swims per week, average swimming speed, average length of a single swim or calories burned, etc.
  • the motion data is synchronized to the terminal device. These motion data can be calculated by the processor according to the action parameters, and there is a relatively mature calculation process in the related technology, so I won't elaborate on it here.
  • the electronic device may also include collecting the user's voice signal and processing the voice signal to obtain a true and clear voice signal and input it into the voice recognition device to realize the voice recognition device. control.
  • the user's action parameters are collected through the sensor group, and after the noise reduction is performed by the noise reduction device, the processor extracts the state recognition feature vector from the data after the noise reduction processing, and determines the corresponding event information according to the state recognition feature vector , So that the electronic device executes the corresponding operation function after recognizing the user's event information.
  • the electronic device itself can be made richer in functions, and it can also establish a communication connection with other terminal devices.
  • some control instructions from the user can be collected to control other terminal devices. To better serve users.
  • Fig. 2 is a schematic diagram of an electronic device according to a second embodiment of the present invention.
  • the operation process of the electronic device when the event information is voice information is mainly introduced.
  • the electronic device 1 may further include: a sound activity detection device 50, echo cancellation devices 60 and 70, a sound enhancement processing device 80, and a sound fusion device 90.
  • the acceleration sensor in the sensor group 10 is a bone conduction sensor 101, and the sensor group 10 further includes a microphone 102 and a microphone 103, where the number of microphones corresponds to the number of echo cancellation devices.
  • This embodiment uses two microphones As an example, it should be understood that this embodiment does not limit this.
  • the sound activity detection device 50 is configured to wake up the sensors in the sensor group 10 other than the preset sound sensor, the sound enhancement processing device 80 and the sound signal when the voice signal transmitted by the preset sensor in the sensor group 10 is received.
  • the fusion device 90 enters the working state from the sleep state.
  • at least one sensor bone conduction sensor 101, microphone 102 or microphone 103 in the sensor group 10 is set as a preset sensor.
  • the bone conduction sensor 101 is preset as a preset sensor.
  • the bone conduction sensor 101 is configured to collect the user's voice signal, and when the user's voice signal is received, the sliding detection device 50 controls the microphone 102, the microphone 103, the echo cancellation devices 60 and 70, the sound enhancement processing device 80, and the sound fusion device 90 enters the working state from the sleep state, and transmits the user's voice signal to the voice fusion device 90.
  • the microphone 102 and the microphone 103 are configured to collect sound signals emitted from the outside world, and transmit the sound signals emitted from the outside world to the echo cancellation device 60 and the echo cancellation device 70, respectively.
  • the echo cancellation device 60 and the echo cancellation device 70 are configured to perform echo cancellation processing on the external sound signals collected by the microphone 102 and the microphone 103 respectively, and transmit the echo cancellation processing sound signals to the sound enhancement processing device 80.
  • the sound enhancement processing device 80 is configured to perform a noise reduction process on the sound signal after the echo cancellation process, and input the signal after the noise reduction process once to the sound fusion device 90.
  • the sound fusion device 90 is configured to use an adaptive filtering method to perform fusion processing on the user's voice signal (including the voice signal obtained by the bone conduction sensor and the voice signal obtained by the two microphones) to obtain the input signal Sin1, and the input signal Sin1 again After being processed by the noise reduction device 20, it is input to the processor 30 to determine the corresponding event information and operation instructions.
  • the input signal Sin1 can be directly input into the processor 30 to determine the corresponding event information and operation instructions, and the electronic device 1 performs corresponding operations based on the operation instructions, or sends 40 to The operation instruction is sent to the corresponding terminal device so that the terminal device can perform the corresponding operation according to the operation instruction.
  • the input signal Sin1 can also be directly sent to the corresponding terminal device to control the corresponding terminal device to perform the corresponding operation.
  • the bone conduction sensor 101 closely fits the ear bone of the user to fully collect the propagation of the user's voice along the skull when speaking.
  • the vibration of the sound can drive the skull and muscle tissue to vibrate, and this vibration frequency is just collected by the bone conduction sensor 101.
  • the advantage of the bone conduction sensor 101 is that the sampling frequency and interval are much lower than ordinary microphones, can effectively collect low-frequency signals, isolate high-frequency noise, basically cover most of the formants of human voices, and can contain effective information of human voices.
  • the microphones 102 and 103 are respectively used to collect sound signals from the outside world, which include the user's voice signal and noise in the external environment.
  • the reason for using the microphones 102 and 103 is that they can collect external environmental noise and the voice of the user speaking, which covers all frequency band signals in the speaker's voice.
  • microphones 102 and 103 will collect part of the streaming media signal, which will become interference signals, which will further affect the final output. The signal interferes. Therefore, the echo cancellation devices 60 and 70 can be used in combination with the original reference signal of the streaming media signal to perform echo cancellation on the streaming media signals collected by the microphones 102 and 103.
  • the specific working principle of echo cancellation is the prior art, and it will not be repeated here. In a specific example, if the number of microphones is at least two, the number of echo cancellation devices is also at least two. Moreover, each echo cancellation device corresponds to a microphone.
  • the sound enhancement processing device 80 needs to perform a noise reduction process. That is, the sound enhancement processing device 80 is used to perform a noise reduction process on the sound signal after the echo cancellation process, and then input the signal after the noise reduction process once into the sound fusion device 90.
  • the sound enhancement processing device 80 is mainly used to perform beamforming processing on the sound signal after echo cancellation processing.
  • beamforming processing spatial filtering of the sound signal after echo cancellation processing is realized.
  • the purpose of beamforming is to perform directional filtering and remove interference.
  • the signal after spatial filtering processing is subjected to noise suppression processing.
  • the beamforming processing and the noise suppression processing actually belong to the prior art, and will not be introduced here.
  • the sound fusion device 80 mainly performs fusion processing on the user's voice signal and (the voice signal collected by each sensor in the sensor group) after a noise reduction process.
  • the processing method can adopt adaptive filtering method for fusion processing.
  • the specific processing procedure is to extract the effective low-frequency signal from the user's voice signal collected by the bone conduction sensor 101, and then replace the effective low-frequency signal with the low-frequency signal in the signal after a noise reduction process by the sound enhancement device.
  • further filtering processing of all signals is implemented in the replacement process, so that the obtained output signal is clean and closer to the human voice, which is used as the voice recognition signal of the voice recognition device.
  • the noise reduction device 20 is also used to perform secondary noise reduction processing on the input signal Sin1, the main purpose of which is to further eliminate environmental noise and non-stationary noise. For example, noisy street noise, engine noise, or wind noise. And the signal after the second noise reduction process is finally input into the speech recognition signal of the speech recognition device.
  • a bone conduction sensor collects the user's action parameters, and then after the noise reduction device performs noise reduction, the processor extracts the state recognition feature vector from the noise reduction processed data, and recognizes according to the state The feature vector determines the corresponding event information, so that the electronic device executes the corresponding operation function after identifying the user's event information.
  • the function of the electronic device is no longer single, for example, matching operation instructions corresponding to event information, and controlling the terminal device to perform corresponding operations through the operation instructions. Or, periodically count the movement data corresponding to the action, and synchronize to the terminal device.
  • the electronic device collects the user's low-frequency voice signal through the bone conduction sensor, and then collects the sound signal from the outside through the microphone. After noise reduction, the signal collected by the microphone is input to the sound fusion device together with the low-frequency voice signal for fusion. , Obtain a true and clear user voice signal and input it to the terminal device to control the terminal device to perform corresponding operations.
  • the electronic device itself has more functions, and it can also establish a communication connection with other terminal devices. When it is not convenient for the user to directly control other terminal devices, collect some control instructions from the user to control other terminal devices To better serve users.
  • Fig. 3 is a schematic diagram of an electronic device according to a third embodiment of the present invention.
  • the only difference from the second embodiment of the present invention is that the microphones 102 and 103 are preset as predetermined sensors, that is, the microphones 102 and 103 are receiving the user's voice signal.
  • the sliding detection device 50 controls the bone conduction sensor 101, the echo cancellation devices 60 and 70, the sound enhancement processing device 80 and the sound fusion device 90 enter the working state from the dormant state, and transmits the user's voice signal to the sound fusion device 90.
  • Other similar devices and their energy supply will not be repeated here.
  • the user's action parameters are collected by the bone conduction sensor, and then the noise is reduced by the noise reduction device.
  • the processor extracts the state recognition feature vector from the data after the noise reduction processing, and determines the corresponding event according to the state recognition feature vector Information, enabling the electronic device to execute the corresponding operation function after recognizing the user’s event information.
  • the electronic device itself can be made richer in functions, and it can also establish a communication connection with other terminal devices.
  • some control instructions from the user can be collected to control other terminal devices. To better serve users.
  • Fig. 4 is a schematic diagram of an electronic device according to a fourth embodiment of the present invention.
  • the electronic device 4 includes a voice information processing part and an action information processing part.
  • the voice processing part is similar to the second embodiment of the present invention and the third embodiment of the present invention, and will not be repeated here. Therefore, this embodiment mainly describes the user's action information in further detail.
  • the acceleration sensors in the sensor group 10 still use bone conduction sensors, but it should be understood that other sensors, such as motion sensors, can also be used in this embodiment.
  • the bone conduction sensor 101 collects the user's action parameters, and inputs the user's action parameters as the input signal Sin2 to the noise reduction device 20 for noise reduction processing, and inputs the noise reduction processing data to the processor 30.
  • the processor 30 includes a feature extraction unit 301, an event information determination unit 302, an operation instruction determination unit 303, and a motion data acquisition unit 304.
  • the feature extraction unit 301 is configured to extract a state recognition feature vector of the data after noise reduction processing.
  • the event information determining unit 302 is configured to determine event information according to the state recognition feature vector.
  • the event information is a user's action.
  • the operation instruction determining unit 303 is configured to determine an operation instruction matching the action according to the user's action.
  • the motion data acquiring unit 304 is configured to periodically count event information corresponding to the user's motion when the user's motion is a motion posture.
  • the motion data acquisition unit 304 may include a counter, a motion data analysis subunit, and the like.
  • the counter can be used to periodically count the number of actions of the user, such as counting steps, number of swimming strokes, etc.
  • the exercise data analysis subunit can analyze the data obtained by the counter to obtain the total number of steps, exercise duration, number of times per unit time (rate), total number of strokes, swimming duration, swimming speed, total energy consumption, etc.
  • the acquired motion data can also be synchronized to the terminal device 2 through the transmitter 40.
  • the user's action parameters are collected by the bone conduction sensor, and then the noise is reduced by the noise reduction device.
  • the processor extracts the state recognition feature vector from the data after the noise reduction processing, and determines the corresponding event according to the state recognition feature vector Information, enabling the electronic device to execute the corresponding operation function after recognizing the user’s event information.
  • the electronic device itself can be made richer in functions, and it can also establish a communication connection with other terminal devices.
  • some control instructions from the user can be collected to control other terminal devices. To better serve users.
  • Fig. 5 is a schematic diagram of an equipment system according to a fifth embodiment of the present invention.
  • the device system includes a terminal device and at least one electronic device.
  • the electronic device may be the electronic device in any of the above embodiments or implementations.
  • the device system 5 of this embodiment includes two electronic devices 51 and 52 and a terminal device 53. It should be understood that this embodiment only uses two electronic devices 51 and 52 for illustration, and does not limit the number of electronic devices.
  • the electronic device 51 and the electronic device 52 are set at different target locations, and are configured to determine event information of the corresponding target locations. That is, the electronic device 51 and the electronic device 52 can be respectively worn on different parts of the user.
  • the electronic device 51 is a smart earphone, which is worn on the ear
  • the electronic device 52 is a smart watch, which is worn on the wrist.
  • the terminal device 53 is connected to the electronic device 51 and the electronic device 52 via Bluetooth, and the terminal device 53 can perform corresponding operations based on the operation instruction/event information determined by the electronic device 51 and/or the electronic device 52.
  • the terminal device 53 is further configured to optimize the quantity statistics information of the same type of event information for each target part. That is to say, both the electronic device 51 and the electronic device 52 can obtain exercise data, and both can be synchronized to the terminal device 53, the terminal device 53 can optimize according to the exercise data uploaded by the electronic device 51 and the exercise data uploaded by the electronic device 52 to Get the user's most accurate exercise data.
  • the terminal device 53 generates action completion information corresponding to the corresponding comprehensive action type based on at least one event completion information of each target part.
  • the comprehensive action type includes at least one action type corresponding to at least two target parts respectively.
  • the event completion information may be generated when the terminal device 53 receives the action information sent by the electronic device or the notification information generated by the action.
  • many types of exercises have multiple parts at the same time. For example, yoga may require multiple parts of the head, hands, and legs.
  • the terminal device 53 can combine the action types of the head and hands into a corresponding comprehensive action type according to the event completion information (that is, the action completion information) simultaneously uploaded by the electronic device 51 and the electronic device 52, and obtain the information of the comprehensive action type. Action completion information.
  • the embodiments of the present invention further enrich the functions of the electronic devices, and bring great convenience to users.
  • Fig. 6 is a flowchart of a control method of an electronic device according to a fifth embodiment of the present invention. As shown in FIG. 6, the control method of the electronic device in the embodiment of the present invention includes the following steps:
  • Step S110 Collect the user's action parameters.
  • the user's action parameters are collected in real time.
  • the action parameters can include the action parameters generated when a part (such as the head or hands) performs an action (such as nodding, shaking the head, arm movements, etc.), and can also include the ear bones, skull, jaw, etc. when the user speaks.
  • the operation parameters of vibration, etc. are not limited in this embodiment.
  • Step S120 Perform noise reduction processing on the input signal, and obtain data after noise reduction processing.
  • the input signal is determined by the collected action parameters.
  • the input signal is an action parameter collected by the sensor group.
  • Step S130 extracting a state recognition feature vector based on the data after the noise reduction processing.
  • a windowed feature extraction method is used to extract the state recognition feature vector from the noise-reduction processed data.
  • Step S140 Determine event information according to the state recognition feature vector.
  • the action parameter is an action parameter generated when a part performs an action
  • determining the event information may be determining the user's action.
  • the motion parameter is an action parameter that generates vibrations such as ear bones, skull, jaw bones, etc. when the user speaks
  • the determined event information may be voice information that determines the user.
  • the user's actions may include head movements and/or hand movements. Among them, the head movement may include nodding, shaking the head, etc., and the hand movement may include arm swinging during walking and running.
  • control method of this embodiment further includes: determining an operation instruction corresponding to the above-mentioned event information, and controlling the corresponding electronic device to perform an operation corresponding to the operation instruction.
  • control method of this embodiment further includes: determining the operation instruction corresponding to the above event information, and sending the operation instruction to the corresponding terminal device to control the terminal device to execute the operation instruction The corresponding operation.
  • the event information is the user's voice information
  • the operation instruction corresponding to the voice information can be determined to control the electronic device or the corresponding terminal device to perform the operation corresponding to the operation instruction.
  • control method of this embodiment further includes: waking up the electronic device according to the event information and a local hot word database, the local hot word database including at least one hot word, and the at least one hot word including At least one voice message and/or at least one user action, the electronic device is awakened after recognizing the corresponding hot word. That is, when the electronic device is in the dormant state, when corresponding voice information or action information is detected, the electronic device is awakened. For example, it is recognized that the input voice is "start" or the electronic device is touched and a user action is formed on the electronic device to wake up the sleeping electronic device.
  • control method of this embodiment further includes: determining a corresponding hot word according to the event information and a local hot word database, and performing secondary verification on the corresponding hot word to wake up the electronic device.
  • the first verification of the event information may be preliminary filtering, and the second verification may be accurate verification, so as to avoid the occurrence of similar homophonic words also starting the device, which improves the reliability of the electronic device.
  • control method of this embodiment further includes: periodically counting the amount of the event information to obtain motion data.
  • control method of this embodiment further includes: synchronizing the motion data to the corresponding terminal device.
  • data can be synchronized in real time or periodically.
  • the electronic device 1 is not connected to the corresponding terminal device, the next time it is connected, the movement The data is synchronized to the corresponding terminal device, which is not limited in this embodiment.
  • step S130 includes adopting a windowed feature extraction method.
  • the windowed feature extraction is a custom term, and its actual meaning is: first perform the windowing process on the data after the noise reduction process, and then perform the feature extraction. For example, if the time window is 50 Hz, and the number of points collected by each coordinate axis in the time window is 50 points, then the three axes will collect 50*3 points. These points form a 50*3 point matrix, and Extract the state recognition feature vector from this matrix.
  • the state recognition feature vector may include a series of vectors such as the maximum value, minimum value, variance, or mean value of the data on each coordinate axis. It should be understood that this embodiment does not limit the method for extracting the state recognition feature vector, and other methods such as neural network models can be applied in this embodiment.
  • step S140 includes: inputting the acquired state recognition vector into a pre-trained state recognition model for classification, so as to predict the aforementioned event information (for example, user action or user voice).
  • a pre-trained state recognition model for classification so as to predict the aforementioned event information (for example, user action or user voice).
  • the specific process is conventional machine learning technology, so I won't explain it too much here.
  • step S140 includes: determining the user action according to the determined state recognition feature vector, the preset action type, and the corresponding action type feature vector.
  • the feature vector of the preset action type and the corresponding relationship between the action type and its corresponding feature vector can be obtained in advance, and stored in a predetermined memory.
  • a predetermined threshold for example, 80%.
  • shaking the head can generally be divided into at least two parts. For example, shaking the head can be divided into a leftward shifting part and a rightward shifting part.
  • control method of this embodiment further includes: monitoring the event information according to the time information when the event information is generated, and generating voice information in response to not generating the event information again within a predetermined time threshold.
  • control method of this embodiment further includes: generating voice information in response to the event information being not generated within a predetermined time threshold after the initial action parameters are collected.
  • the user's action parameters are collected through the sensor group, and after the noise reduction is performed by the noise reduction device, the processor extracts the state recognition feature vector from the data after the noise reduction processing, and determines the corresponding event information according to the state recognition feature vector , So that the electronic device executes the corresponding operation function after recognizing the user's event information.
  • the electronic device itself can be made richer in functions, and it can also establish a communication connection with other terminal devices.
  • some control instructions from the user can be collected to control other terminal devices. To better serve users.
  • FIG. 7 is a flowchart of the action recognition method of the electronic device according to the seventh embodiment of the present invention. As shown in FIG. 7, the action recognition method of an electronic device in an embodiment of the present invention includes the following steps:
  • Step S210 Collect at least one action parameter of the target part in real time. That is, the electronic device is set at the target part, and the action parameters of the target part are collected in real time.
  • a bone conduction sensor or a motion sensor may be used to collect the motion parameters of the target part.
  • the target part may refer to the head, arms and other parts of the human body.
  • At least one piece of motion information of the target part may be, for example, one piece of head movement information, two pieces of movement information or three pieces of movement information, etc. Information, two action information or three action information, etc.
  • Step S220 Determine whether at least one piece of movement information of the target part is detected according to the collected movement parameters.
  • the collected action parameter since the action parameter is collected in real time, the collected action parameter cannot be determined as one piece of action information in some cases.
  • the target part is the arm, and the electronic device needs to periodically count the number of steps through the swinging action of the electronic device.
  • the arm movement will generate an action parameter, but the action parameter cannot be determined as an arm swing action, so it cannot be regarded as an action information of the detected arm.
  • Step S230 in response to detecting at least one piece of action information of the target part, generate at least one piece of event information of the target part. That is, assuming that an action information of the arm is detected, an arm swing event information of the arm is generated. It is easy to understand that if the motion information of the target part is not detected, no event information is generated.
  • the method of this embodiment further includes:
  • Step S240 generating at least one first notification information including event information based on at least one action event information of the target part.
  • the first notification information can also be an operation instruction.
  • the motion sensor IMU collects action parameters in real time, and determines event information when the action information is detected.
  • the first notification information is determined according to the event information, that is, the operation that matches the event information is determined. Instructions to control electronic devices to perform corresponding operations.
  • step S240 is implemented by a computer program, and the computer program of step S240 can be written into the motion sensor IMU or the processor of the electronic device.
  • the processor of the electronic device for example, earphone chip, watch chip
  • the processor of the electronic device can perform corresponding processing based on the first notification information (that is, the operation instruction).
  • the first notification information includes time information when the event information is generated.
  • the "at least one event information of the target part" can be one event information, two event information, or three event information in the header. Accordingly, one first notification information, two first notification information, or three events can be generated. First notification information and so on.
  • the method of this embodiment further includes: acquiring initial event information, where the initial event information is determined by initial action parameters.
  • the initial event information includes initial time information; based on the initial time information, the event information is monitored, and when the predetermined time threshold is exceeded and no event information is generated, voice information is generated; or based on the initial time information, the action event information is monitored When the predetermined time threshold is exceeded and the first notification information of the event information is not obtained, voice information is generated.
  • the foregoing initial event information can be obtained through, for example, optical sensors, Bluetooth, etc.
  • the optical sensor of the headset when the headset is initially worn on the head, the optical sensor of the headset generates the foregoing initial event information, or the headset is initially worn on
  • the Bluetooth of the head and earphones are connected with mobile terminals such as mobile phones to generate the aforementioned initial event information.
  • An application scenario of this embodiment is that the headphone is initially worn on the head, that is, the generation time of head motion event information is monitored. If the predetermined time threshold is exceeded, no head motion event information is generated or head motion is not obtained.
  • the first notification information of the event information generates voice information to remind the user to perform head movement.
  • true wireless Bluetooth headsets a type of true wireless Bluetooth headset in related technologies includes a main headset and a secondary headset.
  • the primary headset is connected to mobile terminals such as mobile phones or music players via Bluetooth, and the secondary headset is connected to the primary headset via Bluetooth;
  • Another type of true wireless Bluetooth headset includes two in-ear headsets, and does not distinguish between the main headset and the secondary headset), by configuring the motion sensor IMU or processor to be written into the computer program to implement the above method in the primary headset and the secondary headset.
  • the headset or preferably configured in the main headset, or by configuring the motion sensor IMU or processor written in the computer program to realize the above method in the above two in-ear headsets, the application scenario of the true wireless Bluetooth headset is more Rich, by wearing this true wireless Bluetooth headset, users can detect head movements in learning, work and other scenarios without having to wear special motion detection equipment.
  • the method of this embodiment further includes:
  • Step S250 Generate at least one second notification information based on the at least one first notification information.
  • Step S260 Send at least one second notification information to the terminal device, so that the terminal device generates at least one action completion information according to the second notification information.
  • step S250 and step S260 can be implemented by a computer program, wherein the computer program of the method described in steps S210-S260 can be written into the electronic device of this embodiment (for example, smart earphones, smart watches, etc.).
  • a second notification message, a computer program that generates at least one action completion message is written into the second electronic device (such as a mobile phone, a portable computer, etc.).
  • the computer program of the method described in steps S210-S240 is written into the motion sensor IMU in the electronic device, and the computer program of the method described in steps S250-S260 is written into the processor of the headset , Write the computer program of "generating at least one action completion message based on at least one second notification message" into terminal devices such as mobile phones. If multiple second notification messages are generated, correspondingly, multiple action completion messages are generated, that is, the user has completed multiple actions.
  • the "action completion information" can be, for example, pop-up information, text record information, chart information, and so on.
  • Fig. 8 is a flowchart of an action recognition method of an electronic device according to an eighth embodiment of the present invention. As shown in FIG. 8, the action recognition method of an electronic device of this embodiment includes the following steps:
  • Step S310 Collect at least one action parameter of the target part in real time. That is, the electronic device is set at the target part, and the action parameters of the target part are collected in real time.
  • Step S320 Determine whether at least one piece of action information of the target part is detected according to the collected action parameters.
  • Step S330 in response to detecting at least one piece of action information of the target part, generate at least one piece of event information of the target part.
  • step S310-step S330 are similar to step S210-step S230 in the seventh embodiment of the present invention, and will not be repeated here.
  • Step S340 Perform quantitative statistics on various types of event information of the target part.
  • the computer program of the method described in step S340 may be written into a processor of an electronic device, such as a headset processor, a watch processor, and the like.
  • the head horizontal displacement and/or vertical displacement of the head can be detected by the motion sensor IMU of the headset to generate the head horizontal displacement action information, and the head horizontal displacement action information can be detected in real time by the method shown in FIG. 8 , Generate head horizontal displacement action event information, and the processor of the headset will make statistics on the “event information of head horizontal displacement”, so as to realize the step counter function of the headset.
  • the processor of the headset will make statistics on the “event information of head horizontal displacement”, so as to realize the step counter function of the headset.
  • the method of this embodiment further includes:
  • step S350 the quantity statistics of each type of event information are sent to the terminal device so that the terminal device can optimize the data statistics, and obtain optimized quantity statistics information of the same type of event information.
  • the smart headset and smart watch of the same user send their respective step counts to the user's mobile phone, and optimize the step count of the two to improve the accuracy of step counting .
  • the smart headset and smart watch of the same user send their respective step counts to the user's mobile phone, and optimize the step count of the two to improve the accuracy of step counting .
  • the method of this embodiment before detecting at least one action information of the target part, further includes: preset at least one action type of the target part and at least one corresponding to the at least one action type Action type feature information. Therefore, in this embodiment, the corresponding state recognition feature information is obtained according to the collected action parameters, and the obtained state recognition feature information is compared with the pre-stored action type feature information to determine whether the action information is detected. And generate event information based on the detected action information.
  • the action types may include turning the head 40 degrees to the left, turning the head 40 degrees to the right, tilting the head back 20 degrees, etc., and these motion types correspond to the corresponding characteristic information of the motion type.
  • the action type feature information is also feature information corresponding to each action type, and these feature information can be generated based on the action parameters detected by the motion sensor IMU.
  • the method of this embodiment before detecting at least one action information of the target part, further includes: presetting at least one comprehensive action type and at least one comprehensive action type corresponding to the at least one comprehensive action type Feature information; the integrated action type includes at least one action type of each of the at least two target parts; the integrated action type feature information includes at least one action type feature information of each of the at least two target parts.
  • a comprehensive action type such as a yoga action includes a head action type (turn the head 40 degrees to the left) and an arm action type (for example, arms from vertical to horizontal).
  • the comprehensive action type feature information includes the feature information of the head action and the feature information of the arm action.
  • “preset at least one comprehensive action type” may preset one comprehensive action type, two comprehensive action types, or three comprehensive action types, and so on.
  • the terminal device can generate each second notification information corresponding Based on the action completion information corresponding to at least one action type of each target part, the action completion information corresponding to each comprehensive action type is generated.
  • the action recognition method described above in the embodiment of the present invention when the action information corresponding to an action type of the target part is detected, an event information will be generated, and the event information will cause a first notification message containing the event information to be generated.
  • the first notification message will cause a second notification message to be generated, and the second notification message will cause an action completion message to be generated, for example, when the action information corresponding to the action type "turn your head 40 degrees to the left" is detected When, it will result in an action completion message that the action type of "turn the head to the left 40 degrees" is completed.
  • an action completion message that the action type of "turn the head to the left 40 degrees” is completed.
  • the embodiment of the present invention detects at least one action parameter of the target part in real time, determines whether at least one action information of the target part is detected according to the collected action parameters, and generates the target part in response to detecting at least one action information of the target part.
  • At least one event information enables the electronic device to execute the corresponding operation function after recognizing the user's event information.
  • the electronic device itself can be made richer in functions, and it can also establish a communication connection with other terminal devices.
  • some control instructions from the user can be collected to control other terminal devices. To better serve users.
  • FIG. 9 is a flowchart of an action recognition method of an electronic device according to a ninth embodiment of the present invention.
  • the target part corresponding to the electronic device is used as the head for specific description.
  • the action recognition method of the electronic device in the embodiment of the present invention includes the following steps:
  • Step S410 collecting action parameters in real time.
  • a bone conduction sensor or a motion sensor may be used to collect the motion parameters of the target part.
  • Step S420 Determine whether head motion information is detected according to the collected motion parameters.
  • the action parameters are collected in real time, in some cases the collected action parameters cannot be determined as a piece of action information. Therefore, in this embodiment, whether it is head action information is detected according to the collected action parameters, To improve the accuracy of action recognition.
  • Step S430 in response to detecting head motion information, generate head motion event information. It is easy to understand that if head motion information is not detected, no head motion event information is generated.
  • steps S410-S430 may be implemented by a computer program, and the computer program that implements the above method may be written into the motion sensor IMU or processor of the electronic device.
  • the method of this embodiment further includes:
  • Step S440 generating notification information including the event information of the head movement according to the event information of the head movement.
  • the notification information can also be an operation instruction.
  • the motion sensor IMU collects action parameters in real time, and determines the event information of the head movement when the head movement information is detected.
  • the notification information is determined according to the event information of the head movement, that is, the The operation instruction matched with the event information is used to control the electronic device to perform the corresponding operation.
  • step S440 is implemented by a computer program, and the computer program of step S440 can be written into the motion sensor IMU or processor of the electronic device.
  • the processor for example, earphone chip, watch chip
  • the electronic device can perform corresponding processing based on the first notification information (that is, the operation instruction).
  • the method of this embodiment before detecting at least one head motion information, further includes: preset at least one head motion type and at least one head corresponding to the at least one head motion type. Part movement type characteristic information; real-time detection of movement information; judging whether the detected movement information belongs to the at least one head movement type characteristic information; if the detected movement information belongs to the at least one head movement type characteristic information , The event information of the head action is generated, and if the detected action information does not belong to the at least one feature information of the head action type, the event information of the head action is not generated.
  • the head movement types can include head turning 40 degrees to the left, head turning 40 degrees to the right, head tilting back 20 degrees, etc. These head movement types will correspond to the corresponding head movement type characteristics information.
  • Head movement type characteristic information that is, characteristic movement information corresponding to each head movement type, these characteristic movement information can be generated based on the movement parameters detected by the motion sensor IMU.
  • the detected motion information matches one of the head motion type feature information, it means that the head has performed a head motion type, that is A type of head movement has occurred, and the event information of the head movement is generated.
  • the notification information in this embodiment includes time information, and the method in this embodiment further includes:
  • Step S450 based on the time information in the notification information, monitor the head motion event information corresponding to the notification information, and generate voice information in response to exceeding the preset time threshold without executing the notification information and not obtaining new notification information.
  • the head action event information corresponding to a notification message is "head turned 40 degrees to the left", and the notification message contains the time information when "head turned 40 degrees to the left", that is, the time of occurrence.
  • the method monitors the head movement event information of "head turning 40 degrees to the left” from the occurrence time, and generates voice information when the notification information of the head movement event information is not obtained after a predetermined time period threshold is exceeded. That is, when the user wears the electronic device of this embodiment, the head action of "turning the head to the left 40 degrees” is performed, but the head of "head moving to the right” is not obtained within a predetermined time threshold. Action information, the electronic device cannot determine whether the user is shaking his head, so it can remind the user through voice.
  • the "predetermined time period threshold” refers to a preset time period threshold, such as 10 seconds, 20 seconds, 30 seconds, etc., which can be set reasonably by those skilled in the art.
  • the computer program of the method described in steps S410-S440 can be written into the motion sensor IMU or processor, and the computer program of the method described in step S450 is written into the processor of the electronic device, such as a headset chip, and the headset chip executes This part of the computer program generates voice information and sends the voice information to the audio decoding device of the headset.
  • the voice information may be digital audio information.
  • the notification information obtained in step S440 is the first notification information.
  • the method of this embodiment further includes: monitoring each head motion event information based on the time information in the first notification information, and when it exceeds For each predetermined time period threshold and the notification information of each head movement event information is not obtained, the corresponding voice information is generated.
  • the "first notification information” refers to the notification information corresponding to the head motion event information generated by the first head motion information detected by the method.
  • the notification information corresponding to the head motion event information generated by the first head motion information for example, after the user wears, the first head motion is "head turn left 40 degrees", this action will cause the corresponding head action event information and corresponding notification information to be generated
  • the notification information includes the time information (ie, the occurrence time) of the first head action, starting from the occurrence time , Continuously monitor the event information of each head movement, such as each head corresponding to "head turned 40 degrees to the left", “head turned 40 degrees to the right", and "head tilted back 20 degrees” The notification information of the action event information is monitored.
  • each "predetermined time period threshold” is exceeded and each notification information is not generated, each corresponding voice message is generated, and each voice message can remind the user to do the above three actions.
  • each "predetermined time period threshold” may be the same or different.
  • each "predetermined time period threshold” is different.
  • the method of this embodiment further includes: acquiring initial header information, where the initial header information includes initial time information; and monitoring head movement event information based on the initial time information When the threshold of the predetermined time period is exceeded and no head movement event information is generated, voice information is generated; or based on the initial time information, the head movement event information is monitored, when the threshold of the predetermined time period is exceeded and no head movement event is obtained When the information is notified, the voice message is generated.
  • the aforementioned initial head information can be obtained through, for example, optical sensors, Bluetooth, etc.
  • the optical sensor of the headset when the headset is initially worn on the head, the optical sensor of the headset generates the aforementioned initial head information, or the headset is initially worn on the head.
  • the Bluetooth of the headset is connected to a mobile terminal such as a mobile phone to generate the aforementioned initial head information.
  • An application scenario of this embodiment is to start monitoring the generation time of head motion event information from when the headset is initially worn on the head. If the threshold of a predetermined time period is exceeded, no head motion event information is generated or no head motion event is obtained.
  • the notification information of the information generates voice information to remind the user to perform head movement.
  • the embodiment of the present invention detects the action parameters in real time, determines whether head action information is detected according to the collected action parameters, and generates head action event information in response to detecting the head action information, so that the electronic device recognizes the user's event After the information, execute the corresponding operation function.
  • the electronic device itself can be made richer in functions, and it can also establish a communication connection with other terminal devices.
  • some control instructions from the user can be collected to control other terminal devices. To better serve users.
  • any process or method description in the flowchart or described in other ways herein can be understood as a module, segment, or part of code that includes one or more executable instructions for implementing specific logical functions or steps of the process , And the scope of the preferred embodiment of the present disclosure includes additional implementations, which may not be in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. It is understood by those skilled in the art to which the embodiments of the present disclosure belong.
  • the processor executes the various methods and processes described above.
  • the method implementation in the present disclosure may be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory.
  • part or all of the software program may be loaded and/or installed via the memory and/or communication interface.
  • the software program When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be executed.
  • the processor may be configured to perform one of the aforementioned methods in any other suitable manner (for example, by means of firmware).
  • a "readable storage medium” can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, device, or device or in combination with these instruction execution systems, devices, or devices. More specific examples (non-exhaustive list) of readable storage media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable and editable read only memory (EPROM or flash memory), fiber optic devices, and portable read only memory (CDROM).
  • the readable storage medium can even be paper or other suitable media on which the program can be printed, because it can be used for example by optically scanning the paper or other media, and then editing, interpreting, or other suitable media if necessary.
  • the program is processed in a way to obtain the program electronically and then stored in the memory.
  • each part of the present disclosure can be implemented by hardware, software or a combination thereof.
  • multiple steps or methods can be implemented by software stored in a memory and executed by a suitable instruction execution system.
  • a logic gate circuit for implementing logic functions for data information
  • PGA programmable gate array
  • FPGA field programmable gate array
  • a person of ordinary skill in the art can understand that all or part of the steps in the implementation of the above-mentioned implementation methods can be completed by a program instructing relevant hardware.
  • the program can be stored in a readable storage medium, and when the program is executed , Including one of the steps of the method embodiment or a combination thereof.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium.
  • the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种电子设备、控制方法及设备系统，通过传感器组采集用户的动作参数(S110)，然后通过降噪装置进行降噪后(S120)，处理器从经过降噪处理后的数据中提取状态识别特征向量(S130)，根据状态识别特征向量确定对应的事件信息(S140)，使得电子设备识别用户的事件信息后，执行对应的操作功能，由此，可以扩展电子设备的功能。

Description

一种电子设备、控制方法及设备系统
本申请要求了2019年04月19日提交的、申请号为201910319222.2、发明名称为“一种电子设备及其控制方法”的中国专利申请，2019年05月11日提交的、申请号为201910391230.8、发明名称为“头部动作识别方法、装置、设备及存储介质”的中国专利申请，2019年05月11日提交的、申请号为201910391236.5、发明名称为“动作识别方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明实施例涉及数据处理技术领域，更具体地，涉及一种电子设备、控制方法及设备系统。
背景技术
随着电子技术的不断发展，智能电子设备不断推陈出新，为人们的生活带来越来越多的遍历，例如智能耳机，可以去除环境噪声，使得智能耳机的扬声器传出的声音保质保真。
不过，现在的智能电子设备功能比较单一，如何拓展智能电子设备的功能，进而给用户提供更好的服务是目前需要解决的问题。
发明内容
有鉴于此，本发明实施例提供一种电子设备、控制方法及设备系统，以扩展电子设备的功能。
第一方面，本发明实施例提供一种电子设备，所述电子设备包括：
传感器组，被配置为采集用户的动作参数，所述传感器组包括加速度传感器；
降噪装置，被配置为对输入信号进行降噪处理，获取经过降噪处理后的数据，所述输入信号根据所述动作参数确定；以及
处理器，被配置为根据所述降噪处理后的数据提取状态识别特征向量，并根据所述状态识别特征向量确定事件信息。
可选的，所述输入信号为所述动作参数。
可选的，所述事件信息为用户的动作或用户的语音信息，所述处理器还被配置为 确定与所述用户的动作对应的操作指令。
可选的，所述处理器还被配置为控制所述电子设备执行与所述操作指令相对应的操作；或者
所述处理器被配置为控制所述电子设备的发送器将所述操作指令发送至对应的终端设备，以控制所述终端设备执行与所述操作指令对应的操作。
可选的，所述处理器还被配置为响应于所述用户的动作为运动姿态，周期性地统计与所述用户的动作对应的事件信息。
可选的，所述电子设备中的发送器受控将所述运动数据同步至对应的终端设备。
可选的，所述处理器被配置为采用分窗特征提取方法从所述经过降噪处理后的数据中提取所述状态识别特征向量。
可选的，所述传感器组还包括麦克风，所述麦克风被配置为采集外界发出的声音信号；
所述电子是设备还包括：
声音活动检测装置，被配置为在接收到所述传感器组中的预设传感器传输的语音信号时，唤醒所述传感器组中除所述预设传感器之外的传感器；
其中，所述加速度传感器或所述麦克风被设置为预设传感器。
可选的，所述电子设备还包括声音增强处理装置、声音融合装置以及回声消除装置；
所述回声消除装置被配置为接收所述麦克风发送的声音信号，并对所述声音信号进行回声消除处理；
所述声音增强处理装置被配置为将经过回声消除处理后的声音信号进行降噪处理，以增强声音信号；
声音融合装置被配置为采用自适应滤波方法将增强后的声音信号和传感器组采集的动作参数进行融合处理，获取所述输入信号。
可选的，所述处理器还被配置为根据所述事件信息和本地热词库唤醒所述电子设备，所述本地热词库包括至少一个热词，所述至少一个热词包括至少一个语音信息和/或至少一个用户动作，所述电子设备在识别到对应的热词后被唤醒。
可选的，所述处理器还被配置为根据所述事件信息和本地热词库确定对应的热词，并对所述对应的热词进行二次验证以唤醒所述电子设备。
可选的，所述处理器根据所述状态识别特征向量、预设的动作类型及对应的动作 类型特征向量确定所述用户动作。
可选的，所述处理器被配置为根据生成所述事件信息的时间信息，对所述事件信息进行监测，响应于在预定时间阈值内未再次生成所述事件信息，生成语音信息。
可选的，所述处理器还被配置为响应于在采集初始动作参数之后预定时间阈值内未生成所述事件信息，生成语音信息。
可选的，所述处理器还被配置为响应于在采集初始动作参数之后预定时间阈值内未确定所述操作指令，生成语音信息。
第二方面，本发明实施例提供一种电子设备的控制方法，所述方法包括：
采集用户的动作参数；
对输入信号进行降噪处理，获取经过降噪处理后的数据，所述输入信号根据所述动作参数确定；
根据所述降噪处理后的数据提取状态识别特征向量；以及
根据所述状态识别特征向量确定事件信息。
可选的，所述事件信息为用户的动作的语音信息；
所述方法还包括：
确定与所述事件信息对应的操作指令。
可选的，所述方法还包括：
控制对应的电子设备执行与所述操作指令向对应的操作；或者
控制所述电子设备的发送器将所述操作指令发送至对应的终端设备，以控制所述终端设备执行与所述操作指令对应的操作。
可选的，所述方法还包括：
响应于所述用户的动作为运动姿态，周期性地统计与所述用户的动作对应的事件信息。
第三方面，本发明实施例提供一种设备系统，其特征在于，所述设备系统包括：
终端设备；以及
至少一个如本发明实施例第一方面所述的电子设备。
可选的，各所述电子设备设置于不同的目标部位，被配置为确定对应的目标部位的事件信息；
所述终端设备被配置为对各所述目标部位的相同类型的事件信息的数量统计信息进行优化。
可选的，所述终端设备基于各所述目标部位的至少一个事件完成信息，生成对应的综合动作类型对应的动作完成信息；
其中，所述综合动作类型包括至少两个目标部位分别对应的至少一个动作类型。
本发明实施例通过传感器组采集用户的动作参数，然后通过降噪装置进行降噪后，处理器从经过降噪处理后的数据中提取状态识别特征向量，根据状态识别特征向量确定对应的事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能，由此，可以扩展电子设备的功能，并且，还可以通过其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
附图说明
图1是本发明第一实施例的电子设备的示意图；
图2是本发明第二实施例的电子设备的示意图；
图3是本发明第三实施例的电子设备的示意图；
图4是本发明第四实施例的电子设备的示意图；
图5是本发明第五实施例的设备系统的示意图；
图6是本发明第六实施例的电子设备的控制方法的流程图；
图7是本发明第七实施例的电子设备的动作识别方法的流程图；
图8是本发明第八实施例的电子设备的动作识别方法的流程图；
图9是本发明第九实施例的电子设备的动作识别方法的流程图。
具体实施方式
以下基于实施例对本发明进行描述，但是本发明并不仅仅限于这些实施例。在下文对本发明的细节描述中，详尽描述了一些特定的细节部分。对本领域技术人员来说没有这些细节部分的描述也可以完全理解本发明。为了避免混淆本发明的实质，公知的方法、过程、流程、元件和电路并没有详细叙述。
此外，本领域普通技术人员应当理解，在此提供的附图都是为了说明的目的，并且附图不一定是按比例绘制的。
除非上下文明确要求，否则在说明书的“包括”、“包含”等类似词语应当解释为包含的含义而不是排他或穷举的含义；也就是说，是“包括但不限于”的含义。
在本发明的描述中，需要理解的是，术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性。此外，在本发明的描述中，除非另有说明，“多个”的含义是两个或两个以上。
本发明实施例提供了一种电子设备，本实施例的电子设备主要是可穿戴设备，其本身具有一定的功能。例如智能耳机，自身具有的功能就是播放音乐或者在用户通话时传输语音信号，包括用户发出的语音信号或者电话对端传输的语音信号等；在例如手表，自身具有的功能包括查看时间、日历、设置闹钟等。在本实施例中，给终端设备附加一些其他的实用功能，利用这些使用功能给用户带来一定的便利。
图1是本发明第一实施例的电子设备的示意图。如图1所示，电子设备1包括传感器组10、降噪装置20以及处理器30。其中，传感器组10至少包括加速度传感器。在一种可选的实现方式中，传感器组10可以包括骨传导传感器或者运动传感器。其中，骨传导传感器可以根据运动部位(例如头部等)的振动频率来检测该运动部位的动作信息，或根据耳骨、头骨、颌骨等部位的振动频率获取语音信号。容易理解，骨传导传感器本质上也是一种加速度传感器。运动传感器，也称为IMU(Inertial measurement unit，惯性测量单元)，运动传感器是测量物体三轴姿态角(或角速度)以及加速度的装置。一般来说，IMU包括各轴的加速度传感器和陀螺仪，以检测物体(例如固定为某一部位的电子设备)在三维空间中的加速度信息和角速度信息，由此可以得到物体的运动姿态。
在本实施例中，传感器组10被配置为采集用户的动作参数。其中，动作参数可以包括一部位(例如头部或手部)做动作(例如点头、摇头、手臂活动等)时产生的动作参数，也可以包括用户在说话时耳骨、头骨、颌骨等产生振动的动作参数等，本实施例并不对此进行限制。
降噪装置20被配置为对输入信号进行降噪处理，获取经过降噪处理后的数据。其中，输入信号由采集的动作参数确定。在一种可选的实现方式中，所述输入信号为传感器组采集到的动作参数。
处理器30被配置为根据降噪处理后的数据提取状态识别特征向量，并根据该状态识别特征向量确定事件信息。在一种可选的实现方式中，若动作参数为一部位做动作时产生的动作参数，则确定事件信息可以为确定用户的动作。若运动参数为用户在说话时耳骨、头骨、颌骨等产生振动的动作参数，则确定事件信息可以为确定用户的语音信息。可选的，用户的动作可以包括头部动作和/或手部动作等。其中，头部动 作可以包括点头、摇头等，手部动作可以包括走路以及跑步时的摆臂动作等。
在一种可选的实现方式中，处理器30还被配置为确定与上述事件信息对应的操作指令，并控制对应的电子设备执行与该操作指令相对应的操作。例如，假设事件信息为用户的动作，以电子设备1为安装了音乐播放器的智能手表为例，在确定用户的动作为水平摇动手臂后，确定的操作指令为“停止播放当前音乐”，则处理器响应于该操作指令控制智能手表停止播放当前音乐。
在一种可选的实现方式中，电子设备1还包括发送器40。处理器30还被配置为确定与上述事件信息对应的操作指令，控制发送器40将该操作指令发送至对应的终端设备，以控制该终端设备执行与该操作指令对应的操作。例如，假设事件信息为用户的动作，以电子设备1为智能耳机为例，其中，智能耳机通过蓝牙与对应的终端设备连接。在处理器30确定用户的动作为摇头动作后，确定的操作指令为“停止播放当前音乐”，处理器30控制发送器40将“停止播放当前音乐”的操作指令通过蓝牙通信发送给对应的终端设备，以控制该终端设备根据该操作指令停止播放当前音乐。
在其他实施例中，事件信息为用户的语音信息，可以确定该语音信息对应的操作指令，以控制电子设备或对应的终端设备执行与该操作指令对应的操作。
在一种可选的实现方式中，处理器还被配置为根据事件信息和本地热词库唤醒电子设备，所述本地热词库包括至少一个热词，所述至少一个热词包括至少一个语音信息和/或至少一个用户动作，所述电子设备在识别到对应的热词后被唤醒。也即，在电子设备处于休眠状态时，在检测到对应的语音信息或者动作信息时，唤醒该电子设备。例如，识别到输入语音为“启动”或者触碰电子设备并在电子设备上形成一用户动作，唤醒休眠中的电子设备。
进一步可选的，所述处理器还被配置为根据所述事件信息和本地热词库确定对应的热词，并对所述对应的热词进行二次验证以唤醒所述电子设备。可选的，对事件信息的一次验证可以为初步过滤，二次验证可以为精准验证，以避免出现类似的谐音词也启动设备的情况，提高了电子设备的可靠性。
应理解，本实施例的用户动作(或用户语音)与操作指令的对应关系仅仅是示例性的，在实际应用场景中，可以根据电子设备的类型来确定各种用户动作(或用户语音)与各操作指令的对应关系，本实施例并不对此进行限制。
在一种可选的实现方式中，在事件信息表征一种运动姿态时，周期性统计该事件信息的数量，以获取运动数据。具体地，当确定的事件信息为用户的动作，并且用户 的动作包括任一种运动姿态时，处理器30还被配置为周期性的统计与用户动作对应的运动数据。例如，运动姿态可以为游泳姿态，电子设备可以包括智能耳机和/或智能手表，其中，智能耳机可以检测在游泳过程中的头部动作、智能手表可以检测游泳过程中的手部动作。由此，可以通过周期性统计用户的动作，可以得到相关的运动信息，例如游泳的划动次数、游泳距离等；或者跑步距离、跑步步数等。
在一种可选的实现方式中，处理器30还被配置为控制发送器40将运动数据同步至对应的终端设备。可选的，在电子设备1与对应的终端设备连接时，可以实时同步数据，也可以周期性地同步数据，在电子设备1与对应的终端设备未连接时，可以在下次连接时，将运动数据同步至对应的终端设备，本实施例并不对此进行限制。
在一种可选的实现方式中，在处理器30提取状态识别特征向量时，可以采用分窗特征提取方法。其中，分窗特征提取是自定义名词，其实际含义为：先对经过降噪处理后的数据进行分窗处理，然后再进行特征提取。例如，时窗为50Hz，每个坐标轴在该时窗内采集的点数为50个点，那么三轴则采集的是50*3的点数，将这些点构成一个50*3的点数矩阵，并从这个矩阵中提取状态识别特征向量。可选的，状态识别特征向量可以包括：每个坐标轴上数据的最大值、最小值、方差或者均值等一系列的向量。应理解，本实施例并不对状态识别特征向量的提取方法进行限制，其他方法例如神经网络模型等均可以应用于本实施例中。
在一种可选的实现方式中，通过处理器30将获取的状态识别向量输入至预先训练的状态识别模型中进行分类，以预测上述事件信息(例如用户动作或用户语音)。具体过程为常规的机器学习技术，这里不做过多说明。
在另一种可选的实现方式中，处理器30根据确定的状态识别特征向量、预设的动作类型及对应的动作类型特征向量确定用户动作。具体地，可以预先获取预设的动作类型的特征向量以及动作类型及其对应的特征向量的对应关系，并存储至预定内存中。在获取当前的状态识别特征向量时，与预定内存中的特征类型的特征向量进行相似度比对，在对应的相似度大于预定阈值(例如80％)时，确定用户的动作。
以传感器组10包括骨传导传感器为例，骨传导传感器可以采集用户的头部动作参数，即头部运动时的加速度数据，由于用户运动时属于三维向量，包括X、Y和Z等三轴数据。降噪装置20分别对三轴数据进行降噪处理，去除头部动作参数中的毛刺等噪声，获取数据。处理器30可以从经过降噪处理后的数据中提取状态识别特征向量，并根据状态识别特征向量获取事件信息。可选的，从预建立的数据库中匹配与 确定的事件信息对应的操作指令。其中，预建立的数据库中包括各事件信息与操作指令的对应关系。例如，动作为摇头，而与该摇头动作对应的操作指令时接通手机来电。那么，电子设备就会通过发送器40将该操作指令发送至智能终端，用以智能终端根据该操作指令挂断当前的来电请求。
又或者，当识别动作为点头时，从预建立的数据库中匹配与动作对应的操作指令为接听手机来电。那么，类似的道理，电子设备会将接听手机来电的控制指令发送至智能终端，用以控制智能终端接听手机来电。
如果终端设备是智能家居，动作为手势动作，那么，处理器30还用于匹配与手势动作对应的操作指令，然后利用发送器40将操作指令发送至智能家居，并控制智能家居执行相应的操作。
当然，处理器30匹配与动作对应的操作指令后，也可以根据操作指令控制电子设备执行与操作指令对应的操作。例如，电子设备是智能耳机，那么，向左歪头，则可以是操作智能耳机减小音量，或者向右歪头，则是调大耳机音量。
容易理解，点头摇头动作一般可至少分为两部分，例如摇头，可以分为向左偏移部分和向右偏移部分。在一种可选的实现方式中，处理器30还被配置为根据生成事件信息的时间信息，对事件信息进行监测，响应于在预定时间阈值内未再次生成事件信息，生成语音信息。以摇头为例，处理器确定本次采集的动作参数对应的事件信息为“向左偏移40°”，在预定时间阈值内未收到，也即已经超过预定时间并且还未接收到表征“向右偏移”的事件信息，则可以通过语音提醒用户做下一个动作，以完成用户想要做出的动作。可选的，预定时间阈值可以为10秒、30秒、1分钟等，本实施例并不对此进行限制。
在一种可选的实现方式中，处理器30还被配置为响应于在采集初始动作参数之后预定时间阈值内未生成所述事件信息，生成语音信息。在另一种可选的实现方式中，处理器30还被配置为响应于在采集初始动作参数之后预定时间阈值内未确定所述操作指令，生成语音信息。以电子设备1为智能耳机为例，初始动作参数也即用户将智能耳机放入耳朵的动作参数，在用户将耳机放入耳朵预定时间阈值后，未生成事件信息(或操作指令)，则生成语音信息提醒用户进行下一步的操作。该预定时间阈值可以为1分钟、5分钟、10分钟等，本实施例并不对此进行限制。
可选的，电子设备除了可以实现上述功能外，还可以实现周期性的统计用户的动作数据，并同步至终端设备。例如，终端设备可以是智能穿戴设备或者智能终端，甚 至可以是智能家居等等。在电子设备执行周期性的统计用户的动作数据，并同步至终端设备的功能时，电子设备中的功能模块所执行的功能如下：
骨传导传感器10采集用户的动作参数，该动作参数为与运动姿态对应的动作参数，也即，在本实施例中，可以根据电子设备的佩戴位置不同，采集不同部位的动作参数。通过上文类似的工作原理，利用降噪装置20对动作参数进行降噪后得到相应的数据，处理器30从数据中提取状态识别特征向量，输入至预先训练的动作识别模型中，进而确定用户的动作为运动姿态。运动姿态可以包括跑步、走路、游泳等等。其主要是为了周期性的统计用户的运动数据，例如用户每天跑步时长、消耗的卡路里，或者每周游泳次数、游泳平均速率、游泳单次的平均时长或者消耗的卡路里等等，并且可以将这些运动数据同步至终端设备。这些运动数据都可以处理器根据动作参数计算得到，相关技术中已经存在较为成熟的计算过程，这里不做过多说明。
可选的，电子设备除了可以实现上述功能外，还可以包括采集用户的语音信号，并对语音信号进行处理后，得到真实、清晰的语音信号输入至语音识别设备中，实现对语音识别设备的控制。
本发明实施例通过传感器组采集用户的动作参数，然后通过降噪装置进行降噪后，处理器从经过降噪处理后的数据中提取状态识别特征向量，根据状态识别特征向量确定对应的事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
图2是本发明第二实施例的电子设备的示意图。在本实施例中，主要介绍事件信息为语音信息时电子设备的操作过程。如图2所示，电子设备1还可以包括：声音活动检测装置50、回声消除装置60和70，声音增强处理装置80以及声音融合装置90。在本实施例中，传感器组10中的加速度传感器为骨传导传感器101，传感器组10还包括麦克风102和麦克风103，其中，麦克风的数量与回声消除装置的数量对应，本实施例以两个麦克风为例进行说明，应理解，本实施例并不对此进行限制。
具体的，声音活动检测装置50被配置为当接收到传感器组10中的预设传感器传输的语音信号时，唤醒传感器组10中除预设声音传感器之外的传感器、声音增强处理装置80以及声音融合装置90从休眠状态进入工作状态。其中，将传感器组10中的至少一个传感器(骨传导传感器101、麦克风102或麦克风103)设置为预设传感 器。也就是说，仅设置声音活动检测装置50和传感器组10中的预设传感器处于工作状态，而其他元器件处于休眠状态，只有当声音活动检测装置50接收到传感器组10中的预设传感器传输的用户发出的语音信号时，才会唤醒语音增强系统中的其他部件从休眠状态进入工作状态。
如图2所示，在本实施例中，将骨传导传感器101预先设置为预设传感器。骨传导传感器101被配置为采集用户的语音信号，并在收到用户的语音信号时使得滑动检测装置50控制麦克风102、麦克风103、回声消除装置60和70，声音增强处理装置80以及声音融合装置90从休眠状态进入工作状态，并将用户的语音信号传输至声音融合装置90。
麦克风102和麦克风103被配置为采集外界发出的声音信号，并将外界发出的声音信号分别传输至回声消除装置60和回声消除装置70。
回声消除装置60和回声消除装置70被配置为分别将麦克风102和麦克风103采集的外界发出的声音信号进行回声消除处理，并将经过回声消除处理后的声音信号传输至声音增强处理装置80。
声音增强处理装置80被配置为将经过回声消除处理后的声音信号进行一次降噪处理，并将经过一次降噪处理后的信号输入至声音融合装置90。
声音融合装置90被配置为利用自适应滤波方法对用户的语音信号(包括骨传导传感器获取的语音信号、以及两个麦克风获取的语音信号)进行融合处理，获取输入信号Sin1，将输入信号Sin1再次经过降噪装置20进行处理后，输入至处理器30来确定对应的事件信息以及操作指令。在另一种可选的实现方式中，可以将输入信号Sin1直接输入至处理器30中以确定对应的事件信息以及操作指令，电子设备1基于该操作指令执行对应的操作，或者通过发送40将该操作指令发送至对应的终端设备，以便终端设备根据操作指令执行相应的操作。在另一种可选的实现方式中，也可以直接将输入信号Sin1发送至对应的终端设备以控制对应的终端设备执行对应的操作。
具体的，骨传导传感器101紧密贴合使用者的耳骨，用以充分采集用户在说话时声音延头骨的传播。声音的振动可以带动头骨和肌肉组织振动，这种振动频率刚好被骨传导传感器101采集。骨传导传感器101的优点在于，采样频率和区间相比普通麦克风低很多，能够有效的采集低频信号，隔离高频噪音，基本覆盖大部分人声的共振峰，可以包含人声的有效信息。
而麦克风102和103则用于分别采集外界发出的声音信号，外界发出的声音信号 中包含了用户的语音信号，以及外界环境中的噪音。采用麦克风102和103的原因在于，其可以采集外侧环境噪音和用户说话的声音，其涵盖了说话人声音中所有频带信号。
为了防止用户发出语音信号时，其正在听音乐或者看电影等流媒体信息时，麦克风102和103将会采集到一部分流媒体信号，这部分流媒体信号将成为干扰信号，进一步对最终要输出的信号进行干扰。因此，可以利用回声消除装置60和70，结合流媒体信号的原始参考信号，对麦克风102和103采集到的流媒体信号进行回声消除。具体回声消除的工作原理为现有技术，这里不做过多赘述。在一个具体的例子中，如果麦克风数量为至少两个时，回声消除装置的数量也为至少两个。而且，每一个回声消除装置分别对应一个麦克风。
此外，由于麦克风102和103采集的声音信号中包含外界环境中的噪音信号，因此需要通过声音增强处理装置80进行一次降噪处理。即，声音增强处理装置80用于将经过回声消除处理后的声音信号进行一次降噪处理，然后，将经过一次降噪处理后的信号输入至声音融合装置90中。
可选的，在一个具体的实施例中，声音增强处理装置80主要是用于对经过回声消除处理后的声音信号进行波束成形处理。通过波束成形处理，实现对经过回声消除处理后的声音信号进行空间滤波。波束成形的目的是为了进行有指向性的滤波和去除干扰。然后，再将经过空间滤波处理后的信号进行噪音抑制处理。对于波束成形处理和噪音抑制处理实际都属于现有技术，这里不做过多介绍。
声音融合装置80其主要是对用户的语音信号和(传感器组中的各传感器采集的语音信号)经过一次降噪处理后的信号进行融合处理。其处理方式可以采用自适应滤波方法进行融合处理。具体的处理过程就是提取骨传导传感器101采集的用户的语音信号中的有效低频信号，然后将该有效低频信号替代经过声音增强装置一次降噪处理后的信号中的低频信号。同时，在替代过程中实现对所有信号的进一步滤波处理，从而使得获取的输出信号是干净的，更加贴近人声音的信号，用以作为语音识别设备的语音识别信号。
可选的，为了使得语音识别设备的语音信号更加清楚自然，降噪装置20还用于对输入信号Sin1进行二次降噪处理，其主要目的是为了进一步消除环境噪音和非平稳噪音。例如嘈杂的街边噪音、引擎噪音或者风噪等。并将经过二次降噪处理后的信号最为最终输入到语音识别设备的语音识别信号。
本发明实施例提供的一种电子设备，骨传导传感器采集用户的动作参数，然后通过降噪装置进行降噪后，处理器从经过降噪处理后的数据中提取状态识别特征向量，根据状态识别特征向量确定对应的事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备的功能不再单一，例如，匹配与事件信息对应的操作指令，通过操作指令控制终端设备执行相应的操作。或者，周期性的统计与动作对应的运动数据，并同步至终端设备。又或者，电子设备采通过骨传导传感器采集用户的低频语音信号，再通过麦克风采集外界发出的声音信号，将麦克风采集的信号进行降噪处理后，与低频语音信号共同输入至声音融合装置进行融合，得到真实清晰的用户语音信号，并输入至终端设备，用以控制终端设备执行相应的操作。通过上述方式，实现电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
图3是本发明第三实施例的电子设备的示意图。如图3所示，在本实施例中，与本发明第二实施例的唯一区别为被预先设置为预定传感器的为麦克风102和103，也即，麦克风102和103在收到用户的语音信号时使得滑动检测装置50控制骨传导传感器101、回声消除装置60和70，声音增强处理装置80以及声音融合装置90从休眠状态进入工作状态，并将用户的语音信号传输至声音融合装置90。其他相似的装置及其供能在此不再赘述。
本发明实施例通过骨传导传感器采集用户的动作参数，然后通过降噪装置进行降噪后，处理器从经过降噪处理后的数据中提取状态识别特征向量，根据状态识别特征向量确定对应的事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
图4是本发明第四实施例的电子设备的示意图。如图4所示，在本实施例中，电子设备4包括语音信息处理部分和动作信息处理部分。其中，语音处理部分与本发明第二实施例和本发明第三实施例类似，在此不再赘述，因此，本实施例主要对用户的动作信息做进一步详细描述。
在本实施例中，传感器组10中的加速度传感器依旧采用骨传导传感器，但是应理解，其他传感器，例如运动传感器也可运用在本实施例中。骨传导传感器101采集 用户的动作参数，并将用户的动作参数作为输入信号Sin2输入至降噪装置20进行降噪处理，将经过降噪处理后的数据输入至处理器30。
如图4所示，处理器30包括特征提取单元301、事件信息确定单元302、操作指令确定单元303以及运动数据获取单元304。其中，特征提取单元301被配置为降噪处理后的数据提取状态识别特征向量。事件信息确定单元302被配置为根据所述状态识别特征向量确定事件信息。可选的，事件信息为用户的动作。操作指令确定单元303被配置为根据用户的动作确定与该动作匹配的操作指令。运动数据获取单元304被配置为在用户的动作为运动姿态时，周期性地统计与所述用户的动作对应的事件信息。
在一种可选的实现方式中，运动数据获取单元304可以包括计数器、运动数据分析子单元等。其中，计数器可以用于周期性地统计用户的动作次数，例如计步、游泳划动次数等。运动数据分析子单元可以对计数器得到的数据进行分析，以获取总步数、运动时长、单位时间次数(速率)、总划动次数、游泳时长、游泳速率、总消耗能量等。在本实施例中，还可以将获取的运动数据通过发送器40同步至终端设备2。
本发明实施例通过骨传导传感器采集用户的动作参数，然后通过降噪装置进行降噪后，处理器从经过降噪处理后的数据中提取状态识别特征向量，根据状态识别特征向量确定对应的事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
图5是本发明第五实施例的设备系统的示意图。设备系统包括终端设备和至少一个电子设备。其中，电子设备可以为上述任一实施例或实现方式中的电子设备。如图5所示，本实施例的设备系统5包括两个电子设备51和52、以及终端设备53。应理解，本实施例只是以两个电子设备51和52进行举例说明，并不对电子设备的数量进行限制。
在一种可选的实现方式中，电子设备51和电子设备52设置于不同的目标部位，被配置为确定对应的目标部位的事件信息。也即电子设备51和电子设备52可以分别佩戴在用户的不同部位。例如电子设备51为智能耳机，则佩戴在耳朵上，电子设备52为智能手表，则佩戴在手腕上。
其中，终端设备53与电子设备51、电子设备52通过蓝牙连接，终端设备53可以基于电子设备51和/或电子设备52确定的操作指令/事件信息执行对应的操作。
在一种可选的实现方式中，终端设备53还被配置为对各目标部位的相同类型的事件信息的数量统计信息进行优化。也就是说，电子设备51和电子设备52均可以获取运动数据，并均同步至终端设备53，则终端设备53可以根据电子设备51上传的运动数据和电子设备52上传的运动数据进行优化，以得到用户最准确的运动数据。
在一种可选的实现方式中，终端设备53基于各目标部位的至少一个事件完成信息，生成对应的综合动作类型对应的动作完成信息。其中，综合动作类型包括至少两个目标部位分别对应的至少一个动作类型。事件完成信息可以为终端设备53在接收到电子设备发送的动作信息或者动作产生的通知信息等情况下生成的。在实际生活中，很多运动类型在同一时刻存在多种部位的动作，例如瑜伽，可能需要头部、手部、腿部等多个部位的动作。终端设备53可以根据电子设备51和电子设备52同时上传的事件完成信息(也即动作完成信息)将头部和手部的动作类型组合为一个对应的综合动作类型，并获取该综合动作类型的动作完成信息。
本发明实施例通过和其他电子设备以及终端设备建立通信连接，进一步地丰富了电子设备的功能，为用户带来了极大的便利。
图6是本发明第五实施例的电子设备的控制方法的流程图。如图6所述，本发明实施例的电子设备的控制方法包括以下步骤：
步骤S110，采集用户的动作参数。可选的，在本实施例中，实时采集用户的动作参数。其中，动作参数可以包括一部位(例如头部或手部)做动作(例如点头、摇头、手臂活动等)时产生的动作参数，也可以包括用户在说话时耳骨、头骨、颌骨等产生振动的动作参数等，本实施例并不对此进行限制。
步骤S120，对输入信号进行降噪处理，获取经过降噪处理后的数据。其中，输入信号由采集的动作参数确定。在一种可选的实现方式中，所述输入信号为传感器组采集到的动作参数。
步骤S130，根据所述降噪处理后的数据提取状态识别特征向量。在一种可选的实现方式中，采用分窗特征提取方法从所述经过降噪处理后的数据中提取所述状态识别特征向量。
步骤S140，根据所述状态识别特征向量确定事件信息。在一种可选的实现方式中，若动作参数为一部位做动作时产生的动作参数，则确定事件信息可以为确定用户的动作。若运动参数为用户在说话时耳骨、头骨、颌骨等产生振动的动作参数，则确定事件信息可以为确定用户的语音信息。可选的，用户的动作可以包括头部动作和/ 或手部动作等。其中，头部动作可以包括点头、摇头等，手部动作可以包括走路以及跑步时的摆臂动作等。
在一种可选的实现方式中，本实施例的控制方法还包括：确定与上述事件信息对应的操作指令，并控制对应的电子设备执行与该操作指令相对应的操作。
在一种可选的实现方式中，本实施例的控制方法还包括：确定与上述事件信息对应的操作指令，将该操作指令发送至对应的终端设备，以控制该终端设备执行与该操作指令对应的操作。
在其他实施例中，事件信息为用户的语音信息，可以确定该语音信息对应的操作指令，以控制电子设备或对应的终端设备执行与该操作指令对应的操作。
在一种可选的实现方式中，本实施例的控制方法还包括：根据事件信息和本地热词库唤醒电子设备，所述本地热词库包括至少一个热词，所述至少一个热词包括至少一个语音信息和/或至少一个用户动作，所述电子设备在识别到对应的热词后被唤醒。也即，在电子设备处于休眠状态时，在检测到对应的语音信息或者动作信息时，唤醒该电子设备。例如，识别到输入语音为“启动”或者触碰电子设备并在电子设备上形成一用户动作，唤醒休眠中的电子设备。
进一步可选的，本实施例的控制方法还包括：根据所述事件信息和本地热词库确定对应的热词，并对所述对应的热词进行二次验证以唤醒所述电子设备。可选的，对事件信息的一次验证可以为初步过滤，二次验证可以为精准验证，以避免出现类似的谐音词也启动设备的情况，提高了电子设备的可靠性。
应理解，本实施例的用户动作(或用户语音)与操作指令的对应关系仅仅是示例性的，在实际应用场景中，可以根据电子设备的类型来确定各种用户动作(或用户语音)与各操作指令的对应关系，本实施例并不对此进行限制。
在一种可选的实现方式中，在事件信息表征一种运动姿态时，本实施例的控制方法还包括：周期性统计该事件信息的数量，以获取运动数据。
在一种可选的实现方式中，本实施例的控制方法还包括：将运动数据同步至对应的终端设备。可选的，在电子设备1与对应的终端设备连接时，可以实时同步数据，也可以周期性地同步数据，在电子设备1与对应的终端设备未连接时，可以在下次连接时，将运动数据同步至对应的终端设备，本实施例并不对此进行限制。
在一种可选的实现方式中，步骤S130包括采用分窗特征提取方法。其中，分窗特征提取是自定义名词，其实际含义为：先对经过降噪处理后的数据进行分窗处理， 然后再进行特征提取。例如，时窗为50Hz，每个坐标轴在该时窗内采集的点数为50个点，那么三轴则采集的是50*3的点数，将这些点构成一个50*3的点数矩阵，并从这个矩阵中提取状态识别特征向量。可选的，状态识别特征向量可以包括：每个坐标轴上数据的最大值、最小值、方差或者均值等一系列的向量。应理解，本实施例并不对状态识别特征向量的提取方法进行限制，其他方法例如神经网络模型等均可以应用于本实施例中。
在一种可选的实现方式中，步骤S140包括：将获取的状态识别向量输入至预先训练的状态识别模型中进行分类，以预测上述事件信息(例如用户动作或用户语音)。具体过程为常规的机器学习技术，这里不做过多说明。
在另一种可选的实现方式中，步骤S140包括：根据确定的状态识别特征向量、预设的动作类型及对应的动作类型特征向量确定用户动作。具体地，可以预先获取预设的动作类型的特征向量以及动作类型及其对应的特征向量的对应关系，并存储至预定内存中。在获取当前的状态识别特征向量时，与预定内存中的特征类型的特征向量进行相似度比对，在对应的相似度大于预定阈值(例如80％)时，确定用户的动作。
容易理解，一些动作，例如点头摇头动作一般可至少分为两部分，例如摇头，可以分为向左偏移部分和向右偏移部分。
在一种可选的实现方式中，本实施例的控制方法还包括：根据生成事件信息的时间信息，对事件信息进行监测，响应于在预定时间阈值内未再次生成事件信息，生成语音信息。
在一种可选的实现方式中，本实施例的控制方法还包括：响应于在采集初始动作参数之后预定时间阈值内未生成所述事件信息，生成语音信息。
本发明实施例通过传感器组采集用户的动作参数，然后通过降噪装置进行降噪后，处理器从经过降噪处理后的数据中提取状态识别特征向量，根据状态识别特征向量确定对应的事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
图7是本发明第七实施例的电子设备的动作识别方法的流程图。如图7所示，本发明实施例的电子设备的动作识别方法包括以下步骤：
步骤S210，实时地采集目标部位的至少一个动作参数。也即，电子设备被设置 在该目标部位处，实时采集目标部位的动作参数。在一种可选的实现方式中，可以采用骨传导传感器或运动传感器采集目标部位的运动参数。应理解，目标部位可以指人体的头部、手臂等部位，目标部位的至少一个动作信息，例如可以是头部的一个动作信息、两个动作信息或者三个动作信息等等，手臂的一个动作信息、两个动作信息或者三个动作信息等等。
步骤S220，根据采集的动作参数确定是否检测到目标部位的至少一个动作信息。在本实施例中，由于动作参数是实时采集的，在一些情况下采集的动作参数不能被确定为一个动作信息。例如，目标部位为手臂，电子设备需要通过电子设备的摆臂动作周期性地统计步数。在其他场景下，例如在打字时，手臂移动会产生动作参数，但是该动作参数并不能被确定为摆臂动作，因此不能视为检测到手臂的一个动作信息。
步骤S230，响应于检测到目标部位的至少一个动作信息，生成目标部位的至少一个事件信息。也即，假设检测到手臂的一个动作信息，则生成手臂的一个摆臂事件信息。容易理解，在未检测到目标部位的动作信息，则不生成事件信息。
本领域技术人员应当理解，上述方法例如通过计算机程序实现，可以将实现上述方法的计算机程序写入电子设备的运动传感器IMU或处理器中。
在一种可选的实现方式中，本实施例的方法还包括：
步骤S240，基于目标部位的至少一个动作事件信息生成至少一个包含事件信息的第一通知信息。其中，第一通知信息也可以为操作指令，运动传感器IMU通过实时采集动作参数，并在检测到动作信息时确定事件信息，根据事件信息确定第一通知信息也即确定与该事件信息匹配的操作指令，以控制电子设备执行对应的操作。
可选的，步骤S240通过计算机程序实现，可以将步骤S240的计算机程序写入电子设备的运动传感器IMU或处理器中。在电子设备(例如智能耳机、智能手表等可穿戴设备)中，电子设备的处理器(例如耳机芯片、手表芯片)可以基于第一通知信息(也即操作指令)而进行相应的处理。
在一种可选的实现方式中，第一通知信息包括生成事件信息的时间信息。其中“目标部位的至少一个事件信息”可以是头部的一个事件信息、两个事件信息或者三个事件信息等等，相应地，可以生成一个第一通知信息、两个第一通知信息或者三个第一通知信息等等。
在一种可选的实现方式中，本实施例的方法还包括：获取初始事件信息，其中，初始事件信息由初始动作参数确定。初始事件信息包括初始时间信息；基于所述初始 时间信息，对事件信息进行监测，当超过预定时间阈值且未生成事件信息，生成语音信息；或者基于所述初始时间信息，对动作事件信息进行监测，当超过预定时间阈值且未获得事件信息的第一通知信息时，生成语音信息。
应理解，上述初始事件信息可以通过例如光学传感器、蓝牙等获得，以耳机作为电子设备的示例，当耳机初始被佩戴在头部，耳机的光学传感器生成上述初始事件信息，或者耳机初始被佩戴在头部，耳机的蓝牙与手机等移动终端连接，生成上述初始事件信息。本实施方式的一个应用场景是，从耳机初始被佩戴在头部，即开始监测头部动作的事件信息的生成时间，如果超过预定时间阈值未生成头部动作的事件信息或者未获得头部动作的事件信息的第一通知信息，则生成语音信息来提醒用户进行头部运动。
尤其是对于真无线蓝牙耳机(相关技术中一种类型的真无线蓝牙耳机包括主耳机和副耳机，主耳机通过蓝牙与手机或音乐播放器等移动终端连接，副耳机与主耳机通过蓝牙连接；另一种类型的真无线蓝牙耳机包括两个入耳式耳机，并不区分主耳机和副耳机)，通过将被写入实现上述方法的计算机程序的运动传感器IMU或处理器配置在主耳机和副耳机中，或者优选地配置在主耳机中，或者通过将被写入实现上述方法的计算机程序的运动传感器IMU或处理器配置在上述两个入耳式耳机中，使得真无线蓝牙耳机的应用场景更丰富，通过佩戴这种真无线蓝牙耳机，用户可以在学习、工作等场景下对头部动作进行检测而不必再佩戴专门的动作检测设备。
在一种可选的实现方式中，本实施例的方法还包括：
步骤S250：基于至少一个第一通知信息，生成至少一个第二通知信息。
步骤S260，将至少一个第二通知信息发送给终端设备，以使得终端设备根据第二通知信息生成至少一个动作完成信息。
步骤S250和步骤S260的方法可以通过计算机程序实现，其中，可以将步骤S210-S260所述的方法的计算机程序写入本实施例的电子设备(例如智能耳机、智能手表等)，将“根据至少一个第二通知信息，生成至少一个动作完成信息”的计算机程序写入第二电子设备(例如手机、便携式计算机等)。
在一种可选的实现方式中，步骤S210-S240所述的方法的计算机程序写入电子设备中的运动传感器IMU中，步骤S250-S260所述的方法的计算机程序写入耳机的处理器中，将“根据至少一个第二通知信息，生成至少一个动作完成信息”的计算机程序写入手机等终端设备中。如果生成了多个第二通知信息，相应地，生成多个动作完 成信息，也即用户完成了多个动作。“动作完成信息”可以是例如弹窗信息、文字记录信息、图表信息等。
图8是本发明第八实施例的电子设备的动作识别方法的流程图。如图8所示，本实施例的电子设备的动作识别方法包括以下步骤：
步骤S310，实时地采集目标部位的至少一个动作参数。也即，电子设备被设置在该目标部位处，实时采集目标部位的动作参数。
步骤S320，根据采集的动作参数确定是否检测到目标部位的至少一个动作信息。
步骤S330，响应于检测到目标部位的至少一个动作信息，生成目标部位的至少一个事件信息。
容易理解，步骤S310-步骤S330与本发明第七实施例中的步骤S210-步骤S230类似，在此不再赘述。
步骤S340，对目标部位的各类型的事件信息进行数量统计。可选的，在本实施例中，可以将步骤S340所述的方法的计算机程序写入电子设备的处理器，例如耳机处理器、手表处理器等。
作为一个示例，可以通过耳机的运动传感器IMU来检测头部水平位移和/或头部垂直位移，生成头部水平位移动作信息，通过图8所示的方法对头部水平位移动作信息进行实时检测，生成头部水平位移动作事件信息，耳机的处理器对“头部水平位移这个事件信息”进行统计，从而可以实现耳机的计步功能。本领域技术人员应当理解，通过图8示出的方法，还可以统计例如“头部向左转40度的事件信息”、“手臂水平位移的事件信息”等的数量。
在一种可选的实现方式中，本实施例的方法还包括：
步骤S350，将各类型的事件信息的数量统计发送给终端设备以使得终端设备可以对该数据统计进行优化，得到优化后的相同类型的事件信息的数量统计信息。
作为一个示例，以终端设备为手机为例，同一用户的智能耳机和智能手表将各自的计步数量发送至该用户的手机，对两者的计步数量进行优化，可以提高计步的准确性，简单的，可以取两者计步数量的平均值，当然还可以通过各种数据融合算法来对两者的计步数量进行优化。
根据本发明实施例的一个优选实施方式，在检测目标部位的至少一个动作信息之前，本实施例的方法还包括：预设目标部位的至少一个动作类型以及所述至少一个动作类型对应的至少一个动作类型特征信息。由此，在本实施例中，根据采集的动作参 数获取对应的状态识别特征信息，将获取的状态识别特征信息与预先存储的动作类型特征信息进行比对，则可以确定是否检测到动作信息，并基于检测到的动作信息生成事件信息。
可选的，动作类型可以包括头部向左转40度，头部向右转40度，头部向后仰20度等等，这些动作类型会对应相应的动作类型特征信息。动作类型特征信息也即每个动作类型对应的特征信息，这些特征信息可以基于运动传感器IMU检测到的动作参数而生成。
将实时检测到的动作信息的状态识别特征信息与上述至少一个动作类型特征相相比较，如果检测到的动作信息与其中一个动作类型相吻合，说明目标部位(例如头部)进行了一个动作类型，即发生了一个类型的动作，生成事件信息。
根据本发明实施例的一个优选实施方式，在检测目标部位的至少一个动作信息之前，本实施例的方法还包括：预设至少一个综合动作类型以及至少一个综合动作类型对应的至少一个综合动作类型特征信息；综合动作类型包括至少两个目标部位中的各个目标部位的至少一个动作类型；综合动作类型特征信息包括至少两个目标部位中的各个目标部位的至少一个动作类型特征信息。
作为一个示例，一个综合动作类型例如一个瑜伽动作，该瑜伽动作包括一个头部动作类型(头部向左转40度)和一个手臂动作类型(例如手臂由竖直到水平)。则该综合动作类型特征信息包括上述头部动作的特征信息和上述手臂动作的特征信息。
应理解，“预设至少一个综合动作类型”可以预设一个综合动作类型、两个综合动作类型或者三个综合动作类型等等。
根据本发明实施例的一个优选实施方式，在用户佩戴的各电子设备(例如智能耳机、智能手表等)均向对应的终端设备发送第二通知信息后，终端设备可以生成各第二通知信息对应的动作完成信息，并基于各个目标部位的至少一个动作类型对应的动作完成信息，生成各个综合动作类型对应的动作完成信息。通过本发明实施例前面描述的动作识别方法可知，目标部位的一个动作类型对应的动作信息被检测到时，会生成一个事件信息，该事件信息会导致一个包含该事件信息的第一通知信息产生，该第一通知信息会导致一个第二通知信息产生，该第二通知信息会导致一个动作完成信息产生，例如当“头部向左转40度”这一动作类型对应的动作信息被检测到时，会导致一个“头部向左转40度”这一动作类型被完成的动作完成信息。可以理解，当一个综合动作类型包括的各个动作类型(例如头部向左转40度和手臂由竖直到水平)对 应的动作信息都被检测到时，会生成括号中两个动作类型的动作完成信息，基于这两个动作完成信息，生成该综合动作类型的动作完成信息。显然，当括号中的两个动作类型有一个未被完成时，则不会生成该综合动作类型的动作完成信息。
本发明实施例通过实时地检测目标部位的至少一个动作参数，并根据采集的动作参数确定是否检测到目标部位的至少一个动作信息，响应于检测到目标部位的至少一个动作信息，生成目标部位的至少一个事件信息，使得电子设备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
图9是本发明第九实施例的电子设备的动作识别方法的流程图。在本实施例中，以电子设备对应的目标部位为头部进行具体说明，如图9所示，本发明实施例的电子设备的动作识别方法包括以下步骤：
步骤S410，实时地采集动作参数。在一种可选的实现方式中，可以采用骨传导传感器或运动传感器采集目标部位的运动参数。
步骤S420，根据采集的动作参数确定是否检测到头部动作信息。在本实施例中，由于动作参数是实时采集的，在一些情况下采集的动作参数不能被确定为一个动作信息，因此在本实施例中，根据采集的动作参数检测是否为头部动作信息，以提高动作识别的准确性。
步骤S430，响应于检测到头部动作信息，生成头部动作的事件信息。容易理解，在未检测到头部动作信息，则不生成头部动作的事件信息。
步骤S410-S430所述的方法可以通过计算机程序实现，可以将实现上述方法的计算机程序写入电子设备的运动传感器IMU或处理器中。
在一种可选的实现方式中，本实施例的方法还包括：
步骤S440，根据头部动作的事件信息生成包含该头部动作的事件信息的通知信息。其中，通知信息也可以为操作指令，运动传感器IMU通过实时采集动作参数，并在检测到头部动作信息时确定头部动作的事件信息，根据头部动作的事件信息确定通知信息也即确定与该事件信息匹配的操作指令，以控制电子设备执行对应的操作。
可选的，步骤S440通过计算机程序实现，可以将步骤S440的计算机程序写入电子设备的运动传感器IMU或处理器中。在电子设备(例如智能耳机、智能手表等可穿戴设备)中，电子设备的处理器(例如耳机芯片、手表芯片)可以基于第一通知信息(也 即操作指令)而进行相应的处理。
在一种可选的实现方式中，检测头部的至少一个动作信息之前，本实施例的方法还包括：预设至少一个头部动作类型以及所述至少一个头部动作类型对应的至少一个头部动作类型特征信息；实时地检测动作信息；判断检测到的动作信息是否归属于所述至少一个头部动作类型特征信息；如果检测到的动作信息归属于所述至少一个头部动作类型特征信息，则生成头部动作的事件信息，如果检测到的动作信息不归属于所述至少一个头部动作类型特征信息，则不生成头部动作的事件信息。
可选的，头部动作类型可以包括头部向左转40度，头部向右转40度，头部向后仰20度等等，这些头部动作类型会对应相应的头部动作类型特征信息。头部动作类型特征信息，即每个头部动作类型对应的特征动作信息，这些特征动作信息可以基于运动传感器IMU检测到的动作参数而生成。
将实时检测到的动作信息与上述至少一个头部动作类型特征信息相比较，如果检测到的动作信息与其中一个头部动作类型特征信息相吻合，说明头部进行了一个头部动作类型，即发生了一个类型的头部动作，生成头部动作的事件信息。
在一种可选的实现方式中，本实施例的通知信息中包括时间信息，本实施例的方法还包括：
步骤S450，基于通知信息中的时间信息，对该通知信息对应的头部动作事件信息进行监测，响应于超过预设时间阈值未执行上述通知信息且未获得新的通知信息，生成语音信息。
例如，一个通知信息对应的头部动作事件信息为“头部向左转动40度”，该通知信息中包含了发生“头部向左转动40度”的时间信息，即发生时间，该方法从该发生时间开始对“头部向左转动40度”这一头部动作事件信息进行监测，当超过预定时间段阈值未获得该头部动作事件信息的通知信息时，生成语音信息。也就是说，当用户佩戴本实施例的电子设备时实施了“头部向左转动40度”这一头部动作，但在预定时间阈值内未获得“头部向右移动”的头部动作信息，则电子设备无法判定用户是否在进行摇头操作，因此可以通过语音提醒用户。本领域技术人员应当理解，“预定时间段阈值”即预设的一个时间段阈值，例如10秒、20秒、30秒等，本领域技术人员能够对其进行合理设定。
可选的，步骤S410-S440所述的方法的计算机程序可以写入运动传感器IMU或处理器中，步骤S450所述的方法的计算机程序写入电子设备的处理器例如耳机芯片 中，耳机芯片执行该部分计算机程序，生成语音信息，并将语音信息发送给耳机的音频解码装置。本领域技术人员应当理解，语音信息可以是数字音频信息。
在一种可选的实现方式中，步骤S440获取的通知信息为首次通知信息，本实施例的方法还包括：基于首次通知信息中的时间信息，对各个头部动作事件信息进行监测，当超过各预定时间段阈值且未获得各个头部动作事件信息的通知信息，生成相应的语音信息。
其中，“首次通知信息”即该方法检测到的第一个头部动作信息生成的头部动作事件信息对应的通知信息。例如用户佩戴上电子设备(例如智能耳机)后，第一个头部动作信息生成的头部动作事件信息对应的通知信息，例如用户佩戴后，第一个头部动作为“头部向左转动40度”，该动作会导致生成相应的头部动作事件信息以及相应的通知信息，则通知信息中包括了该第一个头部动作的时间信息(即发生时间)，从该发生时间开始，持续对各个头部动作事件信息进行监测，例如对“头部向左转动40度”、“头部向右转动40度”、“头部向后仰20度”对应的各个头部动作事件信息的通知信息进行监测，如果超过各个“预定时间段阈值”且未生成各个通知信息，则生成各个相应的语音信息，各个语音信息可以提醒用户做上面三个动作。本领域技术人员应当理解，各个“预定时间段阈值”可以相同也可以不相同，优选地，为了使得程序的执行便利，各个“预定时间段阈值”不相同。
在本发明实施例的又一个实施方式中，本实施例的方法还包括：获取初始头部信息，初始头部信息包括初始时间信息；基于所述初始时间信息，对头部动作事件信息进行监测，当超过预定时间段阈值且未生成头部动作事件信息，生成语音信息；或者基于所述初始时间信息，对头部动作事件信息进行监测，当超过预定时间段阈值且未获得头部动作事件信息的通知信息时，生成语音信息。
本领域技术人员应当理解，上述初始头部信息可以通过例如光学传感器、蓝牙等获得，例如当耳机初始被佩戴在头部，耳机的光学传感器生成上述初始头部信息，或者耳机初始被佩戴在头部，耳机的蓝牙与手机等移动终端连接，生成上述初始头部信息。本实施方式的一个应用场景是，从耳机初始被佩戴在头部，即开始监测头部动作事件信息的生成时间，如果超过预定时间段阈值未生成头部动作事件信息或者未获得头部动作事件信息的通知信息，则生成语音信息来提醒用户进行头部运动。
本发明实施例通过实时地检测动作参数，并根据采集的动作参数确定是否检测到头部动作信息，响应于检测到头部动作信息，生成头部动作的事件信息，使得电子设 备识别用户的事件信息后，执行对应的操作功能。由此，可以使得电子设备自身的功能更加丰富，还可以和其他终端设备建立通信连接，在用户不方便直接控制其他终端设备的情况下，采集用户的某些控制指令，用以控制其他终端设备为用户更好的服务。
流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现特定逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本公开的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本公开的实施方式所属技术领域的技术人员所理解。处理器执行上文所描述的各个方法和处理。例如，本公开中的方法实施方式可以被实现为软件程序，其被有形地包含于机器可读介质，例如存储器。在一些实施方式中，软件程序的部分或者全部可以经由存储器和/或通信接口而被载入和/或安装。当软件程序加载到存储器并由处理器执行时，可以执行上文描述的方法中的一个或多个步骤。备选地，在其他实施方式中，处理器可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行上述方法之一。
在流程图中表示或在此以其他方式描述的逻辑和/或步骤，可以具体实现在任何可读存储介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。
就本说明书而言，“可读存储介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。可读存储介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式只读存储器(CDROM)。另外，可读存储介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在存储器中。
应当理解，本公开的各部分可以用硬件、软件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件来实现。例如，如果用硬件来实现，和在另一实施方式中一样，可用本领域公知的下 列技术中的任一项或他们的组合来实现：具有用于对数据信息实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
本技术领域的普通技术人员可以理解实现上述实施方式方法的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种可读存储介质中，该程序在执行时，包括方法实施方式的步骤之一或其组合。
此外，在本公开各个实施方式中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个可读存储介质中。所述存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本发明的优选实施例，并不用于限制本发明，对于本领域技术人员而言，本发明可以有各种改动和变化。凡在本发明的精神和原理之内所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (22)

  1. 一种电子设备，其特征在于，所述电子设备包括：
    传感器组，被配置为采集用户的动作参数，所述传感器组包括加速度传感器；
    降噪装置，被配置为对输入信号进行降噪处理，获取经过降噪处理后的数据，所述输入信号根据所述动作参数确定；以及
    处理器，被配置为根据所述降噪处理后的数据提取状态识别特征向量，并根据所述状态识别特征向量确定事件信息。
  2. 根据权利要求1所述的电子设备，所述输入信号为所述动作参数。
  3. 根据权利要求1或2所述的电子设备，其特征在于，所述事件信息为用户的动作或用户的语音信息，所述处理器还被配置为确定与所述用户的动作对应的操作指令。
  4. 根据权利要求3所述的电子设备，其特征在于，所述处理器还被配置为控制所述电子设备执行与所述操作指令相对应的操作；或者
    所述处理器被配置为控制所述电子设备的发送器将所述操作指令发送至对应的终端设备，以控制所述终端设备执行与所述操作指令对应的操作。
  5. 根据权利要求3所述的电子设备，其特征在于，所述处理器还被配置为响应于所述用户的动作为运动姿态，周期性地统计与所述用户的动作对应的事件信息。
  6. 根据权利要求5所述的电子设备，其特征在于，所述电子设备中的发送器受控将所述运动数据同步至对应的终端设备。
  7. 根据权利要求1-6中任一项所述的电子设备，其特征在于，所述处理器被配置为采用分窗特征提取方法从所述经过降噪处理后的数据中提取所述状态识别特征向量。
  8. 根据权利要求1所述的电子设备，其特征在于，所述传感器组还包括麦克风，所述麦克风被配置为采集外界发出的声音信号；
    所述电子是设备还包括：
    声音活动检测装置，被配置为在接收到所述传感器组中的预设传感器传输的语音信号时，唤醒所述传感器组中除所述预设传感器之外的传感器；
    其中，所述加速度传感器或所述麦克风被设置为预设传感器。
  9. 根据权利要求8所述的电子设备，其特征在于，所述电子设备还包括声音增 强处理装置、声音融合装置以及回声消除装置；
    所述回声消除装置被配置为接收所述麦克风发送的声音信号，并对所述声音信号进行回声消除处理；
    所述声音增强处理装置被配置为将经过回声消除处理后的声音信号进行降噪处理，以增强声音信号；
    声音融合装置被配置为采用自适应滤波方法将增强后的声音信号和传感器组采集的动作参数进行融合处理，获取所述输入信号。
  10. 根据权利要求1所述的电子设备，其特征在于，所述处理器还被配置为根据所述事件信息和本地热词库唤醒所述电子设备，所述本地热词库包括至少一个热词，所述至少一个热词包括至少一个语音信息和/或至少一个用户动作，所述电子设备在识别到对应的热词后被唤醒。
  11. 根据权利要求1所述的电子设备，其特征在于，所述处理器还被配置为根据所述事件信息和本地热词库确定对应的热词，并对所述对应的热词进行二次验证以唤醒所述电子设备。
  12. 根据权利要求3所述的电子设备，其特征在于，所述处理器根据所述状态识别特征向量、预设的动作类型及对应的动作类型特征向量确定所述用户动作。
  13. 根据权利要求3所述的电子设备，其特征在于，所述处理器被配置为根据生成所述事件信息的时间信息，对所述事件信息进行监测，响应于在预定时间阈值内未再次生成所述事件信息，生成语音信息。
  14. 根据权利要求1所述的电子设备，其特征在于，所述处理器还被配置为响应于在采集初始动作参数之后预定时间阈值内未生成所述事件信息，生成语音信息。
  15. 根据权利要求3所述的电子设备，其特征在于，所述处理器还被配置为响应于在采集初始动作参数之后预定时间阈值内未确定所述操作指令，生成语音信息。
  16. 一种电子设备的控制方法，其特征在于，所述方法包括：
    采集用户的动作参数；
    对输入信号进行降噪处理，获取经过降噪处理后的数据，所述输入信号根据所述动作参数确定；
    根据所述降噪处理后的数据提取状态识别特征向量；以及
    根据所述状态识别特征向量确定事件信息。
  17. 根据权利要求16所述的方法，其特征在于，所述事件信息为用户的动作的 语音信息；
    所述方法还包括：
    确定与所述事件信息对应的操作指令。
  18. 根据权利要求17所述的方法，其特征在于，所述方法还包括：
    控制对应的电子设备执行与所述操作指令向对应的操作；或者
    控制所述电子设备的发送器将所述操作指令发送至对应的终端设备，以控制所述终端设备执行与所述操作指令对应的操作。
  19. 根据权利要求17所述的方法，其特征在于，所述方法还包括：
    响应于所述用户的动作为运动姿态，周期性地统计与所述用户的动作对应的事件信息。
  20. 一种设备系统，其特征在于，所述设备系统包括：
    终端设备；以及
    至少一个如权利要求1-15中任一项所述的电子设备。
  21. 根据权利要求20所述的系统，其特征在于，各所述电子设备设置于不同的目标部位，被配置为确定对应的目标部位的事件信息；
    所述终端设备被配置为对各所述目标部位的相同类型的事件信息的数量统计信息进行优化。
  22. 根据权利要求20所述的系统，其特征在于，所述终端设备基于各所述目标部位的至少一个事件完成信息，生成对应的综合动作类型对应的动作完成信息；
    其中，所述综合动作类型包括至少两个目标部位分别对应的至少一个动作类型。
PCT/CN2019/130593 2019-04-19 2019-12-31 一种电子设备、控制方法及设备系统 WO2020211467A1 (zh)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN201910319222.2 2019-04-19
CN201910319222.2A CN110110616A (zh) 2019-04-19 2019-04-19 一种电子设备及控制方法
CN201910391236.5A CN110245328A (zh) 2019-05-11 2019-05-11 动作识别方法
CN201910391230.8 2019-05-11
CN201910391230.8A CN110262654A (zh) 2019-05-11 2019-05-11 头部动作识别方法、装置、设备及存储介质
CN201910391236.5 2019-05-11

Publications (1)

Publication Number Publication Date
WO2020211467A1 true WO2020211467A1 (zh) 2020-10-22

Family

ID=72838042

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/130593 WO2020211467A1 (zh) 2019-04-19 2019-12-31 一种电子设备、控制方法及设备系统

Country Status (1)

Country Link
WO (1) WO2020211467A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102185963A (zh) * 2011-03-03 2011-09-14 曾超宁 耳道控制装置
CN106993243A (zh) * 2016-02-03 2017-07-28 深圳市汇顶科技股份有限公司 一种基于耳机的智能设备控制方法、装置及系统
CN108509048A (zh) * 2018-04-18 2018-09-07 黄忠胜 一种智能设备的操控装置及其操控方法
CN108540669A (zh) * 2018-04-20 2018-09-14 Oppo广东移动通信有限公司 无线耳机、基于耳机检测的控制方法及相关产品
CN109088981A (zh) * 2018-07-10 2018-12-25 奇酷互联网络科技（深圳）有限公司 低头提醒方法、装置、可读存储介质及移动终端
CN110110616A (zh) * 2019-04-19 2019-08-09 出门问问信息科技有限公司 一种电子设备及控制方法
CN110245328A (zh) * 2019-05-11 2019-09-17 出门问问信息科技有限公司 动作识别方法
CN110262654A (zh) * 2019-05-11 2019-09-20 出门问问信息科技有限公司 头部动作识别方法、装置、设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102185963A (zh) * 2011-03-03 2011-09-14 曾超宁 耳道控制装置
CN106993243A (zh) * 2016-02-03 2017-07-28 深圳市汇顶科技股份有限公司 一种基于耳机的智能设备控制方法、装置及系统
CN108509048A (zh) * 2018-04-18 2018-09-07 黄忠胜 一种智能设备的操控装置及其操控方法
CN108540669A (zh) * 2018-04-20 2018-09-14 Oppo广东移动通信有限公司 无线耳机、基于耳机检测的控制方法及相关产品
CN109088981A (zh) * 2018-07-10 2018-12-25 奇酷互联网络科技（深圳）有限公司 低头提醒方法、装置、可读存储介质及移动终端
CN110110616A (zh) * 2019-04-19 2019-08-09 出门问问信息科技有限公司 一种电子设备及控制方法
CN110245328A (zh) * 2019-05-11 2019-09-17 出门问问信息科技有限公司 动作识别方法
CN110262654A (zh) * 2019-05-11 2019-09-20 出门问问信息科技有限公司 头部动作识别方法、装置、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI, MINJIE: "Bone Conduction Combined with Air Conduction to Structure Speech Enhancement System", CHINA MASTER’S THESES FULL-TEXT DATABASE, 15 February 2017 (2017-02-15), DOI: 20200314150727Y *

Similar Documents

Publication Publication Date Title
CN108735219B (zh) 一种声音识别控制方法及装置
CN103929835B (zh) 闹钟振铃的控制方法、装置和系统
WO2020207376A1 (zh) 一种去噪方法及电子设备
JP6812604B2 (ja) オーディオアクティビティ追跡および要約
WO2017113768A1 (zh) 一种耳机的控制方法、控制系统和耳机
WO2020088158A1 (zh) 一种耳机及其播放方法
CN111432303B (zh) 单耳耳机、智能电子设备、方法和计算机可读介质
CN1761265A (zh) 移动设备上的多传感语音增强方法和装置
JP5458027B2 (ja) 次発話者誘導装置、次発話者誘導方法および次発話者誘導プログラム
JP6164076B2 (ja) 情報処理装置、情報処理方法、およびプログラム
CN109032384A (zh) 音乐播放控制方法、装置及存储介质和穿戴式设备
CN111475206A (zh) 用于唤醒可穿戴设备的方法及装置
CN110234044A (zh) 一种语音唤醒方法、语音唤醒装置及耳机
US11895474B2 (en) Activity detection on devices with multi-modal sensing
JP6258172B2 (ja) 音情報処理装置及びシステム
CN111028838A (zh) 一种语音唤醒方法、设备及计算机可读存储介质
US20230239800A1 (en) Voice Wake-Up Method, Electronic Device, Wearable Device, and System
CN109831817A (zh) 终端控制方法、装置、终端及存储介质
WO2020211467A1 (zh) 一种电子设备、控制方法及设备系统
WO2017029850A1 (ja) 情報処理装置、情報処理方法およびプログラム
WO2019198299A1 (ja) 情報処理装置及び情報処理方法
CN108399061A (zh) 语音播放速度的控制方法及系统
CN106713641B (zh) 一种基于穿戴设备的报时方法及穿戴设备
US11997445B2 (en) Systems and methods for live conversation using hearing devices
CN113810819B (zh) 一种基于耳腔振动的静默语音采集处理方法及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19925403

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 04.02.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19925403

Country of ref document: EP

Kind code of ref document: A1