CN113223508B

CN113223508B - Management method of dual-mode TWS Bluetooth headset

Info

Publication number: CN113223508B
Application number: CN202110336581.6A
Authority: CN
Inventors: 鲁鹏飞; 鲁霖; 姚放
Original assignee: Shenzhen Xinzhongxin Technology Co Ltd
Current assignee: Shenzhen Xinzhongxin Technology Co Ltd
Priority date: 2021-03-29
Filing date: 2021-03-29
Publication date: 2023-08-04
Anticipated expiration: 2041-03-29
Also published as: CN113223508A

Abstract

The application relates to a management method of a dual-mode TWS Bluetooth headset, which comprises a state judging step for determining a communication mode of the headset, an information obtaining step for obtaining sound source information, an information concentration step for concentrating sound source information of two headsets, a feature extracting step for extracting sound source information features, a signal transmission step for transmitting the sound source information features to a mobile phone, a mode judging step for analyzing an environmental mode where the sound source information features are located, and a signal processing step for processing the sound source information based on the environmental mode to obtain more accurate standard features. The sound information judging method has the effect of improving judging efficiency of the system based on the sound information obtained by the Bluetooth headset.

Description

Management method of dual-mode TWS Bluetooth headset

Technical Field

The application relates to the technical field of communication equipment, in particular to a management method of a dual-mode TWS Bluetooth headset.

Background

At present, people are not satisfied with simple listening and voice functions brought by Bluetooth headphones, and needs to be diversified, complicated and scenerised. For example, when a user runs on a road, the user is free to use the mobile phone to operate, voice control is performed by using the Bluetooth headset, but the environment where the user runs is usually noisy, the microphone can acquire pedestrian noise, automobile noise, wind noise and the like at the same time, and the accuracy of voice control is usually not high. For example, some headphones have a heart beat abnormal alarm function, but heart functions of some old people are weak, heart rate differences of the old people are large under different situations, for example, compared with young people, the difference of the heart rate of the old people is large when sleeping and when going upstairs, and large errors are easy to generate if the headphones are used for simple judgment.

For the above related art, the inventor believes that if the information directly collected by the earphone is directly judged in a related manner without performing the standardized processing, for example, when the earphone acquires the voice information of a person in a noisy exercise environment to control the mobile phone application, a larger failure probability is generated.

Disclosure of Invention

In order to improve the judging efficiency of the system based on sound information acquired by the Bluetooth headset, the application provides a management method of a dual-mode TWS Bluetooth headset.

The management method of the dual-mode TWS Bluetooth headset provided by the application adopts the following technical scheme:

a management method of a dual-mode TWS Bluetooth headset comprises the following steps:

a state judgment step: the left earphone and the right earphone acquire wireless connection strength of each other, and an earphone communication mode is selected based on the wireless connection strength, wherein the earphone communication mode comprises a working mode and a searching mode;

an information acquisition step, in which two earphones acquire first sound source information based on bone conduction and second sound source information based on air conduction in a working mode;

an information centralizing step of centralizing the first sound source information and the second sound source information in a single earphone;

a feature extraction step, namely extracting environmental features and target features by a single earphone based on the first sound source information and the second sound source information, wherein the environmental features are audio features of environmental sounds, and the target features are audio features of sounds made by a human body;

A signal transmission step, namely, the environmental characteristics and the target characteristics are sent to the mobile phone through Bluetooth signals;

a mode judging step of reading environment characteristics and selecting a local judging mode or a cloud judging mode based on the mobile phone networking state and the training degree of a local environment acoustic model; the local judging mode is to use a local environment acoustic model to carry out environment judgment, and the cloud judging mode is to upload the environment characteristics to the cloud by using a mobile phone, and the cloud environment acoustic model carries out environment judgment on the environment characteristics; the local environment acoustic model is a neural network model trained by environment features acquired by the mobile phone, and the cloud environment acoustic model is a neural network model trained by environment acoustic models with standard training degrees on all networking mobile phones;

a signal processing step, wherein the mobile phone processes the target feature based on the environment type obtained in the mode judging step to obtain a standard feature, wherein the standard feature is a feature which is influenced by environmental sound lower than the target feature;

and executing step, comparing the standard characteristic with the first threshold characteristic, and executing a preset operation instruction according to the comparison result.

By adopting the technical scheme, the earphone is in various states, such as a state that two earphones are separated from each other and far away from each other, or a state that two earphones are put in an earphone box, or a state that two earphones are normally worn on ears, and the two earphones are required to be classified to carry out corresponding communication modes. The operation mode corresponds to a state in which two headphones are worn on ears. The first sound source information is information acquired by bone conduction, has the characteristics of low noise and high fitting degree with the voice information, and the second sound source information is information acquired by air conduction, and the information carries the voice information and the environment information at the same time. The two earphones concentrate and process the sound source information into a single earphone, after extracting information such as frequency spectrum and the like, the environment characteristics and the target characteristics in the information can be extracted by utilizing a differential algorithm or other methods, the coherence of the environment characteristics and the target characteristics is small, compared with the arrangement of an independent noise microphone and a human voice microphone, the obtained audio characteristics are better, the judgment efficiency of the system is improved, but the target characteristics still have the influence of environment and need to be further processed. In addition, the signals acquired by the two earphones can be processed correspondingly, such as averaging, so as to improve the accuracy of the environmental characteristics and the target characteristics.

After the environmental features and the target features are transmitted to the mobile phone, the mode judging step invokes the environmental acoustic model to judge the current environment of the user. The environmental acoustic model is a model which is obtained by teaching various single environmental features, can be a multi-classification neural network model and the like, and can judge the environmental type corresponding to the environmental features after the corresponding environmental features are input. The judgment accuracy of the environmental acoustic model is affected by the size of the training sample, and as the environmental acoustic model in each mobile phone is affected by the voice of a holder, the model has specificity and needs to be used for a long time and trained. Thus, the accuracy of the local environmental acoustic model will be higher than that of the cloud for long term use in terms of accuracy. And proper environmental acoustic models are selected, so that the accuracy of environmental judgment is improved.

The mobile phone is pre-stored with various processing models corresponding to the environment types, after the environment types are judged by the mode judging step, the mobile phone calls the corresponding processing models according to the environment types to process the target features so as to further take out the influence of the environment in the target features, thereby obtaining standard features, comparing the standard features with preset first threshold features, and executing operation instructions preset in the mobile phone according to the comparison results.

Preferably, the state judging step is preceded by a master-slave determining step:

step one: the left earphone and the right earphone are connected through NFMI wireless;

step two: the left earphone and the right earphone respectively acquire the self electric quantity and compare the self electric quantity with each other, when the difference value is larger than or equal to a preset threshold value, the earphone with high electric quantity is used as a main earphone to enter an NFMI information receiving mode and a Bluetooth information output mode, and the earphone with low electric quantity is used as a slave earphone to enter the NFMI signal output mode and the Bluetooth information receiving mode;

the information centralizing step includes:

the earphone with low electric quantity sends the acquired first sound source information and second sound source information to the earphone with high electric quantity through the NFMI signal;

the signal transmission steps are as follows: the earphone with high electricity sends the environmental characteristic and the target characteristic to the mobile phone at fixed time through Bluetooth signals.

By adopting the technical scheme, the first sound source information and the second sound source information need to be continuously collected due to the instantaneity of environment judgment and voice judgment. If a Bluetooth connection is adopted between two earphones, one earphone continuously receives Bluetooth information, and the other earphone continuously sends out the Bluetooth information, so that the earphone in the output mode consumes relatively fast power. Because the two earphones are closer in distance when in use, NFMI with lower energy consumption can be used for information transfer. In theory, NFMI is more energy efficient than bluetooth radio frequency at short range transmissions, with the transmission range depending on the antenna size. The signal strength drops less as the NFMI signal passes through human tissue, while the signal strength of bluetooth radio frequency drops significantly. The specific absorption rate of NFMI is 10000 times lower than bluetooth radio frequency, so NFMI can perform long-time low-power communication between two headphones. Because the distance between the mobile phone and the earphone is not fixed, the Bluetooth is more suitable for wireless connection between the earphone and the mobile phone than the NFMI. Headphones for NFMI reception need to send the extracted features out through bluetooth, so the power consumption of the headphones will be higher than that of the other headphones, and the headphones are switched between an information receiving mode and an information output mode by performing power judgment, so as to balance the power consumption of the two headphones.

In addition, because the extraction of the sound source information characteristics is already realized in the earphone, the Bluetooth signal between the mobile phone and the earphone needs to bear much less information than the information corresponding to the complete audio. The transmission rate of bluetooth is limited, for example, the transmission rate of bluetooth 4.0 is 1Mbps, and the time slot generated by transmitting the sound source information characteristic is far greater than transmitting the complete audio, so that interval communication can be realized in the scheme, and the power consumption is lower.

Preferably, the NFMI wireless connection is unencrypted communication, and the bluetooth connection of the earphone and the mobile phone is symmetric key communication.

By adopting the technical scheme, the effective transmission radius of the NFMI is about 1m, the NFMI has good safety, and information is difficult to be stolen remotely, so that the communication efficiency can be improved well by using unencrypted communication, and the operation amount is reduced. The information quantity of the environment characteristic and the target characteristic is less, and the symmetric key encryption mode has less influence on the transmission of information.

Preferably, the mode judging step includes:

s1, reading environmental characteristics;

s2, acquiring a current networking state and evaluating the training degree of a local environment acoustic model, and if the current networking state is in an offline state and the training degree of the local environment acoustic model does not reach the standard, entering S21; if the training degree of the local environmental acoustic model is not up to the standard and is in the on-line state currently, S22 is entered; if the training degree of the local environmental acoustic model meets the standard, S23 is entered;

S21, calling a locally prestored initial acoustic model and adjustment parameters corresponding to various environments, and initializing a transformation function of the initial acoustic model into a transformation function corresponding to each working environment based on the adjustment parameters; judging environmental characteristics by using an environmental acoustic model generated by each transformation function and outputting an environmental type;

s22, uploading the environmental characteristics to a cloud end by the mobile phone, judging the environmental characteristics by an environmental acoustic model of the cloud end, outputting the environmental types and returning to the mobile phone;

s23, calling a local pre-trained environmental acoustic model, judging environmental characteristics by using the environmental acoustic model and outputting the environmental type; the environmental acoustic model is trained using environmental features.

Through adopting above-mentioned technical scheme, the environment acoustic model of high in the clouds is trained through the ripe environment acoustic model of each local cell-phone training, and the judgement rate of environment acoustic model is relevant with training degree, training sample, therefore, the environment acoustic model of high in the clouds comparatively pervades, and the local environment acoustic model of full training has higher matching degree to the user of cell-phone. When the local environment acoustic model is fully trained, the local environment acoustic model is preferentially selected for judgment, then the cloud environment acoustic model is called in a networking state, and the parameter tuning simulation is performed for the initialized acoustic model for the first time. The mobile phone end can also set a backup database, and after each networking, the adjustment parameters of the environmental acoustic model of the cloud end are selected to be downloaded and stored for being used as the adjustment parameters.

Preferably, the step of judging the environmental characteristics by the acoustic model includes:

step one: establishing posterior probability of each modeling unit according to the environmental characteristics and the local environmental acoustic model;

step two: compressing and smoothing the posterior probability of the modeling units to obtain posterior probability after the modeling units are processed;

step three: and decoding the posterior probability of all frames of the environmental characteristics after being processed by the modeling unit to obtain the environmental type.

In an actual working environment, the working environments are different, and corresponding background noise is different, for example, engine noise with automobile discontinuity on a highway, walking sound on a sidewalk, business street advertising sound and the like. Under different working environments, the characteristics of training samples corresponding to the models are not completely the same, so that the transformation function adjusted by training data corresponding to one working environment is not applicable to another working environment, and therefore, different transformation functions are needed to build different neural network models or build multi-classification neural network models. If the model is trained again by adjusting the training data corresponding to the working environment, so as to obtain a new transformation function, the required training data amount will be more, and the training period will be prolonged. Therefore, the scheme selects to input the environmental characteristics first and establish the posterior probability of each modeling unit, and 0/1 of the posterior probability has more mutation at the moment. And then compressing and smoothing the posterior probability of the modeling unit, so that 0/1 mutation of the posterior probability is reduced, an effect similar to large sample training is generated, the upper and lower jitter ranges of the posterior probability are smaller than the fluctuation range of the posterior probability of the modeling unit before processing, and the coverage of the candidate sequence of the posterior probability of the modeling unit on the correct environmental characteristic recognition result is increased. And then decoding the posterior probability processed by the modeling unit of all frames of the environmental characteristics, and outputting the environmental type.

Preferably, the step of evaluating the training degree of the local environmental acoustic model in S2 includes an adjustment step and a scoring step, and the adjustment step includes:

step one: the method comprises the steps of calling a locally prestored initial acoustic model and adjustment parameters corresponding to various environments, and initializing a transformation function of the initial acoustic model into a transformation function corresponding to each working environment based on the adjustment parameters to obtain a reference acoustic model;

step two: acquiring posterior probability of each modeling unit corresponding to a transformation function of the reference acoustic model;

step three: initializing transformation function parameters corresponding to a local model, and acquiring posterior probability of each modeling unit corresponding to the initialized local model, wherein the local model is a local pre-trained environmental acoustic model;

step four: calculating posterior probability distribution distances between each modeling unit of the local model and each modeling unit of the reference acoustic model, and adjusting transformation function parameters corresponding to the working environment according to the distances to obtain optimal transformation function parameters of the local model aiming at the corresponding working environment;

step five: according to the obtained optimal transformation function parameters, the transformation function of the corresponding local model is adjusted;

The scoring step includes:

step one: reading standard environmental characteristics pre-stored in a mobile phone;

step two: and (3) adjusting the corresponding local model by using the optimal transformation function parameters to score the standard environmental characteristics, judging that the local model meets the standard if the score is higher than a second fixed value, and otherwise, judging that the local model does not meet the standard.

By adopting the technical scheme, when the training samples of the pre-trained environmental acoustic model are insufficient in local training, the posterior probability distribution of the modeling unit of the output node is sharp, and the recognition failure rate of the environmental scene can be amplified. And (3) approaching and optimizing the local pre-trained environmental acoustic model to a preset reference acoustic model, so that the posterior probability histogram of the modeling unit of the output node is smoother, and the influence of a small sample can be reduced. If the score of the optimized local model in the scoring step is also lower than the second fixed value, the score of the local model before optimization is also lower than the second fixed value, and the model is judged to not reach the standard.

Preferably, the method further comprises the following steps:

a path establishment step: the method comprises the steps that a left earphone and a right earphone are divided into a receiving earphone and a listening earphone based on detection of Bluetooth signals of a mobile phone, in a searching mode, the listening earphone sends a broadcast packet through a transfer device or the receiving earphone based on a tree topology structure, wherein the tree topology structure takes a Bluetooth device as a host, other Bluetooth devices in a preset Bluetooth transmission signal strength range corresponding to the Bluetooth device are taken as slaves, the broadcast packet carries a forwarding frequency identifier and an address sequence, the address sequence stores an address of the transfer device in a broadcast packet path, and the host selects whether to send the broadcast packet to the slaves or not based on a forwarding frequency identifier and the address sequence of the broadcast packet;

The connection preferably comprises the following steps: the mobile phone or the listening earphone obtains an optimal path based on the received broadcast packet, and constructs a temporary communication path with the listening earphone based on the optimal path, wherein a forwarding frequency sign corresponding to the optimal path is the lowest;

interception step: the mobile phone or the answering earphone sends an activating signal to the interception earphone through the temporary communication path, the interception earphone starts to intercept the nearby sound source information after receiving the activating signal, and sends a corresponding audio signal to the mobile phone or the answering earphone through the temporary communication path, and the mobile phone or the answering earphone receives, analyzes and plays the audio signal.

By adopting the technical scheme, in the practical use, people can choose to wear only one earphone in order to listen to the environment sound and the playing content in the earphone at the same time. The TWS Bluetooth headset is inconvenient because the left headset and the right headset are not connected, so that the TWS Bluetooth headset is difficult to find after the next headset is put down. Because the Bluetooth connection strength between the earphone and the mobile phone is poor when the earphone is far away, the left earphone is disconnected with the mobile phone after a user walks or even changes rooms. At this time, because the environments of the two earphones are different, the acquired environmental sounds are not the same, and the two earphones are not suitable for being used as learning samples and noise reduction.

In order to be able to find out the headphones conveniently, the lost headphones are used as listening headphones to establish a connection with surrounding relay devices and broadcast packets are propagated towards the relay devices, and the relay devices diffuse the broadcast packets to the surrounding relay devices based on preset rules until the relay devices transfer the broadcast packets to the mobile phone or the listening headphones. Since the direction of the broadcast packet is not unique, multiple paths will be generated, and when the communication path is long, the delay of the signal is serious and the transmission effect is relatively poor. And all transit devices continuously transmit and receive broadcast packets, a great deal of energy loss is caused. Therefore, the mobile phone or the answering earphone preferably selects a path with the lowest forwarding frequency from all feasible paths as a temporary communication path, so as to reduce communication delay and participating transit equipment. After the temporary communication path is established, the mobile phone or the listening earphone triggers the listening earphone to listen to the nearby sound source information, the user can send out sound, and the distance between the user and the listening earphone is judged according to the sound received by the earphone until the user enters a room where the listening earphone is located, and Bluetooth connection is reestablished with the listening earphone. Along with the movement of a user, the position relation between the listening earphone or the mobile phone and surrounding transit equipment is adjusted to ensure and improve the communication quality.

Preferably, the state judgment step includes the steps of:

step one: the master earphone acquires the NFMI signal intensity of the slave earphone, compares the signal intensity with the upper limit and the lower limit of a first preset intensity range, and selects a corresponding sub-item entering the next step based on a comparison result;

step two:

the sub-item a. If the signal intensity is greater than the upper limit of the preset intensity range and the variation of the signal intensity is less than the preset threshold value in the preset time period, the master earphone and the slave earphone enter dormancy after the preset time period until the master earphone or the slave earphone detects a wake-up signal, and the step one is returned;

the sub-item b. If the signal intensity is within the preset intensity range, the master earphone and the slave earphone enter into a working mode;

and c, if the signal intensity is smaller than the lower limit of the preset intensity range, entering a path establishment step.

By adopting the technical scheme, as the master earphone and the slave earphone are connected through the NFMI, the signal connection of the NFMI is strong and the attenuation along with the change of the distance is serious, and therefore, the distance between the master earphone and the slave earphone can be judged through the NFMI connection strength of the master earphone and the slave earphone. For bluetooth, need carry out accurate location then need corresponding bluetooth chip, the volume of this type of chip module is great, is difficult to integrate to bluetooth headset.

Sub item a corresponds to the situation that two earphones are taken down and put together, and a user often takes down the two earphones and puts them in a pocket, on a table or on a bed, and the earphones in the related art will continuously lose power. In this scheme, both the master earphone and the slave earphone go to sleep until they are picked up again and receive a wake-up signal. The current main stream earphone is internally provided with corresponding acceleration sensors, and can receive knocking or changing acceleration as a wake-up signal.

The sub-item b corresponds to a normal operating state of the headset, and the third intensity threshold may correspond to a normal human ear distance.

The sub-item c corresponds to a state where the two headphones are far away from each other, a path establishment step may be entered to enter a search mode.

Preferably, the method for selecting whether to send the broadcast packet to the slave machine or not by the host machine based on the forwarding time mark and the address sequence of the broadcast packet comprises the following steps:

the host reads the current forwarding frequency mark of the broadcast packet and judges whether the forwarding frequency is smaller than the allowed maximum forwarding frequency; discarding the broadcast packet if not;

the host reads the current address sequence of the broadcast packet, judges whether the starting address of the current broadcast packet is the address of the transfer equipment where the broadcast packet is currently located, discards the broadcast packet if yes, and writes the address of the transfer equipment where the broadcast packet is currently located into the address sequence if not;

The host reads the current address sequence, the forwarding frequency mark and the history record of the located transfer equipment of the broadcast packet, if the starting address of the broadcast packet exists in the history address and the forwarding frequency of the broadcast packet is larger than the history hop count of the corresponding broadcast packet in the history address, the broadcast packet is abandoned, otherwise, the starting address and the forwarding frequency of the broadcast packet are written into the history record and the broadcast packet is forwarded, wherein the history record comprises the history address and the history hop count, the history address is the starting address of the transfer equipment for historically transferring the broadcast packet, and the history hop count is the corresponding forwarding frequency of the transfer equipment for historically transferring the broadcast packet when the transfer equipment is located in the transfer equipment.

By adopting the technical scheme, because the transfer equipment forwards the broadcast packets in an unordered way, when the forwarding times are not limited, the broadcast packets can be repeatedly forwarded in the whole communication network without interruption, which can cause channel congestion and energy waste. Therefore, the forwarding times are required to be limited, the occupation of the whole bandwidth of the communication network is reduced, and the efficiency of the interception earphone for searching the mobile phone or the interception earphone is improved.

Because the forwarding direction of the broadcast packet is disordered, the host machine of the current tree topology structure is also a slave machine of the adjacent tree topology structure, and therefore, the invalid forwarding of the data packet in the communication network is reduced by judging whether the sender and the receiver of the broadcast packet are consistent to determine whether to discard the data packet. In addition, because the broadcast packet is forwarded unordered, the broadcast packet passes through the same transfer equipment through different paths easily, and the comparison of the forwarding times and the historical hop count on the broadcast packet is adopted to judge whether the path corresponding to the broadcast packet is a longer one, if so, the broadcast packet is discarded, and the invalid forwarding of the broadcast packet in the communication network is reduced.

Preferably, the connecting preferably includes:

step one: the mobile phone acquires the quantity of transfer devices corresponding to all feasible paths from the interception earphone to the mobile phone and takes the quantity as a feasible set;

step two: the mobile phone judges whether the number of the minimum elements in the feasible set is greater than or equal to N, if yes, the feasible path corresponding to the minimum elements in the feasible set is used as a standby path; if the number of the feasible paths is less than N, the feasible paths corresponding to the smallest N elements in the feasible set are used as standby paths, wherein N is a preset number threshold;

step three: the mobile phone calculates the sum of the times that each transfer device passes by other optimal paths in each standby path, and takes the sum as the feature number of the corresponding standby path, and selects the standby path with the minimum feature number as the optimal path.

By adopting the technical scheme, a plurality of feasible paths with the minimum number of hops are selected, and the number of the standby paths selected under the condition that the feasible paths are enough is ensured to be greater than or equal to N as much as possible, so that communication attempts can be conveniently carried out on enough standby paths when the optimal path communication is interrupted, and smooth communication is ensured.

Drawings

Fig. 1 is a flowchart of a method for managing a dual-mode TWS bluetooth headset according to an embodiment of the present application.

Fig. 2 is a flow chart of the mode judgment step in the embodiment of the present application.

Detailed Description

The present application is described in further detail below in conjunction with figures 1-2.

At present, people show the trend of diversification, complexity and scene on the function of the Bluetooth headset, and the demands on the accuracy of voice recognition and signal acquisition are increasing. People have various application scenes of the earphone, various environmental sounds have different influences on the earphone to acquire target signals, for example, when a user runs on the road, the user does not need to use a mobile phone to operate, therefore, a Bluetooth earphone is generally adopted to read in voice to perform a corresponding control method, the environment in which the user is positioned in the running process is generally noisy, a microphone can acquire pedestrian noise, automobile noise, wind noise and the like at the same time, and the accuracy of voice control is generally low. For example, if the earphone directly reads the heart rate to judge the current state of the old person, larger errors are easy to generate.

For example, when the behavior state of the user going upstairs is obtained, the detected heart rate is corrected according to the corresponding algorithm of the state, and the corrected heart rate is used as the judgment data of the physical state of the user, so that the judgment accuracy rate is improved. Or when the user runs outdoors, the earphone records running sound and wheezing sound as characteristic audio, so that the state of the user is judged, and the voice signal of the user is processed pertinently based on the state, so that the voice recognition capability of the earphone can be improved.

The embodiment of the application discloses a management method of a dual-mode TWS Bluetooth headset. Referring to fig. 1, the management method includes the steps of:

master-slave determination:

step two: the left earphone and the right earphone respectively acquire own electric quantity and compare with each other, when the difference value is greater than or equal to a preset threshold value, the earphone with high electric quantity is used as a main earphone to enter an NFMI information receiving mode and a Bluetooth information output mode, and the earphone with low electric quantity is used as a slave earphone to enter the NFMI signal output mode and the Bluetooth information receiving mode.

NFMI, near field magnetic induction, is a near field wireless technology that communicates by coupling a compact, low power, non-propagating magnetic field between devices. A transmitter coil in one device modulates an electromagnetic field that may be measured by a receiver coil in another device.

Because the two earphones are closer in distance when in use, the NFMI with lower energy consumption can be used for information transfer between the earphones. In theory, NFMI is more energy efficient than bluetooth radio frequency at short range transmissions, with the transmission range depending on the antenna size. The signal strength drops less as the NFMI signal passes through human tissue, while the signal strength of bluetooth radio frequency drops significantly. Experimental tests show that the specific absorption rate of NFMI is 10000 times lower than bluetooth, whereas the brain is a good bluetooth radio frequency absorbing organ, so that NFMI can adapt to long-term signaling between two headphones.

A state judgment step: the master earphone acquires the NFMI signal intensity of the slave earphone, compares the signal intensity with the upper limit and the lower limit of a first preset intensity range, and selects to enter corresponding sub-items corresponding to different comparison results;

the sub-item a. If the signal intensity is greater than the upper limit of the preset intensity range and the variation of the signal intensity is less than the preset threshold value in the preset time period, the master earphone and the slave earphone enter dormancy after the preset time period until the master earphone or the slave earphone detects a wake-up signal, and then the state judgment step is carried out again;

Since the master earphone and the slave earphone are connected through the NFMI, the signal connection of the NFMI is strong and the attenuation along with the change of the distance is serious, so that the distance between the master earphone and the slave earphone can be judged through the NFMI connection strength of the master earphone and the slave earphone. For bluetooth, need carry out accurate location then need corresponding bluetooth chip, the volume of this type of chip module is great, is difficult to integrate to bluetooth headset. Therefore, as an implementation mode of the method, the NFMI connection is adopted to measure the distance between the master earphone and the slave earphone, so that the effects of lower cost and smaller size can be achieved.

Sub item a corresponds to the situation that two earphones are taken down and put together, and a user often takes down the two earphones and puts them in a pocket, on a table or on a bed, and the earphones in the related art will continuously lose power. In this scheme, both the master earphone and the slave earphone go to sleep until they are picked up again and receive a wake-up signal. The current main stream earphone is internally provided with corresponding acceleration sensors, and can receive knocking or changing acceleration as a wake-up signal. The sub-item b corresponds to a normal operating state of the headset, and the third intensity threshold may correspond to a normal human ear distance. The sub-item c corresponds to a state where the two headphones are far away from each other, a path establishment step may be entered to enter a search mode. In the working mode, the mobile phone and the Bluetooth headset work according to the following steps:

The first step: and an information acquisition step, wherein the two earphones acquire first sound source information based on bone conduction and second sound source information based on air conduction.

The bone conduction path is a conduction method that human body sign signals are conducted to the earphone through the contact skull, and most of information contained in the first sound source information obtained through bone conduction is human sound information, so that the bone conduction method has the characteristics of low noise and high fitting degree with the human sound information. The air conduction path is a conduction method that the vocal cord vibration signal is conducted to the earphone through air, and a large amount of environment noise is mixed in the conduction process, so that the second sound source information carries the voice information and the environment information at the same time.

And a second step of: and the information centralizing step is used for centralizing the first sound source information and the second sound source information in a single earphone. Specifically, the earphone with low electric quantity sends the acquired first sound source information and second sound source information to the earphone with high electric quantity through the NFMI signal.

The signals acquired by the two earphones can be correspondingly processed, such as weighted average processing, so that influence caused by interference of a single earphone is reduced, and fidelity of the first sound source information and the second sound source information is improved.

Due to the instantaneity of environment judgment and voice judgment, the scheme needs to continuously collect the first sound source information and the second sound source information. If a Bluetooth connection is adopted between two earphones, one earphone continuously receives Bluetooth information, and the other earphone continuously sends out the Bluetooth information, so that the earphone in the output mode consumes relatively fast power. If the two earphones are used for directly transmitting the first sound source information and the second sound source information to the mobile phone without mutual communication, the two earphones are in a Bluetooth information output mode, power consumption is high, and endurance is reduced. In the scheme, the first sound source information and the second sound source information are concentrated on the same earphone and are uniformly and outwards conveyed, and the earphone output is switched according to the electric quantity conditions of the left earphone and the right earphone, so that the cruising ability of the earphone set is effectively improved.

Specifically, for example, when the current left earphone is in the NFMI information output mode, the electric quantity is 70%, the right earphone is in the NFMI information receiving mode, the electric quantity is 60%, and the preset threshold is 10%, the left earphone and the right earphone transmit own electric quantity information to the other earphone through NFMI, then the two earphones compare the electric quantities of the two earphones, the left earphone is controlled to enter the NFMI information receiving mode and the bluetooth information output mode based on the comparison result, and the right earphone enters the NFMI signal output mode and the bluetooth information receiving mode.

In this embodiment, the NFMI wireless connection is unencrypted communication, and the bluetooth connection of the headset and the handset is symmetric key communication.

The effective radius of transmission of the NFMI is about 1m, so that the NFMI has good safety, and information is not easy to be stolen remotely, so that the communication efficiency can be improved well by using unencrypted communication, the operation amount is reduced, and the NFMI has the effect of saving electric quantity. In addition, the information quantity of the environment characteristic and the target characteristic is less, and the symmetric key encryption mode has less influence on the transmission of information.

And a third step of: and a feature extraction step, wherein the single earphone extracts environmental features and target features based on the first sound source information and the second sound source information, the environmental features are audio features of environmental sounds, and the target features are audio features of sounds made by a human body.

In brief, an audio feature is a sequence of frames, and each frame is a multidimensional vector. This frame sequence contains information such as the spectrum and amplitude of the ambient sound signal. The method of extracting audio features generally comprises the steps of: analog-to-digital conversion, DC removal, framing, pre-emphasis, windowing, fast Fourier transformation, mel domain filter bank, taking logarithm, discrete cosine transformation, MFCC, differential operation, obtaining audio characteristics. The logarithmic energy is obtained after framing and is used as a parameter of differential operation.

Fourth step: and in the signal transmission step, the earphone with high electricity quantity sends the environmental characteristic and the target characteristic to the mobile phone at fixed time through the Bluetooth signal.

The first sound source information and the second sound source information are extracted from the earphone, so that the information quantity required to be carried by Bluetooth signals between the mobile phone and the earphone is far smaller than the information quantity corresponding to the complete audio, and the time slot generated by transmitting the sound source information characteristics is far larger than the time slot generated by transmitting the complete audio because the transmission rate of Bluetooth is limited, for example, the transmission rate of Bluetooth 4.0 is 1Mbps and the content of bytes in lossless audio information is large, so that electricity is saved.

Fifth step: a mode judging step, namely reading environmental characteristics by the mobile phone, and selecting a local judging mode or a cloud judging mode based on the current networking state and the training degree of a local environmental acoustic model; the local judging mode is to use a local environment acoustic model to carry out environment judgment, and the cloud judging mode is to upload the environment characteristics to the cloud by using a mobile phone, and the cloud environment acoustic model carries out environment judgment on the environment characteristics; the local environment acoustic model is a neural network model trained by environment features acquired by the mobile phone, and the cloud environment acoustic model is a neural network model trained by environment acoustic models with standard training degrees on all networking mobile phones;

the environmental acoustic model is a model obtained by training a plurality of single environmental features, and can be a multi-classification neural network model, a deep neural network and the like, and after the corresponding environmental features are input, the environmental features can be scored and the environmental types corresponding to the environmental features can be judged. The single environmental characteristic can be the audio characteristic sent by a single sound source such as wind noise, footstep sound, wheezing sound and the like, and also can be the audio characteristic sent by a plurality of sound sources such as downtown sound, wave sound and the like, the judgment accuracy degree of the environmental acoustic model is influenced by the size of a training sample and the content of the sample, and the environmental acoustic model in each mobile phone is mainly influenced by the voice of a user, and meanwhile, the activity range of the user is usually limited, so that the mobile phone has specificity.

For a new mobile phone, the training degree of the carried environmental acoustic model is low, the judging effect is poor, so that the environmental acoustic model which needs to be trained in the cloud is temporarily replaced, for the environmental acoustic model which is stored in the cloud, a training sample of the training sample is usually the environmental acoustic model which is trained in the cloud, the generated environmental characteristics are uploaded to the cloud to train the environmental acoustic model of the cloud, and the environmental characteristics generated by the environmental acoustic model of the cloud can be identified by a plurality of local acoustic models. Therefore, in terms of accuracy, the accuracy of the judgment of the mature local environment acoustic model is higher than that of the cloud environment acoustic model due to the fact that the mature local environment acoustic model is matched with a user, and the proper environment acoustic model is selected according to the situation of the mobile phone, so that the accuracy of environment judgment is improved.

Specifically, referring to fig. 2, the mode determining step includes:

s1, reading environmental characteristics;

s22, uploading the environmental characteristics to a cloud end by using the mobile phone, judging the environmental characteristics by an environmental acoustic model of the cloud end, outputting the environmental types and returning to the mobile phone;

When the local environment acoustic model is fully trained, the local environment acoustic model is preferentially selected for judgment, then the cloud environment acoustic model is called in a networking state, and the parameter tuning simulation is performed for the initialized acoustic model for the first time. Optionally, the mobile phone end may set a backup database, and after each networking, select to download and save the adjustment parameters of the environmental acoustic model of the cloud end, so as to be used as the adjustment parameters for initializing the transformation function.

In particular, the acoustic model includes the following steps of:

Under different working environments, the characteristics of training samples corresponding to the models are not completely the same, so that the transformation function adjusted by training data corresponding to one working environment is not applicable to another working environment, and therefore, different transformation functions are needed to build different neural network models or build multi-classification neural networks. If the model is trained again by adjusting the training data corresponding to the working environment, so as to obtain a new transformation function, the required training data amount will be more, and the training period will be prolonged. Therefore, the scheme selects to input the environmental characteristics first and establish the posterior probability of each modeling unit, and 0/1 of the posterior probability has more mutation at the moment. And then compressing and smoothing the posterior probability of the modeling unit, so that 0/1 mutation of the posterior probability is reduced, an effect similar to large sample training is generated, the upper and lower jitter ranges of the posterior probability are smaller than the fluctuation range of the posterior probability of the modeling unit before processing, and the coverage of the candidate sequence of the posterior probability of the modeling unit on the correct environmental characteristic recognition result is increased. And then decoding the posterior probability processed by the modeling unit of all frames of the environmental characteristics, and outputting the environmental type.

In addition, the step of evaluating the training degree of the local environmental acoustic model in S2 includes an adjustment step and a scoring step, the adjustment step including:

The scoring step comprises the following steps:

When the training samples of the local training pre-trained environmental acoustic model are insufficient, the posterior probability distribution of the modeling unit of the output node is sharp, and the recognition failure rate of the environmental scene is amplified. And (3) approaching and optimizing the local pre-trained environmental acoustic model to a preset reference acoustic model, so that the posterior probability histogram of the modeling unit of the output node is smoother, and the influence of a small sample can be reduced. And if the score of the local model optimized in the scoring step is also lower than a second constant value, judging that the model does not reach the standard. The second constant value may be a score average value of the cloud environment acoustic model for the standard environment feature, or may be some weighted average value based on the score average value.

Sixth step: and a signal processing step, wherein the mobile phone processes the target feature based on the environment type obtained in the mode judging step to obtain a standard feature, wherein the standard feature is a feature which is affected by environmental sound less than the target feature.

The environmental influence factors contained in the target characteristics are eliminated by processing the environmental acoustic model of the corresponding environment, for example, a differential method is adopted to remove the corresponding parts.

Seventh step: and executing step, comparing the standard characteristic with the first threshold characteristic, and executing a preset operation instruction according to the comparison result.

The mobile phone is pre-stored with various processing models corresponding to the environment types, after the environment types are judged by the mode judging step, the mobile phone calls the corresponding processing models according to the environment types to process the target features so as to further take out the influence of the environment in the target features, thereby obtaining standard features, comparing the standard features with preset first threshold features, and executing operation instructions preset in the mobile phone according to the comparison results. For example, when the standard characteristic is snoring of a person and the volume of the snoring is greater than the threshold, the user can be judged to be in a sleep state, and the mobile phone automatically reduces the volume of the played music.

In real-world use, people choose to wear only one earphone in order to be able to listen to the environment sound and the playing content in the earphone at the same time. The TWS Bluetooth headset is inconvenient because the left headset and the right headset are not connected, so that the TWS Bluetooth headset is difficult to find after the next headset is put down. Because the Bluetooth connection strength between the earphone and the mobile phone is poor when the earphone is far away, the left earphone is disconnected with the mobile phone after a user walks or even changes rooms. At this time, because the environments of the two earphones are different, the acquired environmental sounds are not the same, and the two earphones are not suitable for being used as learning samples and noise reduction.

In order to be able to find the headset conveniently, in the search mode, the handset and the bluetooth headset operate according to the following steps:

the method comprises the steps of establishing a path, namely dividing a left earphone and a right earphone into a receiving earphone and a listening earphone based on detection of Bluetooth signals of a mobile phone, and sending a broadcast packet by the listening earphone through a transfer device or the receiving earphone based on a tree topology structure in a searching mode, wherein the tree topology structure takes a Bluetooth device as a host and other Bluetooth devices in a preset Bluetooth transmission signal intensity range corresponding to the Bluetooth device as slaves, the broadcast packet carries a forwarding frequency identifier and an address sequence, the address sequence stores addresses of transfer devices in a broadcast packet path, and the host selects whether to send the broadcast packet to the slaves or not based on the forwarding frequency identifier and the address sequence of the broadcast packet;

specifically, the strategy that the host selects whether to send the broadcast packet to the slave based on the forwarding time flag and the address sequence of the broadcast packet includes one or more of the following:

strategy 1: the host reads the current forwarding frequency mark of the broadcast packet and judges whether the forwarding frequency is smaller than the allowed maximum forwarding frequency; discarding the broadcast packet if not;

Strategy 2: the host reads the current address sequence of the broadcast packet, judges whether the starting address of the current broadcast packet is the address of the transfer equipment where the broadcast packet is currently located, discards the broadcast packet if yes, and writes the address of the transfer equipment where the broadcast packet is currently located into the address sequence if not;

strategy 3: the host reads the current address sequence, the forwarding frequency mark and the history record of the located transfer equipment of the broadcast packet, if the starting address of the broadcast packet exists in the history address and the forwarding frequency of the broadcast packet is larger than the history hop count of the corresponding broadcast packet in the history address, the broadcast packet is abandoned, otherwise, the starting address and the forwarding frequency of the broadcast packet are written into the history record and the broadcast packet is forwarded, wherein the history record comprises the history address and the history hop count, the history address is the starting address of the transfer equipment for historically transferring the broadcast packet, and the history hop count is the corresponding forwarding frequency of the transfer equipment for historically transferring the broadcast packet when the transfer equipment is located in the transfer equipment.

Because the transfer device forwards the broadcast packets in an unordered way, when the forwarding times are not limited, the broadcast packets can be repeatedly forwarded in the whole communication network without interruption, which can cause channel congestion and energy waste. Therefore, the forwarding times are required to be limited, the occupation of the whole bandwidth of the communication network is reduced, and the efficiency of the interception earphone for searching the mobile phone or the interception earphone is improved.

And a second step of: and a connection optimization step, namely acquiring an optimal path by the mobile phone or the listening earphone based on the received broadcast packet, and constructing a temporary communication path corresponding to the listening earphone based on the optimal path, wherein a forwarding frequency mark corresponding to the optimal path is the lowest.

Specifically, the connection preference step may consist of the steps of:

Since the devices that use the transit device to perform bluetooth connection do not necessarily have only bluetooth headset and mobile phone, other devices may be connected to the server through the transit device, so each transit device may be passed through multiple optimal paths, where each optimal path corresponds to a different device, such as multiple bluetooth headsets. Through the three steps, a plurality of feasible paths with the minimum jump times can be selected, and the number of the standby paths selected under the condition that the feasible paths are enough is ensured to be greater than or equal to N as much as possible, so that communication attempts can be conveniently carried out on enough standby paths when the optimal path communication is interrupted, and smooth communication is ensured.

And a third step of: and in the interception step, the mobile phone or the answering earphone sends an activation signal to the interception earphone through the temporary communication path, the interception earphone starts to intercept the nearby sound source information after receiving the activation signal, and sends a corresponding audio signal to the mobile phone or the answering earphone through the temporary communication path, and the mobile phone or the answering earphone receives, analyzes and plays the audio signal.

The mobile phone or the answering earphone preferably selects a path with the lowest forwarding times from all feasible paths as a temporary communication path so as to reduce communication delay and participating transfer equipment. After the temporary communication path is established, the mobile phone or the listening earphone triggers the listening earphone to listen to the nearby sound source information, the user can send out sound, and the distance between the user and the listening earphone is judged according to the sound received by the earphone until the user enters a room where the listening earphone is located, and Bluetooth connection is reestablished with the listening earphone. Along with the movement of a user, the position relation between the listening earphone or the mobile phone and surrounding transit equipment is adjusted to ensure and improve the communication quality.

The foregoing are all preferred embodiments of the present application, and are not intended to limit the scope of the present application in any way, therefore: all equivalent changes in structure, shape and principle of this application should be covered in the protection scope of this application.

Claims

1. The management method of the dual-mode TWS Bluetooth headset is characterized by comprising the following steps of:

an information centralizing step, namely centralizing the first sound source information and the second sound source information of the two earphones in a single earphone;

the method comprises the steps of executing steps of comparing standard features with first threshold features and executing preset operation instructions according to comparison results;

the mode judging step includes:

s1, reading environmental characteristics;

s23, a local pre-trained environmental acoustic model is called, environmental characteristics are judged by using the environmental acoustic model, and the environmental type is output; the environmental acoustic model is trained using environmental features.

2. The method for managing a dual-mode TWS bluetooth headset according to claim 1, wherein the state determining step is preceded by a master-slave determining step:

the information centralizing step includes:

3. The method for managing a dual-mode TWS bluetooth headset according to claim 1, wherein the determining the environmental characteristic by the acoustic model includes:

4. The method for managing a dual-mode TWS bluetooth headset according to claim 3, wherein the step of evaluating the training level of the local environmental acoustic model in S2 includes an adjusting step and a scoring step, and the adjusting step includes:

the scoring step includes:

5. The method for managing a dual-mode TWS bluetooth headset of claim 4, further comprising the steps of:

6. The method for managing a dual-mode TWS bluetooth headset of claim 5, wherein the status determining step comprises the steps of:

step two:

7. The method for managing dual-mode TWS bluetooth headset according to claim 6, wherein the method for selecting whether to transmit the broadcast packet to the slave based on the broadcast packet forwarding number flag and the address sequence by the master comprises:

8. The method for managing a dual-mode TWS bluetooth headset of claim 7, wherein the connection optimization step comprises: