CN113223508A

CN113223508A - Management method of dual-mode TWS Bluetooth headset

Info

Publication number: CN113223508A
Application number: CN202110336581.6A
Authority: CN
Inventors: 鲁鹏飞; 鲁霖; 姚放
Original assignee: Shenzhen Xinzhongxin Technology Co Ltd
Current assignee: Shenzhen Xinzhongxin Technology Co Ltd
Priority date: 2021-03-29
Filing date: 2021-03-29
Publication date: 2021-08-06
Anticipated expiration: 2041-03-29
Also published as: CN113223508B

Abstract

The application relates to a management method of a dual-mode TWS Bluetooth headset, which comprises a state judgment step for determining a headset communication mode, an information acquisition step for acquiring sound source information, an information concentration step for concentrating sound source information of two headsets, a feature extraction step for extracting sound source information features, a signal transmission step for transmitting the sound source information features to a mobile phone, a mode judgment step for analyzing an environment mode in which the sound source information is located from the sound source information features, and a signal processing step for processing the sound source information based on the environment mode to acquire more accurate standard features. This application has the effect that improves the system and based on the judgement efficiency that bluetooth headset acquireed sound information.

Description

Management method of dual-mode TWS Bluetooth headset

Technical Field

The application relates to the technical field of communication equipment, in particular to a management method of a dual-mode TWS Bluetooth headset.

Background

At present, people have not satisfied simple listening and voice functions brought by bluetooth headsets, and the demands are diversified, complicated and scenized. For example, when the user runs on the road, the user is flawless to use the cell-phone to operate, and it is the first choice to use bluetooth headset to carry out speech control, but the environment that the in-process of running was located is comparatively noisy usually, and the microphone can acquire pedestrian's noise, car noise, wind noise etc. simultaneously, and speech control's rate of accuracy is usually not high. For example, some earphones have a heartbeat abnormality alarm function, but the heart function of some old people is weak, the heartbeat rate of the old people is greatly different under different situations, for example, compared with the young people, the difference between the heartbeat rate of the old people during sleeping and the heartbeat rate of the old people during going upstairs is larger, and if the earphones are used for simple judgment, a larger error is easily generated.

For the above related technologies, the inventor believes that there is a high failure probability when the information directly acquired by the headset is directly subjected to the related determination without being subjected to the standardization process, for example, when the headset acquires the voice information of a person in a noisy motion environment to control the mobile phone application.

Disclosure of Invention

In order to improve the judgment efficiency of the system based on the sound information acquired by the Bluetooth headset, the application provides a management method of the dual-mode TWS Bluetooth headset.

The application provides a management method of a dual-mode TWS Bluetooth headset, which adopts the following technical scheme:

a management method of a dual-mode TWS Bluetooth headset comprises the following steps:

a state judgment step: the method comprises the steps that a left earphone and a right earphone acquire the wireless connection strength of each other, and an earphone communication mode is selected based on the wireless connection strength, wherein the earphone communication mode comprises a working mode and a searching mode;

an information acquisition step, in which two earphones acquire first sound source information based on bone conduction and acquire second sound source information based on air conduction in a working mode;

an information concentration step of concentrating the first sound source information and the second sound source information in a single earphone;

a feature extraction step, wherein a single earphone extracts environmental features and target features based on first sound source information and second sound source information, wherein the environmental features are audio features of environmental sounds, and the target features are audio features of sounds made by a human body;

a signal transmission step, wherein the environment characteristics and the target characteristics are sent to the mobile phone through Bluetooth signals;

a mode judgment step, namely reading environmental characteristics, and selecting a local judgment mode or a cloud judgment mode based on the mobile phone networking state and the training degree of a local environmental acoustic model; the local judgment mode is that a local environment acoustic model is used for carrying out environment judgment, the cloud judgment mode is that a mobile phone is used for uploading environment characteristics to a cloud, and the environment acoustic model of the cloud carries out environment judgment on the environment characteristics; the local environment acoustic model is a neural network model trained by environment characteristics acquired by the mobile phone, and the cloud environment acoustic model is a neural network model trained by environment acoustic models with standard training degrees on all networked mobile phones;

a signal processing step, in which the mobile phone processes the target feature based on the environment type obtained in the mode judging step to obtain a standard feature, wherein the standard feature is a feature which is less influenced by environment sound than the target feature;

and an executing step, comparing the standard characteristic with the first threshold characteristic, and executing a preset operation instruction according to the comparison result.

By adopting the technical scheme, the earphones are in various states, for example, the two earphones are separated from each other and are far away from each other, or the two earphones are placed in an earphone box, or the two earphones are normally worn on ears, and need to be classified to carry out corresponding communication modes. The working mode corresponds to the state that the two earphones are worn on the ears. The first sound source information is information obtained by bone conduction and has the characteristics of low noise and high fitting degree with human voice information, and the second sound source information is information obtained by air conduction and carries the human voice information and the environmental information at the same time. The two earphones centralize the sound source information into a single earphone and process the sound source information, after information such as frequency spectrum and the like is extracted, the environment characteristic and the target characteristic can be extracted by using a difference algorithm or other methods, the coherence of the environment characteristic and the target characteristic is small, compared with the arrangement of a noise microphone and a human sound microphone which are independent, the obtained audio characteristic is better, the judgment efficiency of a system is improved, but the target characteristic still has the influence generated by the environment and needs to be further processed. In addition, the signals acquired by the two earphones are concentrated and can be processed correspondingly, such as averaging, so that the accuracy of the environmental characteristics and the target characteristics is improved.

After the environmental characteristics and the target characteristics are transmitted to the mobile phone, the mode judging step calls an environmental acoustic model to judge the current environment of the user. The environmental acoustic model is a model obtained by teaching various single environmental features, and can be a multi-classification neural network model and the like, and after corresponding environmental features are input, the environmental type corresponding to the environmental features can be judged. The judgment accuracy of the environmental acoustic model is influenced by the size of the training sample, and the environmental acoustic model in each mobile phone is influenced by the voice of the holder, has specificity and needs to be used for a long time and trained. Therefore, the accuracy of the local environment acoustic model used for a long time is higher than that of the environment acoustic model in the cloud. And a proper environment acoustic model is selected, so that the accuracy of environment judgment is improved.

Various processing models corresponding to the environment types are prestored in the mobile phone, after the environment types are judged in the mode judging step, the mobile phone calls the corresponding processing models according to the environment types to process the target characteristics so as to further take out the influence of the environment in the target characteristics, thereby obtaining the standard characteristics, then the standard characteristics are compared with the preset first threshold characteristics, and the operation instruction preset in the mobile phone is executed according to the comparison result.

Preferably, a master-slave determining step is provided before the state judging step:

the method comprises the following steps: the left earphone and the right earphone are wirelessly connected through NFMI;

step two: the left earphone and the right earphone respectively acquire the electric quantity of the left earphone and the right earphone and compare the electric quantity with the electric quantity of the right earphone, when the difference value is larger than or equal to a preset threshold value, the earphone with high electric quantity is used as a main earphone to enter an NFMI information receiving mode and a Bluetooth information output mode, and the earphone with low electric quantity is used as a slave earphone to enter an NFMI signal output mode and a Bluetooth information receiving mode;

the information centralizing step comprises:

the earphone with low electric quantity transmits the acquired first sound source information and second sound source information to the earphone with high electric quantity through an NFMI (network function and broadcast interface) signal;

the signal transmission step comprises the following steps: the earphone with high electric quantity sends the environment characteristic and the target characteristic to the mobile phone at regular time through the Bluetooth signal.

By adopting the technical scheme, due to the immediacy of environment judgment and human voice judgment, the scheme needs to continuously collect the first sound source information and the second sound source information. If the two earphones are connected by the Bluetooth, one earphone continuously receives the Bluetooth information, and the other earphone continuously sends the Bluetooth information, so that the earphone in the output mode consumes more power. Because the two earphones are close in use, the NFMI with lower energy consumption can be used for information transfer. Theoretically, NFMI is more energy efficient than bluetooth radio frequency at short range transmission, which depends on antenna size. The signal strength drops less when the NFMI signal passes through human tissue, while the signal strength of bluetooth radio frequency drops significantly. The specific absorption rate of NFMI is 10000 times lower than bluetooth radio frequency, so NFMI enables long-time low-power communication between two earphones. Moreover, because the distance between the mobile phone and the earphone is not constant, the mobile phone can be far away, so that the Bluetooth is more suitable for wireless connection between the earphone and the mobile phone compared with NFMI. The earphone for NFMI reception needs to transmit the extracted features to the outside through bluetooth, so the power consumption of the earphone is higher than that of the other earphone, and the earphone is switched between the information receiving mode and the information outputting mode by performing power judgment to balance the power consumption of the two earphones.

In addition, because the extraction of the sound source information features is realized in the earphone, the information quantity carried by the Bluetooth signal between the mobile phone and the earphone is far less than the information quantity corresponding to the complete audio. The transmission rate of bluetooth is limited, for example, the transmission speed of bluetooth 4.0 is 1Mbps, and the time slot generated by transmitting the sound source information characteristic is far greater than that of transmitting complete audio, so that interval communication can be realized in the scheme, and the power consumption is lower.

Preferably, the NFMI wireless connection is an unencrypted communication, and the bluetooth connection between the headset and the handset is a symmetric key communication.

By adopting the technical scheme, the effective transmission radius of the NFMI is about 1m, the NFMI has good safety, and information is difficult to steal remotely, so that the communication efficiency can be improved well by using unencrypted communication, and the calculation amount is reduced. The information quantity of the environment characteristic and the target characteristic is less, and the influence of a symmetric key encryption mode on information transmission is less.

Preferably, the mode determining step includes:

s1, reading environmental characteristics;

s2, obtaining a current networking state and evaluating the training degree of the local environment acoustic model, and if the current networking state is in an off-line state and the training degree of the local environment acoustic model does not reach the standard, entering S21; if the current online state is achieved and the training degree of the local environment acoustic model does not reach the standard, the method enters S22; if the training degree of the local environment acoustic model reaches the standard, entering S23;

s21, calling an initial acoustic model and adjusting parameters corresponding to various environments, which are stored locally in advance, and initializing a transformation function of the initial acoustic model into a transformation function corresponding to each working environment based on the adjusting parameters; judging the environmental characteristics by using an environmental acoustic model generated by each transformation function and outputting an environmental type;

s22, uploading the environment characteristics to a cloud end by the mobile phone, judging the environment characteristics by an environment acoustic model of the cloud end, outputting the environment type and returning to the mobile phone;

s23, calling a local pre-trained environmental acoustic model, judging environmental characteristics by using the environmental acoustic model and outputting an environmental type; the environmental acoustic model is trained using the environmental features.

By adopting the technical scheme, the cloud environment acoustic model is trained through the mature environment acoustic model trained by each local mobile phone, and the judgment accuracy of the environment acoustic model is related to the training degree and the training sample, so that the cloud environment acoustic model is relatively universal, and the fully-trained local environment acoustic model has higher matching degree for users of the mobile phones. When the local environment acoustic model is fully trained, the local environment acoustic model is preferentially selected for judgment, then the environment acoustic model of the cloud is called in a networking state, and the initialized acoustic model is used for parameter adjusting simulation for the first time. The mobile phone end can also be provided with a backup database, and the adjustment parameters of the cloud environment acoustic model are downloaded and stored after each networking so as to be used as the adjustment parameters.

Preferably, the step of judging the environmental characteristics by the acoustic model is as follows:

the method comprises the following steps: establishing posterior probability of each modeling unit according to the environment characteristics and a local environment acoustic model;

step two: compressing and smoothing the posterior probability of the modeling units to obtain the posterior probability processed by each modeling unit;

step three: and decoding the posterior probabilities processed by the modeling units of all the frames of the environmental characteristics to obtain the environmental type.

In actual working environments, the working environment is different, and the corresponding background noise is different, for example, the engine noise of a vehicle on a road is often intermittent, the walk on a sidewalk is often louder, a commercial street is often louder, and the like. Under different working environments, the features of training samples corresponding to the models are not completely the same, so that a transformation function adjusted from training data corresponding to one working environment is not suitable for another working environment, and therefore different transformation functions are required to establish different neural network models or establish multi-classification neural network models. If the model is trained again by adjusting the training data corresponding to the working environment, so as to obtain a new transformation function, the required training data volume will be large, and the training period will be prolonged. Therefore, according to the scheme, the environmental characteristics are input firstly, the posterior probability of each modeling unit is established, and 0/1 of the posterior probability is more abrupt change at the moment. And then, the posterior probability of the modeling unit is subjected to compression smoothing, 0/1 mutation of the posterior probability is reduced, the effect similar to large sample training is generated, the up-down jitter range of the posterior probability is smaller than the fluctuation range of the posterior probability of the modeling unit before processing, and therefore the coverage of the candidate sequence of the posterior probability of the modeling unit on the correct environment feature recognition result is increased. And then decoding the posterior probabilities processed by the modeling units of all the frames of the environmental characteristics, and outputting the environmental types.

Preferably, the step of evaluating the training degree of the local environmental acoustic model in S2 includes an adjusting step and a scoring step, and the adjusting step includes:

the method comprises the following steps: calling an initial acoustic model and adjusting parameters corresponding to various environments, which are locally pre-stored, and initializing a transformation function of the initial acoustic model into transformation functions corresponding to various working environments based on the adjusting parameters to obtain a reference acoustic model;

step two: obtaining posterior probabilities of each modeling unit corresponding to a transformation function of a reference acoustic model;

step three: initializing transformation function parameters corresponding to a local model, and acquiring posterior probabilities of modeling units corresponding to the initialized local model, wherein the local model is a local pre-trained environmental acoustic model;

step four: calculating the posterior probability of each modeling unit of the local model and the posterior probability distribution distance of each modeling unit of the reference acoustic model, and adjusting the transformation function parameters corresponding to the working environment according to the distance to obtain the optimal transformation function parameters of the local model aiming at the corresponding working environment;

step five: adjusting the corresponding transformation function of the local model according to the obtained optimal transformation function parameter;

the scoring step comprises:

the method comprises the following steps: reading standard environment characteristics prestored in the mobile phone;

step two: and adjusting the corresponding local model by using the optimal transformation function parameter to grade the standard environment characteristic, judging that the local model reaches the standard if the grade is higher than a second fixed value, and otherwise, judging that the local model does not reach the standard.

By adopting the technical scheme, when training samples of the environment acoustic model trained in advance in local training are insufficient, the posterior probability distribution of the modeling unit of the output node is sharp, and the recognition failure rate of the environment scene is amplified. The local pre-trained environmental acoustic model is adjusted and optimized towards the preset reference acoustic model, so that the posterior probability histogram of the modeling unit of the output node is smoother, and the influence of small samples can be reduced. And if the score of the optimized local model is lower than a second fixed value in the scoring step, the score of the local model before optimization is lower than the second fixed value, and the model is judged not to reach the standard.

Preferably, the method further comprises the following steps:

a path establishing step: the left earphone and the right earphone are divided into an answering earphone and an intercepting earphone based on detection of mobile phone Bluetooth signals, and the intercepting earphone sends a broadcast packet to the mobile phone or the answering earphone through a transfer device based on a tree-shaped topological structure in a search mode, wherein the tree-shaped topological structure takes a Bluetooth device as a host, other Bluetooth devices in a preset Bluetooth transmission signal intensity range corresponding to the Bluetooth device as slaves, the broadcast packet carries a forwarding frequency identifier and an address sequence, the address sequence stores an address of the transfer device in a broadcast packet path, and the host selects whether the broadcast packet is sent to the slaves or not based on the forwarding frequency identifier and the address sequence of the broadcast packet;

connection optimization step: the method comprises the steps that a mobile phone or a receiving earphone obtains an optimal path based on a received broadcast packet, and a temporary communication path of the receiving earphone is constructed based on the optimal path, wherein the forwarding time mark corresponding to the optimal path is the lowest;

an interception step: the mobile phone or the answering earphone sends an activation signal to the listening earphone through the temporary communication path, the listening earphone starts to listen to nearby sound source information after receiving the activation signal, and sends a corresponding audio signal to the mobile phone or the answering earphone through the temporary communication path, and the mobile phone or the answering earphone receives, analyzes and plays the audio signal.

By adopting the technical scheme, in the practical use, people can choose to wear only one earphone in order to listen to the environmental sound and the playing content in the earphone at the same time. Since the left and right earphones of the TWS bluetooth earphone are not connected, the left and right earphones are often difficult to find after putting down one earphone, which is very inconvenient. Because the Bluetooth connection strength between the earphone and the mobile phone is poor when the earphone is far away, the left earphone can be disconnected with the mobile phone when a user walks or even changes a room. At the moment, because the two earphones are in different environments, the obtained environmental sounds are different, and the earphones are not suitable for being used as learning samples and are also not suitable for noise reduction.

In order to find the earphones conveniently, the lost earphones are used as listening earphones to establish a connection relation with surrounding transfer equipment and propagate the broadcast packet towards the transfer equipment, and the transfer equipment transmits the broadcast packet to the surrounding transfer equipment based on a preset rule until the transfer equipment transmits the broadcast packet to a mobile phone or a listening earphone. Since the direction of the broadcast packet is not unique, multiple paths are generated, and when the communication path is long, the signal delay is serious and the transmission effect is relatively poor. And all the relay devices continuously receive and transmit the broadcast packets, which will cause a great loss of energy. Therefore, the mobile phone or the receiving earphone selects a path with the lowest forwarding times from all the feasible paths as a temporary communication path, so as to reduce communication delay and reduce participating relay equipment. After the temporary communication path is established, the mobile phone or the listening earphone triggers the listening earphone to listen to nearby sound source information, a user can make a sound, the distance between the user and the listening earphone is judged according to the sound intensity heard by the earphone until the user enters a room where the listening earphone is located, and Bluetooth connection is reestablished with the listening earphone. With the movement of the user, the position relationship between the listening earphone or the mobile phone and the surrounding transfer equipment is adjusted to ensure and improve the communication quality.

Preferably, the state judging step includes the steps of:

the method comprises the following steps: the main earphone acquires the NFMI signal intensity of the auxiliary earphone, compares the signal intensity with the upper limit and the lower limit of a first preset intensity range, and selects a corresponding sub-item entering the next step based on the comparison result;

step two:

if the signal intensity is greater than the upper limit of the preset intensity range and the variation of the signal intensity is smaller than the preset threshold value in the preset time length, the master earphone and the slave earphone enter the sleep mode after the preset time length, and the step one is returned until the master earphone or the slave earphone detects a wake-up signal;

if the signal intensity is within the preset intensity range, the master earphone and the slave earphone enter a working mode;

if the signal strength is less than the lower limit of the preset strength range, entering a path establishing step.

By adopting the technical scheme, the main earphone and the slave earphone are connected through the NFMI, and the NFMI has strong signal connection and serious attenuation along with distance change, so that the distance between the main earphone and the slave earphone can be judged according to the NFMI connection strength of the main earphone and the slave earphone. For the bluetooth, need carry out accurate location then need corresponding bluetooth chip, the volume of this type of chip module is great, is difficult to integrate to bluetooth headset on.

The sub-item a corresponds to the situation that two earphones are taken down and put together, the user often takes the two earphones down and puts them in a pocket, on a desk or on a bed, and the earphones are continuously powered off in the related art. In the scheme, the master earphone and the slave earphone both go to sleep until being picked up again and receiving the awakening signal. Corresponding acceleration sensors are arranged in all the current mainstream earphones, and knocking or changed acceleration can be received as a wake-up signal.

The sub-item b corresponds to a normal working state of the earphone, and the third intensity threshold may correspond to an ear distance of a normal person.

The sub-item c corresponds to a state where the two earphones are far away from each other, and the path establishing step may be entered to enter the seeking mode.

Preferably, the method for the master to select whether to transmit the broadcast packet to the slave based on the flag of the number of times of forwarding the broadcast packet and the address sequence includes:

the host reads the current forwarding times mark of the broadcast packet and judges whether the forwarding times are less than the allowed maximum forwarding times; if not, discarding the broadcast packet;

the host reads the current address sequence of the broadcast packet, judges whether the starting address of the current broadcast packet is the address of the current transfer equipment of the broadcast packet, if so, discards the broadcast packet, and otherwise, writes the address of the current transfer equipment of the broadcast packet into the address sequence;

the host reads the current address sequence, the forwarding number mark and the history record of the relay equipment, if the starting address of the broadcast packet exists in the history address and the forwarding number of the broadcast packet is greater than the history hop number of the corresponding broadcast packet in the history address, the broadcast packet is discarded, otherwise, the starting address and the forwarding number of the broadcast packet are written into the history record and the broadcast packet is forwarded, wherein the history record comprises the history address and the history hop number, the history address is the starting address of the history transmission broadcast packet of the relay equipment, and the history hop number is the corresponding forwarding number when the history transmission broadcast packet of the relay equipment is located in the relay equipment.

By adopting the technical scheme, because the transfer equipment carries out the unordered transfer on the broadcast packet, when the transfer times are not limited, the broadcast packet can be repeatedly transferred in the whole communication network without interruption, which causes channel congestion and energy waste. Therefore, the forwarding times need to be limited, the occupation of the whole communication network bandwidth is reduced, and the efficiency of searching for the mobile phone by the listening earphone or receiving the earphone is improved.

Because the forwarding direction of the broadcast packet is unordered, the host of the current tree topology structure can also be the slave of the adjacent tree topology structure, and whether the data packet is discarded or not is determined by judging whether the sender and the receiver of the broadcast packet are consistent, so that the invalid forwarding of the data packet in the communication network is reduced. In addition, because the broadcast packet is forwarded out of order, the broadcast packet can easily pass through the same transit equipment through different paths, and the comparison between the forwarding times on the broadcast packet and the historical hop count can be used for judging whether the path corresponding to the broadcast packet is a longer one, if so, the broadcast packet is discarded, and the invalid forwarding of the broadcast packet in the communication network is reduced.

Preferably, the connecting step includes:

the method comprises the following steps: the method comprises the steps that the mobile phone acquires the number of transfer devices corresponding to all feasible paths from a monitoring earphone to the mobile phone, and the transfer devices serve as a feasible set;

step two: the mobile phone judges whether the number of the minimum elements in the feasible set is larger than or equal to N, and if the number of the minimum elements in the feasible set is larger than N, the feasible path corresponding to the minimum elements in the feasible set is used as a standby path; if the number of the feasible paths is smaller than N, the feasible paths corresponding to the minimum N elements in the feasible sets are used as standby paths, wherein N is a preset number threshold;

step three: and the mobile phone calculates the sum of the times of the other optimal paths of the transit equipment in each standby path, and takes the sum as the characteristic number of the corresponding standby path, and selects the standby path with the minimum characteristic number as the optimal path.

By adopting the technical scheme, a plurality of feasible paths with the least switching times are selected, and the number of standby paths selected under the condition that the feasible paths are enough is ensured to be more than or equal to N as much as possible, so that the communication attempt is carried out on enough standby paths when the communication of the optimal path is interrupted, and the smooth communication is ensured.

Drawings

Fig. 1 is a flowchart of a management method of a dual-mode TWS bluetooth headset in an embodiment of the present application.

Fig. 2 is a block diagram of a flow of a mode determination step in an embodiment of the present application.

Detailed Description

The present application is described in further detail below with reference to figures 1-2.

At present, people have a trend of diversification, complication and scene for the function of the bluetooth headset, and the requirement for the accuracy of voice recognition and signal acquisition is increasing. People are changeable to the application scene of earphone, and all kinds of environment sound acquire the influence diverse that target signal produced to the earphone, for example, when the user is running on the road, the user is flawless and uses the cell-phone to operate, consequently can adopt bluetooth headset read-in pronunciation to carry out the method of corresponding control, but the environment that the in-process of running was located is comparatively noisy usually, and the microphone can acquire pedestrian's noise simultaneously, car noise, wind noise etc. and speech control's rate of accuracy is usually not high. For example, compared with the young, the difference between the heartbeat rates of the old during sleep and during going upstairs is larger, and if the earphone directly reads the heartbeat rate to judge the current state of the old, a larger error is easily generated.

For another example, when the behavior state of the user going upstairs is obtained, the detected heart rate is corrected according to the corresponding algorithm of the state, and the corrected heart rate is used as the judgment data of the body state of the user, so that the judgment accuracy rate is improved. Or, when the user runs outdoors, the earphone records running sound and wheezing sound as characteristic audio, thereby judging the state of the user, and then the voice signal of the user is processed in a targeted manner based on the state, so that the voice recognition capability of the earphone can be improved.

The embodiment of the application discloses a management method of a dual-mode TWS Bluetooth headset. Referring to fig. 1, the management method includes the steps of:

a master-slave determination step:

step two: the left earphone and the right earphone respectively acquire the electric quantity of the left earphone and the right earphone and compare the electric quantity with the electric quantity of the right earphone, when the difference value is larger than or equal to a preset threshold value, the earphone with high electric quantity is used as a main earphone to enter an NFMI information receiving mode and a Bluetooth information output mode, and the earphone with low electric quantity is used as a slave earphone to enter an NFMI signal output mode and a Bluetooth information receiving mode.

NFMI, near field magnetic induction, is a short-range wireless technology that communicates by coupling a tight, low-power, non-propagating magnetic field between devices. A transmitter coil in one device modulates an electromagnetic field that can be measured by a receiver coil in another device.

Because the two earphones are close to each other in use, the NFMI with lower energy consumption can be used for information transmission between the earphones. Theoretically, NFMI is more energy efficient than bluetooth radio frequency at short range transmission, which depends on antenna size. The signal strength drops less when the NFMI signal passes through human tissue, while the signal strength of bluetooth radio frequency drops significantly. Experimental tests show that the specific absorption rate of the NFMI is 10000 times lower than that of bluetooth, and the brain is a good bluetooth radio frequency absorption organ, so that the NFMI can adapt to long-time signal transmission between two earphones.

A state judgment step: the main earphone acquires the NFMI signal intensity of the auxiliary earphone, compares the signal intensity with the upper limit and the lower limit of a first preset intensity range, and selects corresponding sub-items corresponding to different comparison results;

if the signal intensity is greater than the upper limit of the preset intensity range and the variation of the signal intensity is smaller than the preset threshold value in the preset time length, enabling the master earphone and the slave earphone to enter the sleep state after the preset time length, and performing the state judgment step again until the master earphone or the slave earphone detects the awakening signal;

Since the main earphone and the slave earphone are connected through the NFMI, the distance between the main earphone and the slave earphone can be judged according to the NFMI connection strength of the main earphone and the slave earphone since the NFMI signal connection is strong and the attenuation is severe along with the distance change. For the bluetooth, need carry out accurate location then need corresponding bluetooth chip, the volume of this type of chip module is great, is difficult to integrate to bluetooth headset on. Therefore, as an embodiment of the present application, the NFMI connection is used to perform ranging on the master and slave earphones, so that the effects of lower cost and smaller size can be achieved.

The sub-item a corresponds to the situation that two earphones are taken down and put together, the user often takes the two earphones down and puts them in a pocket, on a desk or on a bed, and the earphones are continuously powered off in the related art. In the scheme, the master earphone and the slave earphone both go to sleep until being picked up again and receiving the awakening signal. Corresponding acceleration sensors are arranged in all the current mainstream earphones, and knocking or changed acceleration can be received as a wake-up signal. The sub-item b corresponds to a normal working state of the earphone, and the third intensity threshold may correspond to an ear distance of a normal person. The sub-item c corresponds to a state where the two earphones are far away from each other, and the path establishing step may be entered to enter the seeking mode. In the working mode, the mobile phone and the Bluetooth headset work according to the following steps:

the first step is as follows: and an information acquisition step, wherein the two earphones acquire first sound source information based on bone conduction and acquire second sound source information based on air conduction.

The bone conduction path is a conduction method for conducting human body sign signals to the earphone through contacting with the skull, most of the information contained in the first sound source information obtained through bone conduction is human sound information, and the bone conduction path has the characteristics of low noise and high fitting degree with the human sound information. The air conduction path is a conduction method for conducting vocal cord vibration signals to the earphone through air, and a large amount of environmental noise is mixed in the conduction process, so that the second sound source information simultaneously carries human voice information and environmental information.

The second step is that: and an information concentration step of concentrating the first sound source information and the second sound source information in a single earphone. Specifically, the earphone with low power transmits the acquired first sound source information and second sound source information to the earphone with high power through the NFMI signal.

The signals acquired by the two earphones can be processed correspondingly, for example, weighted average processing is carried out, the influence of interference on a single earphone is reduced, and the fidelity of the first sound source information and the second sound source information is improved.

Due to the immediacy of environment judgment and human voice judgment, the scheme needs to continuously collect the first sound source information and the second sound source information. If the two earphones are connected by the Bluetooth, one earphone continuously receives the Bluetooth information, and the other earphone continuously sends the Bluetooth information, so that the earphone in the output mode consumes more power. If adopt two earphones all directly with first sound source information and second sound source information transfer to the cell-phone, and do not carry out intercommunications, then can make two earphones all be in bluetooth information output mode, power consumptive very fast, the continuation of the journey reduces. According to the scheme, the first sound source information and the second sound source information are concentrated on the same earphone and are conveyed outwards in a unified mode, the earphone output is switched according to the electric quantity conditions of the left earphone and the right earphone, and the cruising ability of the earphone set is effectively improved.

Specifically, for example, the current left earphone is in the NFMI information output mode, the electric quantity is 70%, the right earphone is in the NFMI information receiving mode, the electric quantity is 60%, and the preset threshold is 10%, the left earphone and the right earphone transmit the electric quantity information of the left earphone and the right earphone to the other earphone through the NFMI, then both earphones compare the electric quantities of the two earphones, and control the left earphone to enter the NFMI information receiving mode and the bluetooth information output mode based on the comparison result, and the right earphone enters the NFMI signal output mode and the bluetooth information receiving mode.

In this embodiment, the NFMI wireless connection is an unencrypted communication, and the bluetooth connection of the headset and handset is a symmetric key communication.

The effective transmission radius of the NFMI is about 1m, the NFMI has good safety, and information is not easy to steal remotely, so that the communication efficiency can be improved well by using unencrypted communication, the calculation amount is reduced, and the effect of saving electric quantity is achieved. In addition, the information quantity of the environment characteristic and the target characteristic is small, and the influence of the symmetric key encryption mode on the information transmission is small.

The third step: and a feature extraction step, wherein the single earphone extracts environmental features and target features based on the first sound source information and the second sound source information, wherein the environmental features are audio features of environmental sounds, and the target features are audio features of sounds made by a human body.

Briefly, an audio feature is a sequence of frames, and each frame is a multi-dimensional vector. This frame sequence contains information such as the frequency spectrum and amplitude of the ambient sound signal. The method of extracting audio features generally comprises the steps of: analog-to-digital conversion, direct current removal, framing, pre-emphasis, windowing, fast Fourier transform, Mel domain filter bank, logarithm taking, discrete cosine transform, MFCC, and differential operation to obtain audio features. Wherein, the logarithmic energy is obtained after framing for being used as the parameter of the difference operation.

The fourth step: and a signal transmission step, namely, the earphone with high electric quantity sends the environmental characteristics and the target characteristics to the mobile phone at regular time through a Bluetooth signal.

The extraction of sound source information characteristics is carried out on the first sound source information and the second sound source information in the earphone, so that the information quantity required to be borne by a Bluetooth signal between the mobile phone and the earphone is far less than the information quantity corresponding to a complete audio frequency, and because the transmission rate of the Bluetooth is limited, for example, the transmission speed of the Bluetooth is 1Mbps, and the byte content in lossless audio information is large, the time slot generated by transmitting the sound source information characteristics is far greater than the time slot generated by transmitting the complete audio frequency, so that more electricity can be saved.

The fifth step: a mode judgment step, namely reading the environmental characteristics by the mobile phone, and selecting a local judgment mode or a cloud judgment mode based on the current networking state and the training degree of a local environmental acoustic model; the local judgment mode is that a local environment acoustic model is used for carrying out environment judgment, the cloud judgment mode is that a mobile phone is used for uploading environment characteristics to a cloud, and the environment acoustic model of the cloud carries out environment judgment on the environment characteristics; the local environment acoustic model is a neural network model trained by environment characteristics acquired by the mobile phone, and the cloud environment acoustic model is a neural network model trained by environment acoustic models with standard training degrees on all networked mobile phones;

the environmental acoustic model is a model obtained by training a plurality of single environmental features, can be a multi-classification neural network model, a deep neural network and the like, and can score the environmental features and judge the environmental types corresponding to the environmental features after inputting the corresponding environmental features. The single environmental characteristic can be the audio frequency characteristic that single sound sources such as wind noise, footstep sound, wheeze sound sent, also can be the audio frequency characteristic that many sound sources such as city sound, sea wave sound sent, and the judgement accuracy degree of environment acoustics model receives the influence of training sample size, also receives the influence of sample content, because environment acoustics model mainly receives the human voice influence of user in every cell-phone, and user's home range is comparatively limited usually simultaneously, consequently has the specificity.

For being used for new cell-phone, the training degree of its environment acoustic model who carries on is low, and it is relatively poor to judge the effect, therefore needs the environment acoustic model of the ripe training in high in the clouds to carry out temporary substitution work, to the environment acoustic model of saving in the high in the clouds, its training sample is the ripe environment acoustic model generation environmental characteristic of training on each cell-phone end usually to upload the environmental characteristic that generates in the high in the clouds in order to train the environment acoustic model in the high in the clouds, the environmental characteristic that the environment acoustic model of the high in the clouds generated can be discerned by more quantity local acoustic models. Therefore, in terms of accuracy, the judgment accuracy of the trained local environment acoustic model is higher than that of the cloud environment acoustic model due to the fact that the trained local environment acoustic model is matched with the user, and the proper environment acoustic model is selected according to the mobile phone condition, so that the accuracy of environment judgment is improved.

Specifically, referring to fig. 2, the mode determining step includes:

s1, reading environmental characteristics;

s22, uploading the environmental characteristics to a cloud end by using the mobile phone, judging the environmental characteristics by using an environmental acoustic model of the cloud end, outputting an environmental type and returning the environmental type to the mobile phone;

When the local environment acoustic model is fully trained, the local environment acoustic model is preferentially selected for judgment, then the environment acoustic model of the cloud is called in a networking state, and the initialized acoustic model is used for parameter adjusting simulation for the first time. Optionally, the mobile phone end may set a backup database, and download and store the adjustment parameters of the cloud environment acoustic model after each networking so as to serve as the adjustment parameters for initializing the transformation function.

Specifically, the step of judging the environmental characteristics by the acoustic model comprises the following steps:

Under different working environments, the features of training samples corresponding to the models are not completely the same, so that a transformation function adjusted from training data corresponding to one working environment is not suitable for another working environment, and therefore different transformation functions are required to establish different neural network models or establish a multi-classification neural network. If the model is trained again by adjusting the training data corresponding to the working environment, so as to obtain a new transformation function, the required training data volume will be large, and the training period will be prolonged. Therefore, according to the scheme, the environmental characteristics are input firstly, the posterior probability of each modeling unit is established, and 0/1 of the posterior probability is more abrupt change at the moment. And then, the posterior probability of the modeling unit is subjected to compression smoothing, 0/1 mutation of the posterior probability is reduced, the effect similar to large sample training is generated, the up-down jitter range of the posterior probability is smaller than the fluctuation range of the posterior probability of the modeling unit before processing, and therefore the coverage of the candidate sequence of the posterior probability of the modeling unit on the correct environment feature recognition result is increased. And then decoding the posterior probabilities processed by the modeling units of all the frames of the environmental characteristics, and outputting the environmental types.

In addition, the step of evaluating the training degree of the local environmental acoustic model in S2 includes an adjusting step and a scoring step, and the adjusting step includes:

the scoring step comprises the following steps:

When training samples of the environment acoustic model trained in advance in local training are insufficient, posterior probability distribution of a modeling unit of the output node is sharp, and the recognition failure rate of the environment scene is amplified. The local pre-trained environmental acoustic model is adjusted and optimized towards the preset reference acoustic model, so that the posterior probability histogram of the modeling unit of the output node is smoother, and the influence of small samples can be reduced. And if the score of the optimized local model in the scoring step is lower than a second fixed value, judging that the model does not reach the standard. This second definite value can be the average value of grading of cloud environment acoustic model to standard environmental characteristic, also can be based on some weighted average of this average value of grading.

And a sixth step: and a signal processing step, wherein the mobile phone processes the target feature based on the environment type obtained in the mode judging step to obtain a standard feature, wherein the standard feature is a feature which is less influenced by environment sound than the target feature.

The environmental influence factors contained in the target features are eliminated by processing the environmental acoustic model of the corresponding environment, for example, a difference method is adopted to remove the corresponding parts.

The seventh step: and an executing step, comparing the standard characteristic with the first threshold characteristic, and executing a preset operation instruction according to the comparison result.

Various processing models corresponding to the environment types are prestored in the mobile phone, after the environment types are judged in the mode judging step, the mobile phone calls the corresponding processing models according to the environment types to process the target characteristics so as to further take out the influence of the environment in the target characteristics, thereby obtaining the standard characteristics, then the standard characteristics are compared with the preset first threshold characteristics, and the operation instruction preset in the mobile phone is executed according to the comparison result. For example, when the standard characteristic is a person's snore and the volume of the snore is greater than a threshold, it can be determined that the user is in a sleep state and the mobile phone automatically reduces the volume of the music played.

In practical use, people can choose to wear only one earphone in order to listen to the ambient sound and the playing content in the earphone at the same time. Since the left and right earphones of the TWS bluetooth earphone are not connected, the left and right earphones are often difficult to find after putting down one earphone, which is very inconvenient. Because the Bluetooth connection strength between the earphone and the mobile phone is poor when the earphone is far away, the left earphone can be disconnected with the mobile phone when a user walks or even changes a room. At the moment, because the two earphones are in different environments, the obtained environmental sounds are different, and the earphones are not suitable for being used as learning samples and are also not suitable for noise reduction.

In order to conveniently find the earphone, in a searching mode, the mobile phone and the Bluetooth earphone work according to the following steps:

the method comprises the steps that firstly, a path is established, a left earphone and a right earphone are divided into an answering earphone and an intercepting earphone based on detection of mobile phone Bluetooth signals, the intercepting earphone sends broadcast packets to the mobile phone or the answering earphone through a transfer device based on a tree-shaped topological structure in a search mode, wherein the tree-shaped topological structure takes a Bluetooth device as a host computer and takes other Bluetooth devices in a preset Bluetooth transmission signal intensity range corresponding to the Bluetooth device as slave computers, the broadcast packets carry forwarding time marks and address sequences, the address sequences store addresses of the transfer device in a broadcast packet path, and the host computer selects whether the broadcast packets are sent to the slave computers or not based on the forwarding time marks and the address sequences of the broadcast packets;

specifically, the policy that the master selects whether to send the broadcast packet to the slave or not based on the forwarding number flag and the address sequence of the broadcast packet includes one or more of the following:

strategy 1: the host reads the current forwarding times mark of the broadcast packet and judges whether the forwarding times are less than the allowed maximum forwarding times; if not, discarding the broadcast packet;

strategy 2: the host reads the current address sequence of the broadcast packet, judges whether the starting address of the current broadcast packet is the address of the current transfer equipment of the broadcast packet, if so, discards the broadcast packet, and otherwise, writes the address of the current transfer equipment of the broadcast packet into the address sequence;

strategy 3: the host reads the current address sequence, the forwarding number mark and the history record of the relay equipment, if the starting address of the broadcast packet exists in the history address and the forwarding number of the broadcast packet is greater than the history hop number of the corresponding broadcast packet in the history address, the broadcast packet is discarded, otherwise, the starting address and the forwarding number of the broadcast packet are written into the history record and the broadcast packet is forwarded, wherein the history record comprises the history address and the history hop number, the history address is the starting address of the history transmission broadcast packet of the relay equipment, and the history hop number is the corresponding forwarding number when the history transmission broadcast packet of the relay equipment is located in the relay equipment.

Because the relay device forwards the broadcast packet in a non-sequential manner, when the forwarding times are not limited, the broadcast packet is repeatedly forwarded in the whole communication network without interruption, which causes channel congestion and energy waste. Therefore, the forwarding times need to be limited, the occupation of the whole communication network bandwidth is reduced, and the efficiency of searching for the mobile phone by the listening earphone or receiving the earphone is improved.

The second step is that: and a connection optimization step, wherein the mobile phone or the receiving earphone obtains an optimal path based on the received broadcast packet, and a temporary communication path with the receiving earphone is constructed based on the optimal path, wherein the forwarding time mark corresponding to the optimal path is the lowest.

Specifically, the connection optimization step may consist of the following steps:

Since the devices using the relay device for bluetooth connection do not necessarily have only a bluetooth headset and a mobile phone, and other devices may be connected to the server through the relay device, each relay device may be passed by multiple optimal paths, where each optimal path corresponds to a different device, for example, multiple bluetooth headsets. Through the three steps, a plurality of feasible paths with the minimum hop frequency can be selected, and the number of standby paths selected under the condition that the feasible paths are enough is ensured to be more than or equal to N as far as possible, so that the communication attempt can be carried out on the enough standby paths when the communication of the optimal path is interrupted, and the smooth communication is ensured.

The third step: and an interception step, wherein the mobile phone or the answering earphone sends an activation signal to the interception earphone through the temporary communication path, the interception earphone starts to intercept nearby sound source information after receiving the activation signal, and sends a corresponding audio signal to the mobile phone or the answering earphone through the temporary communication path, and the mobile phone or the answering earphone receives, analyzes and plays the audio signal.

The mobile phone or the receiving earphone selects a path with the lowest forwarding times from all the feasible paths as a temporary communication path so as to reduce communication delay and reduce participating transfer equipment. After the temporary communication path is established, the mobile phone or the listening earphone triggers the listening earphone to listen to nearby sound source information, a user can make a sound, the distance between the user and the listening earphone is judged according to the sound intensity heard by the earphone until the user enters a room where the listening earphone is located, and Bluetooth connection is reestablished with the listening earphone. With the movement of the user, the position relationship between the listening earphone or the mobile phone and the surrounding transfer equipment is adjusted to ensure and improve the communication quality.

The above embodiments are preferred embodiments of the present application, and the protection scope of the present application is not limited by the above embodiments, so: all equivalent changes made according to the structure, shape and principle of the present application shall be covered by the protection scope of the present application.

Claims

1. A management method of a dual-mode TWS Bluetooth headset is characterized by comprising the following steps:

an information concentration step, wherein first sound source information and second sound source information of two earphones are concentrated in a single earphone;

2. The management method of a dual-mode TWS Bluetooth headset of claim 1, wherein the state determining step is preceded by a master-slave determining step:

the information centralizing step comprises:

3. The management method of a dual mode TWS bluetooth headset of claim 1, wherein the mode determining step comprises:

s1, reading environmental characteristics;

4. The management method of a dual-mode TWS Bluetooth headset of claim 3, wherein the step of determining the environmental characteristics by the acoustic model comprises:

5. The method for managing a dual-mode TWS Bluetooth headset of claim 4, wherein the step of evaluating the degree of training of the local environmental acoustic model in S2 includes an adjusting step and a scoring step, the adjusting step including:

the scoring step comprises:

6. The method for managing a dual mode TWS Bluetooth headset of claim 5, further comprising the steps of:

7. The method for managing a dual mode TWS Bluetooth headset of claim 6, wherein the state determining step comprises the steps of:

step two:

8. The method for managing a dual mode TWS bluetooth headset of claim 7, wherein the method for the master to select whether to send the broadcast packet to the slave or not based on the flag of the number of times of forwarding the broadcast packet and the address sequence comprises:

9. The method for managing a dual mode TWS bluetooth headset of claim 8, wherein the connection preference step includes: