WO2020133183A1

WO2020133183A1 - Audio data synchronization method and device

Info

Publication number: WO2020133183A1
Application number: PCT/CN2018/124768
Authority: WO
Inventors: 朱宇洪; 张景云; 倪观军; 王良
Original assignee: 华为技术有限公司
Priority date: 2018-12-28
Filing date: 2018-12-28
Publication date: 2020-07-02
Also published as: CN113228701B; CN113228701A; CN115426064A

Abstract

Disclosed are an audio data synchronization method and device, which relate to the field of short-distance communications. When a wireless connection is established between a mobile phone and a plurality of Bluetooth devices, play-level synchronization of audio data between the plurality of Bluetooth devices is realized. The method comprises: an electronic device sending, by means of a first CIS of a CIG, a data packet to a first earbud that establishes a wireless connection with the electronic device, and sending, by means of a second CIS of the CIG, the data packet to a second earbud that establishes a wireless connection with the electronic device, wherein the data packet comprises: audio data, an audio timestamp of the data packet and an initial Bluetooth timestamp; the initial Bluetooth timestamp is a Bluetooth timestamp of a first data packet transmitted after the electronic device establishes the first CIS with the first earbud and establishes the second CIS with the second earbud; the audio timestamp and the initial Bluetooth timestamp are used for the first earbud and the second earbud to determine a CIG play point; and the CIG play point is a time point at which the first earbud and the second earbud play the audio data.

Description

Audio data synchronization method and equipment

Technical field

The present application relates to the field of short-range communications, and in particular to a method and device for synchronizing audio data.

Background technique

Bluetooth (Bluetooth) is a wireless technology standard that enables short-range data interaction between different devices. For example, the mobile phone can turn on the Bluetooth module and perform short-range data interaction with the Bluetooth headset to use the Bluetooth headset as the audio input/output device of the mobile phone to implement functions such as calling and music playback.

When a mobile phone establishes a wireless connection with multiple (two or more) Bluetooth devices, there may be a need to achieve playback-level synchronization of audio data between the multiple Bluetooth devices.

Summary of the invention

The purpose of the present application is to provide a method and device for synchronizing audio data. When a wireless connection is established between a mobile phone and multiple Bluetooth devices, playback level synchronization of audio data between multiple Bluetooth devices is achieved.

The above and other objectives will be achieved by the features in the independent claims. Further implementations are reflected in the dependent claims, the description and the drawings.

In a first aspect, a method for synchronizing audio data is provided. The method can be applied to an electronic device that can establish a wireless connection with a first earplug and a second earplug, respectively; the method can include: The first isochronous stream (connected, isochronous stream, CIS) of the connected isochronous group (CIG) sends a data packet to the first earbud, and sends the data to the second earbud through the second CIS of the CIG Packet; wherein, the data packet may include: audio data, an audio time stamp of the data packet, and an initial Bluetooth time stamp. The initial Bluetooth timestamp is the Bluetooth timestamp of the first data packet transmitted after the electronic device establishes the first CIS with the first earbud, and establishes the second CIS with the second earbud; the audio timestamp and the initial Bluetooth timestamp are used for the first earbud Determine the CIG play point with the second earplug, and the CIG playpoint is the time point when the first earplug and the second earplug play the audio data in the data packet.

Wherein, the above-mentioned electronic device may be a device such as a mobile phone, and the above-mentioned first earplug and the above-mentioned second earplug may be Bluetooth earphones, such as the left earplug and the right earplug of a true wireless stereo (TWS) headset, respectively. In addition, the first earplug and the second earplug may be replaced with two other Bluetooth devices, respectively. The device shape of the two Bluetooth devices can be the same (such as two wireless speakers) or different (such as one wireless speaker and one wireless headset).

With this technical solution, the electronic device sends the audio time stamp carrying the data packet and the initial Bluetooth time stamp data packet to the two Bluetooth devices, such as the first earbud and the second earbud, so that the first earbud and the second earbud can According to the audio timestamp and the initial Bluetooth timestamp, the playback time point of the audio data in the data packet is determined, that is, the CIG playback point. The audio timestamp and initial Bluetooth timestamp in the data packet can ensure the synchronization of the Bluetooth clock and the audio clock. In other words, the first earplug and the second earplug can determine the CIG play point of the audio data according to the synchronized Bluetooth clock and the synchronized audio clock. In this way, the play level synchronization of audio data is realized by using the determined CIG play point. In addition, in this embodiment, the Bluetooth clock and the audio clock are decoupled and synchronized independently, and the hardware clock synchronization of the Bluetooth clock and the audio clock is not required. For the Bluetooth clock, it is only necessary to carry the BTS of the first data packet in each data packet, that is, the initial Bluetooth time stamp, and it is not necessary to obtain the BTS of each data packet to reduce the load. Corresponding to the audio clock, each data packet is independently labeled. Therefore, while achieving playback-level synchronization of audio data, it can also support independent playback of each data packet. In this way, the technical solution of this embodiment can be applied to scenarios where single earplugs are used alone, both earplugs are used simultaneously, and single ear and binaural switching is performed.

In some embodiments, in the scenario of single ear use (eg, using the first earplug), the electronic device may send a first data packet to the first earplug through the first CIS of CIG, and the first data packet may include: Audio data, audio timestamp of the first data packet, and initial Bluetooth timestamp. The initial Bluetooth timestamp is the Bluetooth timestamp of the first data packet transmitted after the electronic device establishes the first CIS with the first earbud; the audio timestamp and initial Bluetooth timestamp of the first data packet can be used for the first earbud to determine CIG playback Point, the CIG play point is the time point when the first earbud plays the audio data in the first data packet.

In the scenario where one ear is converted to two ears, the electronic device can not only send the second data packet to the first earplug through the first CIS of CIG, but also send the second data packet to the second earplug through the second CIS of CIG. A data packet, the second data packet may include: audio data, an audio time stamp of the second data packet, and an initial Bluetooth time stamp. The audio timestamp and initial Bluetooth timestamp of the second data packet can be used to determine the CIG playback point of the first earplug and the second earplug, and the CIG playback point is the time point at which the first earplug and the second earplug play the audio data in the second data packet .

That is to say, in this embodiment, the electronic device may calculate the audio timestamp of the data packet for each data packet, and carry it in the data packet to notify the first earplug and/or the second earplug.

In a possible implementation, before the electronic device sends a data packet to the first earplug through the first CIS of CIG, and sends a data packet to the second earplug through the second CIS of CIG, the method may further include: The device establishes the CIG's first CIS with the first earplug and the CIG's second CIS with the second earplug.

In another possible implementation manner, the electronic device establishes the first CIS of the CIG with the first earplug and the second CIS of the CIG with the second earplug, which may specifically include: the electronic device sends a CIS establishment request message to the first earplug, Receive the CIS establishment response message sent by the first earbud, send the CIS establishment message to the first earbud; the electronic device sends the CIS establishment request message to the second earbud, receive the CIS establishment response message sent by the second earbud, and send the CIS establishment to the second earbud Message; wherein, the CIS establishment request message includes transmission delay and playback delay, and the transmission delay and playback delay are used for the first earplug and the second earplug to determine the CIG play point.

In another possible implementation manner, before the electronic device establishes the first CIS of the CIG with the first earplug and the second CIS of the CIG with the second earplug, the method may further include: the electronic device and the first earplug respectively Pair the second earbuds; the electronic device establishes a first asynchronous connection-oriented (ACL) link with the first earbud; the electronic device establishes a second ACL link with the second earbud; the electronic device establishes with the first earbud The first CIS of CIG establishes the second CIS of CIG with the second earplug, which may specifically include: the electronic device establishes the first CIS of CIG with the first earplug through the first ACL link, and the second earplug through the second ACL link Established CIG's second CIS.

In another possible implementation, the time offset between the audio time stamp of the data packet and the Bluetooth time stamp of the data packet meets at least one of the following conditions: less than or equal to the transmission delay, and less than or equal to the playback delay , Less than or equal to the sum of transmission delay and playback delay. In this way, the continuity of audio data at the playback level can be guaranteed.

In a second aspect, a method for synchronizing audio data is provided, which is applied to a first earplug and a second earplug that establish a wireless connection with an electronic device. The method may include: the first earplug receives a first CIS from the electronic device through the first CIS of the CIG The data packet; the second earplug receives the data packet from the electronic device through the second CIS of CIG; the data packet may include: audio data, an audio timestamp of the data packet, and an initial Bluetooth timestamp, the initial Bluetooth timestamp is for the electronic device Establish the first CIS with the first earbud, and the Bluetooth timestamp of the first data packet transmitted after establishing the second CIS with the second earbud; the first earbud and the second earbud are based on the audio timestamp, the initial Bluetooth timestamp, and the transmission time The delay and playback delay determine the CIG playback point; the first earplug and the second earplug respectively play the audio data in the data packet at the CIG playback point.

In some embodiments, the first earplug and the second earplug may be a left earplug and a right earplug of a Bluetooth headset (such as a TWS headset), respectively. In addition, the first earplug and the second earplug may be replaced with two other Bluetooth devices, respectively. The device shape of the two Bluetooth devices can be the same (such as two wireless speakers) or different (such as one wireless speaker and one wireless headset).

With this technical solution, two Bluetooth devices, such as the first earplug and the second earplug, respectively receive data packets from the electronic device, and the data packet carries the audio timestamp of the data packet, and the initial Bluetooth timestamp, the first earplug And the second earbud can determine the playing time point of the audio data in the data packet according to the audio timestamp and the initial Bluetooth timestamp, that is, the CIG playing point. The audio timestamp and initial Bluetooth timestamp in the data packet can ensure the synchronization of the Bluetooth clock and the audio clock. In other words, the first earplug and the second earplug can determine the CIG play point of the audio data according to the synchronized Bluetooth clock and the synchronized audio clock. In this way, the play level synchronization of audio data is realized by using the determined CIG play point. In addition, in this embodiment, the Bluetooth clock and the audio clock are decoupled and synchronized independently, and no hardware clock synchronization of the Bluetooth clock and the audio clock is required. For the Bluetooth clock, it is only necessary to carry the BTS of the first data packet in each data packet, that is, the initial Bluetooth time stamp, and it is not necessary to obtain the BTS of each data packet to reduce the load. Corresponding to the audio clock, each data packet is independently labeled. Therefore, while achieving playback-level synchronization of audio data, it can also support independent playback of each data packet. In this way, the technical solution of this embodiment can be applied to scenarios where single earplugs are used alone, both earplugs are used simultaneously, and single ear and binaural switching is performed.

In some embodiments, in the scenario of single ear use (eg, using the first earplug), the first earplug may receive the first data packet from the electronic device through the first CIS of CIG, and the first data packet may include : Audio data, audio timestamp of the first data packet, and initial Bluetooth timestamp. The initial Bluetooth timestamp is the Bluetooth timestamp of the first data packet transmitted after the electronic device establishes the first CIS with the first earbud; the first earbud can determine the CIG playback point according to the audio timestamp of the first data packet and the initial Bluetooth timestamp , The first earplug can play the audio data in the first data packet at the CIG play point.

In the scenario where one ear is converted to two ears, not only the first earplug can receive the second data packet from the electronic device through the CIG’s first CIS, the second earplug can also receive the electronic data through the CIG’s second CIS A second data packet of the device. The second data packet may include: audio data, an audio time stamp of the second data packet, and an initial Bluetooth time stamp. The first earplug and the second earplug can determine the CIG playback point according to the audio timestamp and the initial Bluetooth timestamp of the second data packet, respectively, and the first earplug and the second earplug can play the audio data in the second data packet at the CIG playback point, respectively .

In a possible implementation manner, before the first earplug receives the data packet from the electronic device through the first CIS of CIG, and the second earplug receives the data packet from the electronic device through the second CIS of CIG, the method It may include: the first earplug and the electronic device establish a CIG first CIS, and the second earplug and the electronic device establish a CIG second CIS.

In another possible implementation manner, the first earplug and the electronic device establish a CIG first CIS, and the second earplug and the electronic device establish a CIG second CIS, which may specifically include: the first earplug receives the CIS establishment from the electronic device Request message, send CIS establishment response message to the electronic device, receive CIS establishment message from the electronic device; second earbuds receive CIS establishment request message from the electronic device, send CIS establishment response message to the electronic device, receive CIS establishment message from the electronic device Message; Among them, CIS establishment request message includes transmission delay and play delay.

In another possible implementation manner, before the first earplug establishes the first CIS of the CIG with the electronic device, and the second earplug establishes the second CIS of the CIG with the electronic device, the method may further include: the first earplug and the second The earplugs are paired with the electronic device; the first earplug and the electronic device establish a first ACL link; the second earplug and the electronic device establish a second ACL link; the first earplug and the electronic device establish a CIG first CIS, and the second earplug Establishing the second CIS of the CIG with the electronic device may specifically include: establishing the first CIS of the CIG with the electronic device through the first ACL link, and establishing the second of the CIG with the electronic device through the second ACL link CIS.

In another possible implementation manner, the first earplug and the second earplug respectively determine the CIG play point according to the audio timestamp, the initial Bluetooth timestamp, the transmission delay, and the playback delay, which may specifically include: the first earplug according to the audio time Poke, initial Bluetooth timestamp and transmission delay determine the CIG synchronization point, the first earbud determines the CIG playback point based on the determined CIG synchronization point and playback delay; the second earbud determines the audio Bluetooth timestamp, initial Bluetooth timestamp and transmission delay CIG synchronization point, the first earplug determines the CIG playback point according to the determined CIG synchronization point and playback delay. In a possible implementation, the CIG synchronization point is determined according to the audio timestamp, the initial Bluetooth timestamp, and the transmission delay, which may specifically include: SYNC_m_n=Init_BTS+ATS_n+transport_delay. Among them, SYNC_m_n is the CIG synchronization point, Init_BTS is the initial Bluetooth timestamp, ATS_n is the audio timestamp, transport_delay is the transmission delay, m indicates that the packet is the mth packet on the Bluetooth clock, and n indicates that the packet is The nth packet on the audio clock. The CIG playback point is determined according to the determined CIG synchronization point and playback delay, which may specifically include: Render_m_n=SYNC_m_n+Presentation_delay. Among them, Render_m_n is the CIG playback point, and Presentation_delay is the playback delay.

In another possible implementation manner, the above-mentioned determination of the CIG playback point according to the audio timestamp, initial Bluetooth timestamp, transmission delay and playback delay may specifically include: determining CIG playback according to Render_m_n=Init_BTS+transport_delay+ATS_n+Presentation_delay point.

In a third aspect, an electronic device is provided. The electronic device may include: one or more processors, a memory, a wireless communication module, and a mobile communication module; wherein: the memory, the wireless communication module, the mobile communication module, and one or more processes Is coupled to a memory, and the memory is used to store computer program code. The computer program code includes computer instructions. When one or more processors execute the computer instructions, the electronic device may perform any one of the first aspect or any possible implementation manner of the first aspect. A method for synchronizing audio data.

According to a fourth aspect, a Bluetooth headset is provided. The Bluetooth headset may include at least one earphone; the earphone may include: a processor, a memory, a wireless communication module, a receiver, and a microphone; a memory, a wireless communication module, a receiver, and a microphone are coupled to the processor , The memory is used to store computer program code, the computer program code includes computer instructions; when the processor executes the computer instructions, the earplugs can execute the audio data described in any one of the second aspect or any possible implementation manner of the second aspect Synchronization method. For example, the Bluetooth headset includes two earplugs, such as the first earplug and the second earplug, and the first earplug and the second earplug may perform any one of the second aspect or any possible implementation manner of the second aspect. Synchronization method of audio data.

According to a fifth aspect, a Bluetooth audio system is provided. The Bluetooth audio system may include: an electronic device, a first earplug, and a second earplug; wherein: the electronic device may be used to send a data packet to the first earplug through the first CIS of CIG , Sending a data packet to the second earbud through the second CIS of CIG; the data packet may include: audio data, an audio timestamp of the data packet, and an initial Bluetooth timestamp, the initial Bluetooth timestamp is the electronic device and the first earbud Establish the first CIS, and the Bluetooth timestamp of the first data packet transmitted after establishing the second CIS with the second earplug; the first earplug can be used to receive the data packet, according to the audio timestamp and the initial Bluetooth timestamp in the data packet, And the transmission delay and playback delay to determine the CIG playback point, and play the audio data in the received data packet at the determined CIG playback point; the second earplug can be used to receive the data packet, according to the audio timestamp and The initial Bluetooth timestamp, transmission delay and playback delay determine the CIG playback point, and the audio data in the received data packet is played at the determined CIG playback point.

In a sixth aspect, a Bluetooth chip is provided, which can be applied to electronic devices; the Bluetooth chip can include a memory, a processor, and a radio frequency module, and the memory is used to store computer program code, and the computer program code includes computer instructions; wherein: processing The computer executes computer instructions stored in the memory to control the radio frequency module to send a data packet to the first earplug through the first CIS of CIG and a data packet to the second earplug through the second CIS of CIG; wherein, the data packet may include: audio data, The audio timestamp of the data packet and the initial Bluetooth timestamp. The initial Bluetooth timestamp is the Bluetooth timestamp of the first data packet transmitted after the electronic device establishes the first CIS with the first earbud and establishes the second CIS with the second earbud; The audio timestamp and the initial Bluetooth timestamp are used by the first earplug and the second earplug to determine the CIG playback point, and the CIG playback point is the time point at which the first earplug and the second earplug play the audio data in the data packet.

In a seventh aspect, a Bluetooth chip is provided, which can be applied to the earbuds of a Bluetooth headset; the Bluetooth chip includes a memory, a processor, and a radio frequency module, and the memory is used to store computer program code, and the computer program code includes computer instructions; the processor executes The computer instructions stored in the memory control the radio frequency module to receive the data packet from the electronic device through the CIG CIS; wherein, the data packet may include: audio data, audio time stamp of the data packet, and initial Bluetooth time stamp, the initial Bluetooth time stamp is The Bluetooth timestamp of the first data packet transmitted after the CIS is established; the CIG playback point is determined based on the audio timestamp, initial Bluetooth timestamp, transmission delay and playback delay; so that the microphone of the earbud of the Bluetooth headset can be at the CIG playback point Play audio data in the packet.

According to an eighth aspect, a computer storage medium is provided, including computer instructions, which, when run on an electronic device, cause the electronic device to perform audio as described in any one of the first aspect or any possible implementation manner of the first aspect Data synchronization method.

In a ninth aspect, a computer storage medium is provided, including computer instructions, which when executed on an earbud of a Bluetooth headset, causes the earbud to perform any one of the second aspect or any possible implementation manner of the second aspect Synchronization method of audio data.

According to a tenth aspect, there is provided a computer program product which, when the computer program product runs on a computer, causes the computer to execute any one of the above audio data synchronization methods.

According to an eleventh aspect, there is provided a device for synchronizing audio data. The device for synchronizing audio data may be included in an electronic device, and the device has the electronic device in any of the methods for implementing the first aspect and the possible implementation manners of the first aspect. Behavioral function. The function can be realized by hardware, or can also be realized by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a sending module or unit, a processing module or unit, a receiving module or unit, etc.

According to a twelfth aspect, another apparatus for synchronizing audio data is provided. The apparatus for synchronizing audio data may be included in an earplug of a Bluetooth headset. The apparatus has any method for implementing the second aspect and the possible implementation manners of the second aspect. The function of the earplugs of the Bluetooth headset (for example, the first earplug or the second earplug). The function can be realized by hardware, or can also be realized by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a receiving module or unit, a processing module or unit, a playing module or unit, a sending module or unit, etc.

It should be understood that the description of technical features, technical solutions, beneficial effects, or similar language in this embodiment does not imply that all features and advantages can be achieved in any single embodiment. On the contrary, it can be understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can also be combined in any suitable manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of the specific embodiments. In other embodiments, additional technical features and beneficial effects may also be identified in specific embodiments that do not reflect all the embodiments.

BRIEF DESCRIPTION

1 is a schematic structural diagram of a Bluetooth audio system provided by an embodiment;

2 is a schematic structural diagram of an earplug of a TWS earphone provided by an embodiment;

3 is a schematic diagram of an example of a product form of a TWS earphone provided by an embodiment;

4 is a schematic structural diagram of an electronic device according to an embodiment;

5 is a schematic diagram of the composition of a Bluetooth transmission architecture provided by an embodiment;

6 is a schematic flowchart of a method for synchronizing audio data according to an embodiment;

7 is a schematic diagram of an audio data transmission principle provided by an embodiment;

8 is a schematic structural diagram of a data packet provided by an embodiment;

9 is a schematic structural diagram of an earplug of a Bluetooth headset provided by an embodiment;

10 is a schematic structural diagram of an electronic device according to another embodiment.

detailed description

Today, Bluetooth is managed by the Bluetooth Special Interest Group (SIG), which is mainly responsible for the formulation of Bluetooth technical specifications. The current Bluetooth technical specifications are mainly used for point-to-point data transmission based on basic rate (BR)/enhanced data (EDR). If the mobile phone establishes a wireless connection with multiple (such as two) Bluetooth devices, it is necessary to realize point-to-multipoint data transmission between the mobile phone and the two Bluetooth devices. Obviously, the current point-to-point data transmission based on BR/EDR can no longer meet the demand. In addition, these two Bluetooth devices may also have a need to achieve playback-level synchronization of audio data between the devices. For example, with the advancement of technology, Bluetooth headsets with two independent headset bodies gradually enter people's vision. For example, such a Bluetooth headset with two independent headset bodies may be called a TWS headset, and the two independent headset bodies contained therein may be called left earplugs and right earplugs, respectively. If the TWS headset is used as the audio input/output device of the mobile phone, the point-to-multipoint data transmission between the mobile phone and the left and right earplugs of the TWS headset needs to be realized. In addition, the playback level synchronization of audio data between the left and right earbuds of the TWS headset is also required. That is to say, it is necessary to realize the synchronous transmission of the playback level of point-to-multipoint audio data.

The Bluetooth Technology Alliance is already working on the development of point-to-multipoint Bluetooth technical specifications for the realization of playback-level synchronous transmission of point-to-multipoint audio data. For example, the Bluetooth Low Energy (BLE) isochronous (ISO) channel defines a transmission mechanism. Under this transmission mechanism, a master device (Master, referred to as M, such as the above mobile phone) based on the BLE ISO channel can send audio data to multiple slave devices (Slave, referred to as S, such as the left and right earplugs of the above TWS headset) , And can achieve the playback level synchronization of the audio data of the multiple slave devices. That is to say, the transmission mechanism can be used to realize the synchronous transmission of the playback level of point-to-multipoint audio data.

Specifically, CIG is defined in the above transmission mechanism, and CIG is the concept of group. A CIG can include multiple CIS. A master device can send audio data to multiple slave devices through multiple CISs in a CIG. Multiple slave devices correspond to multiple CISs in one-to-one manner, so that point-to-multipoint data transmission can be achieved. It can be understood that the transmission channel between the master device and each slave device is defined as CIS, and multiple CISs belong to the same CIG. In addition, multiple CISs in a CIG can share the same CIG synchronization point (CIG synchronization point) and CIG playback point (CIG presentation point). At the CIG synchronization point, multiple slave devices corresponding to CIS all receive audio data. At the CIG playback point, multiple slave devices corresponding to CIS all play audio data. That is, multiple slave devices can achieve playback-level synchronization of audio data of multiple slave devices according to the shared CIG synchronization point and CIG playback point, respectively.

It should be noted that, in this embodiment, the playback level synchronization of audio data of multiple slave devices may refer to: multiple slave devices may receive audio data from the master device separately, and for the user, the same Play the received audio data at a point in time. For example, the master device may be a mobile phone, and the multiple slave devices may be the left earplug and the right earplug of the TWS headset. When using a TWS headset as an audio input/output device for a mobile phone, the mobile phone can send audio data to the left and right earplugs of the TWS headset through two CISs in a CIG, respectively. After receiving the audio data from the mobile phone, the left earplug and the right earplug of the TWS earphone can play the received audio data at the same time point as perceived by the user, so as to realize the playback level synchronization of the audio data.

In the above transmission mechanism, only theoretical time points (such as the above-mentioned CIG synchronization point and CIG playback point) are given to achieve playback-level synchronization of audio data.

This embodiment provides a method for synchronizing audio data, which can be applied to the process of transmitting audio data between an electronic device and multiple Bluetooth devices. Among them, the electronic device establishes a wireless connection with each of the plurality of Bluetooth devices, and the electronic device sends audio data to each of the plurality of Bluetooth devices based on the wireless connection with each Bluetooth device . These multiple Bluetooth devices can determine the CIG synchronization point and CIG playback point based on the synchronized audio clock (clock) and the synchronized Bluetooth clock (Bluetooth clock), so that the determined CIG synchronization point and CIG playback point can be used to realize audio data Play level synchronization.

It should be noted that, in this embodiment, the above audio data (also called audio stream) may include: voice data during a call, which is performed using an application program (eg, WeChat, Facebook) Voice data or voice messages during voice calls or video calls, prompt tones (such as call prompt tone, ring back tone, SMS prompt tone, etc.), music, voice data when playing video, navigation sound, etc.

The implementation of this embodiment will be described in detail below with reference to the drawings.

As shown in FIG. 1, the audio data synchronization method provided in this embodiment can be applied to a Bluetooth audio system composed of a peripheral device 101 and an electronic device 102.

Among them, the peripheral device 101 and the electronic device 102 may use Bluetooth to establish a Bluetooth connection. Based on the established Bluetooth connection, short-range data interaction can be realized between the peripheral device 101 and the electronic device 102. Exemplarily, the audio data may be transmitted between the peripheral device 101 and the electronic device 102 based on the Bluetooth connection. For example, based on the Bluetooth connection described above, the peripheral device 101 can serve as an audio input/output device of the electronic device 102 to realize a call. For another example, based on the Bluetooth connection described above, the peripheral device 101 can be used as an output device of the electronic device 102, such as a speaker to play music.

In some embodiments, the peripheral device 101 may be a device such as a wireless headset, a wireless speaker, a wireless bracelet, a wireless vehicle, a wireless smart glasses, etc., which includes a main body. For example, the electronic device 102 may serve as a master device to transmit audio data to a peripheral device 101 as a slave device through a CIS in a CIG. Wherein, the wireless earphone may be a headphone, earplugs or other portable listening devices. For another example, the electronic device 102 can serve as a master device to transmit audio data to multiple peripheral devices 101 as slave devices through multiple CISs in a CIG, and the multiple CIS correspond to the multiple peripheral devices 101 in one-to-one correspondence. The audio data synchronization method provided in this embodiment can also be applied to a scenario where the electronic device 102 sends audio data to multiple peripheral devices 101. Using the method provided in this embodiment, multiple peripheral devices 101 can determine the CIG synchronization point and the CIG playback point based on the synchronized audio clock and the synchronized Bluetooth clock, thereby using the determined CIG synchronization point and the CIG playback point to realize audio data Play level synchronization.

In other embodiments, the peripheral device 101 may also be a Bluetooth headset (it may also be referred to as a TWS headset, which will be described using the TWS headset as an example in the following embodiments), a Bluetooth speaker, smart glasses, and other devices. The peripheral device 101 includes two main bodies, and no wire connection is needed between the two main bodies, and they can cooperate with each other and work together. In addition, for the audio data transmitted by the electronic device 102 to the two main bodies of the peripheral device 102, the two main bodies have a requirement to achieve playback-level synchronization of the audio data.

In this embodiment, the electronic device 102 can send audio data to two main bodies of the peripheral device 101 through two CISs in one CIG, and the two CISs correspond to the two main bodies of the peripheral device in one-to-one correspondence. As an example, the peripheral device 101 shown in FIG. 1 is illustrated with a TWS earphone as an example. Among them, the TWS earphone includes two main bodies (such as a headphone main body), such as a left earplug 101-1 and a right earplug 101-2. No wire connection is required between the left earplug 101-1 and the right earplug 101-2, and they can cooperate with each other and work together, for example, to achieve stereo playback. The electronic device 102 can serve as a master device to send audio data to the left earplug 101-1 and the right earplug 101-2 as slave devices through two CISs in one CIG, respectively. Moreover, the left earplug 101-1 and the right earplug 101-2 of the TWS earphone can determine the CIG synchronization point and the CIG playback point based on the synchronized audio clock and the synchronized Bluetooth clock, thereby using the determined CIG synchronization point and CIG playback point Play level synchronization of audio data. In this embodiment, the structures of the left earplug 101-1 and the right earplug 101-2 of the TWS earphone may be as shown in FIG. 2 and will be described in detail in the following embodiments.

In some embodiments, the electronic device 102 may be a mobile phone (as shown in FIG. 1), a tablet computer, a desktop type, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC) , Netbooks, and cellular phones, personal digital assistants (personal digital assistants, PDAs), augmented reality (augmented reality, AR)\virtual reality (virtual reality, VR) devices, media players, televisions and other devices. The specific form of the device is not subject to special restrictions. In this embodiment, the structure of the electronic device 102 may be as shown in FIG. 4 and will be described in detail in the following embodiments.

Please refer to FIG. 2, which is a schematic structural diagram of an earplug (left earplug or right earplug) of a TWS earphone provided by this embodiment. As shown in FIG. 2, the earplugs of the TWS headset may include: a processor 201, a memory 202, a sensor 203, a wireless communication module 204, at least one receiver 205, at least one microphone 206, and a power supply 207.

The memory 202 can be used to store application code, such as an application code for establishing a Bluetooth connection with another earbud of the TWS headset and pairing the earbud with the electronic device 102 described above. The processor 201 can control the execution of the above application program code to implement the function of the earplug of the TWS earphone in this embodiment.

The memory 202 may also store a Bluetooth address for uniquely identifying the earbud, and a Bluetooth address of another earbud where the TWS earphone is stored. In addition, the memory 202 may also store connection data of the electronic device that has been successfully paired with the earplug before. For example, the connection data may be the Bluetooth address of the electronic device that has successfully paired with the earbud. Based on the connection data, the earplug can be automatically paired with the electronic device without having to configure a connection with it, such as performing legality verification. The aforementioned Bluetooth address may be a Media Access Control (Media Access Control, MAC) address.

The sensor 203 may be a distance sensor or a proximity light sensor. The processor 201 of the earplug can determine whether it is worn by the user through the sensor 203. For example, the processor 201 of the earplug may use a proximity light sensor to detect whether there is an object near the earplug, thereby determining whether the earplug is worn by the user. When it is determined that the earplug is worn, the processor 201 of the earplug may turn on the receiver 205. In some embodiments, the earplug may also include a bone conduction sensor, combined with a bone conduction earphone. The bone conduction sensor can obtain the vibration signal of the vibrating bone mass of the voice part, and the processor 201 parses out the voice signal to realize the control function corresponding to the voice signal. In other embodiments, the earplug may further include a touch sensor or a pressure sensor, which are used to detect the user's touch operation and pressing operation, respectively. In other embodiments, the earplug may further include a fingerprint sensor, which is used to detect a user's fingerprint and identify the user's identity. In other embodiments, the earplug may further include an ambient light sensor, and the processor 201 of the earplug may adaptively adjust some parameters, such as volume, according to the brightness of the ambient light sensed by the ambient light sensor.

The wireless communication module 204 is used to support short-distance data interaction between the left earplug and the right earplug of the TWS earphone, and between the earplug and various electronic devices, such as the electronic device 102 described above. In some embodiments, the wireless communication module 204 may be a Bluetooth transceiver. The earplugs of the TWS headset can establish a Bluetooth connection with the electronic device 102 through the Bluetooth transceiver to achieve short-range data interaction between the two.

The receiver 205, which may also be referred to as an "earpiece", may be used to convert audio electrical signals into sound signals and play them. For example, when the earplug of the TWS earphone serves as the audio output device of the electronic device 102 described above, the receiver 205 may convert the received audio electrical signal into a sound signal and play it.

The microphone 206, which may also be referred to as a "microphone" or a "microphone," is used to convert sound signals into audio electrical signals. For example, when the earplug of the TWS earphone is used as the audio input device of the electronic device 102, when the user speaks (such as talking or sending a voice message), the microphone 206 can collect the user's voice signal and convert it into an audio electrical signal. The above audio electrical signal is the audio data in this embodiment.

The power supply 207 can be used to supply power to various components included in the earplugs of the TWS earphone. In some embodiments, the power source 207 may be a battery, such as a rechargeable battery.

Usually, TWS earphones will be equipped with a headphone box (eg, 301 shown in Figure 3). As shown in FIG. 3, the earphone box 301 may include a cavity 301-1 and a box cover 301-2. The cavity 301-1 can be used to store the left earplug and the right earplug of the TWS earphone. As shown in FIG. 3 in combination with FIG. 1, the cavity 301-1 of the earphone box 301 can be used to house the left earplug 101-1 and the right earplug 101-2 of the TWS earphone. In addition, the earphone box 301 can also charge the left earplug and the right earplug of the TWS earphone. Correspondingly, in some embodiments, the earplugs of the TWS earphone may further include: an input/output interface 208.

The input/output interface 208 may be used to provide any wired connection between the earplug of the TWS earphone and the earphone box (such as the cavity 301-1 of the earphone box 301 described above). In some embodiments, the input/output interface 208 may be an electrical connector. For example, when the earplug of the TWS earphone is placed in the cavity 301-1 of the earphone box 301, the earplug of the TWS earphone may be electrically connected to the earphone box 301 (such as the input/output interface with the earphone box 301) through the electrical connector. After the electrical connection is established, the earphone box 301 can charge the power supply 207 of the earplug of the TWS earphone. After the electrical connection is established, the earplugs of the TWS earphone can also perform data communication with the earphone box 301. For example, the processor 201 of the earplug of the TWS earphone may receive the pairing instruction from the earphone box 301 through the electrical connection. The pairing command is used to instruct the processor 201 of the earphone of the TWS earphone to turn on the wireless communication module 204, so that the earphone of the TWS earphone can use a corresponding wireless communication protocol (such as Bluetooth) to pair with the electronic device 102.

Of course, the earplugs of the aforementioned TWS earphones may not include the input/output interface 208. In this case, the earplugs can implement charging or data communication functions based on the Bluetooth connection established with the earphone box 301 through the wireless communication module 204 described above.

In addition, in some embodiments, the earphone box (such as the aforementioned earphone box 301) may further include a processor, a memory, and other components. The memory can be used to store application program codes, and is controlled and executed by the processor of the earphone box 301 to realize the function of the earphone box 301. E.g. When the user opens the lid 301-2 of the earphone box 301, the processor of the earphone box 301 can send the pairing to the earplug of the TWS headset in response to the user's operation of opening the lid 301-2 by executing the application code stored in the memory Orders etc.

It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the earplugs of the TWS earphone. It may have more or fewer components than shown in FIG. 2, two or more components may be combined, or may have different component configurations. For example, the earplug may also include an indicator light (which can indicate the status of the earplug's power level, etc.), a dust filter (which can be used with the earpiece), and other components. The various components shown in FIG. 2 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing or application specific integrated circuits.

It should be noted that the structure of the left earplug and the right earplug of the TWS earphone may be the same. For example, the left earplug and the right earplug of the TWS earphone may both include the components shown in FIG. 2. Alternatively, the structure of the left earplug and the right earplug of the TWS earphone may be different. For example, one earplug (such as the right earplug) of the TWS earphone may include the components shown in FIG. 2, and the other earplug (such as the left earplug) may include other components in FIG. 2 other than the microphone 206.

Please refer to FIG. 4, which is a schematic structural diagram of an electronic device 102 in the Bluetooth audio system shown in FIG. 1 according to an embodiment of the present application. As shown in FIG. 4, the electronic device 102 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, and a battery 142 , Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193 , A display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 102. In other embodiments, the electronic device 102 may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), and an image signal processor. (image)signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Among them, the different processing units may be independent devices or may be integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 102. The controller can generate the operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetch and execution.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. The repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. Interfaces can include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit, sound, I2S) interface, pulse code modulation (pulse code modulation (PCM) interface, universal asynchronous transceiver (universal) asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and /Or universal serial bus (USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, the processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., respectively through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 102.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, to realize the function of answering the phone call through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to implement the function of answering the phone call through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI) and so on. In some embodiments, the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the electronic device 102. The processor 110 and the display screen 194 communicate through the DSI interface to realize the display function of the electronic device 102.

The GPIO interface can be configured via software. The GPIO interface can be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 to the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 102, and can also be used to transfer data between the electronic device 102 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in this embodiment is only a schematic description, and does not constitute a limitation on the structure of the electronic device 102. In other embodiments, the electronic device 102 may also use different interface connection methods in the foregoing embodiments, or a combination of multiple interface connection methods.

The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive the charging input of the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through the wireless charging coil of the electronic device 102. While the charging management module 140 charges the battery 142, it can also supply power to the electronic device through the power management module 141.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, wireless communication module 160, and the like. The power management module 141 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the electronic device 102 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 102 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the electronic device 102. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), and the like. The mobile communication module 150 can receive the electromagnetic wave from the antenna 1 and filter, amplify, etc. the received electromagnetic wave, and transmit it to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it to electromagnetic wave radiation through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110, and may be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (Bluetooth, BT), and global navigation satellites that are applied to the electronic device 102 Wireless communication solutions such as global navigation (satellite system, GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives the electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and convert it to electromagnetic waves through the antenna 2 to radiate it out.

In some embodiments, the antenna 1 of the electronic device 102 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160 so that the electronic device 102 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include a global mobile communication system (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long-term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation system (BDS), and a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite-based augmentation system (SBAS). For example, in this embodiment, the electronic device 102 may utilize the wireless communication module 160 to establish a Bluetooth connection with a peripheral device through a wireless communication technology, such as Bluetooth (BT). Based on the established Bluetooth connection, the electronic device 102 can send audio data to the peripheral device and can also receive audio data from the peripheral device.

The electronic device 102 realizes a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations, and is used for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can use a liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active matrix organic light-emitting diode (active-matrix organic light-emitting diode) emitting diode, AMOLED, flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 102 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The electronic device 102 can realize a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP processes the data fed back by the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted to the photosensitive element of the camera through the lens, and the optical signal is converted into an electrical signal. The photosensitive element of the camera transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also optimize the algorithm of image noise, brightness and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be set in the camera 193.

The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other format image signals. In some embodiments, the electronic device 102 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The digital signal processor is used to process digital signals. In addition to digital image signals, it can also process other digital signals. For example, when the electronic device 102 selects at a frequency point, the digital signal processor is used to perform Fourier transform on the energy at the frequency point.

The video codec is used to compress or decompress digital video. The electronic device 102 may support one or more video codecs. In this way, the electronic device 102 can play or record videos in various encoding formats, such as: moving picture experts group (moving picture experts, MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information and can continue to self-learn. The NPU can realize applications such as intelligent recognition of the electronic device 102, such as image recognition, face recognition, voice recognition, and text understanding.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 102. The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save music, video and other files in an external memory card.

The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 102. For example, in this embodiment, the processor 110 may execute instructions stored in the internal memory 121, establish a Bluetooth connection with the peripheral device through the wireless communication module 160, and perform short-range data interaction with the peripheral device to implement the peripheral device Call, play music and other functions. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area can store the operating system, at least one function required application programs (such as sound playback function, image playback function, etc.). The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 102 and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and so on. In this embodiment, the wireless communication technology is adopted between the electronic device 102 and the peripheral device. For example, after the Bluetooth connection is established by the Bluetooth, the electronic device 102 can store the Bluetooth address of the peripheral device in the internal memory 121. In some embodiments, when the peripheral device is a device that includes two main bodies, such as a TWS headset, the left and right earbuds of the TWS headset have respective Bluetooth addresses, and the electronic device 102 can use the left and right earbuds of the TWS headset The associated Bluetooth address is stored in the internal memory 121.

The electronic device 102 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, and an application processor. For example, music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called "speaker", is used to convert audio electrical signals into sound signals. The electronic device 102 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also known as "handset", is used to convert audio electrical signals into sound signals. When the electronic device 102 answers a call or a voice message, it can answer the voice by holding the receiver 170B close to the ear.

The microphone 170C, also known as "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through the human mouth, and input the sound signal to the microphone 170C. The electronic device 102 may be provided with at least one microphone 170C. In other embodiments, the electronic device 102 may be provided with two microphones 170C. In addition to collecting sound signals, it may also implement a noise reduction function. In other embodiments, the electronic device 102 may also be provided with three, four, or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

In this embodiment, when the electronic device 102 establishes a Bluetooth connection with a peripheral device such as a TWS headset, the TWS headset can be used as an audio input/output device of the electronic device 102. Exemplarily, the audio module 170 may receive the audio electrical signal transmitted by the wireless communication module 160 to realize functions such as answering a phone call and playing music through a TWS headset. For example, during a user's phone call, the TWS headset can collect the user's voice signal, convert it into an audio electrical signal, and send it to the wireless communication module 160 of the electronic device 102. The wireless communication module 160 transmits the audio electrical signal to the audio module 170. The audio module 170 can convert the received audio electrical signal into a digital audio signal, encode it, and pass it to the mobile communication module 150. It is transmitted by the mobile communication module 150 to the call peer device to realize the call. For another example, when the user uses the media player of the electronic device 102 to play music, the application processor may transmit the audio electrical signal corresponding to the music played by the media player to the audio module 170. The audio electrical signal is transmitted to the wireless communication module 160 by the audio module 170. The wireless communication module 160 may send the audio electrical signal to the TWS headset, so that the TWS headset converts the audio electrical signal into a sound signal and plays it. In some embodiments, the audio electrical signal may be the audio data in this embodiment.

The headset interface 170D is used to connect wired headsets. The earphone interface 170D may be a USB interface 130 or a 3.5mm open mobile electronic device (open mobile terminal) platform (OMTP) standard interface, the American Telecommunications Industry Association (cellular telecommunications industry association of the United States, CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. The capacitive pressure sensor may be a parallel plate including at least two conductive materials. When force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 102 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 102 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 102 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch position but have different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the movement posture of the electronic device 102. In some embodiments, the angular velocity of the electronic device 102 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the angle at which the electronic device 102 shakes, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 102 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 102 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 102 may use the magnetic sensor 180D to detect the opening and closing of the flip holster. In some embodiments, when the electronic device 102 is a clamshell machine, the electronic device 102 can detect the opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the holster or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 102 in various directions (generally three axes). When the electronic device 102 is stationary, the magnitude and direction of gravity can be detected. It can also be used to recognize the posture of electronic devices, and can be used in horizontal and vertical screen switching, pedometer and other applications.

The distance sensor 180F is used to measure the distance. The electronic device 102 can measure the distance by infrared or laser. In some embodiments, when shooting scenes, the electronic device 102 may use the distance sensor 180F to measure distance to achieve fast focusing.

The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 102 emits infrared light outward through the light emitting diode. The electronic device 102 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 102. When insufficient reflected light is detected, the electronic device 102 may determine that there is no object near the electronic device 102. The electronic device 102 can use the proximity light sensor 180G to detect that the user holds the electronic device 102 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 102 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 102 is in a pocket to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 102 can use the collected fingerprint characteristics to unlock the fingerprint, access the application lock, take a picture of the fingerprint, answer the call with the fingerprint, and so on.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the electronic device 102 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 102 performs performance reduction of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 102 heats the battery 142 to avoid abnormal shutdown of the electronic device 102 due to low temperature. In some other embodiments, when the temperature is below another threshold, the electronic device 102 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

Touch sensor 180K, also known as "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 constitute a touch screen, also called a "touch screen". The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 102, which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive a blood pressure beating signal. In some embodiments, the bone conduction sensor 180M may also be provided in the earphone and combined into a bone conduction earphone. The audio module 170 may parse out the voice signal based on the vibration signal of the vibrating bone block of the voice part acquired by the bone conduction sensor 180M to realize the voice function. The application processor may analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M to implement the heart rate detection function.

The key 190 includes a power-on key, a volume key, and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 102 can receive key input and generate key signal input related to user settings and function control of the electronic device 102.

The motor 191 may generate a vibration prompt. The motor 191 can be used for vibration notification of incoming calls and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, audio playback, etc.) may correspond to different vibration feedback effects. For the touch operation of different areas of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the amount of power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 102. The electronic device 102 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. The same SIM card interface 195 can insert multiple cards at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 102 interacts with the network through the SIM card to realize functions such as call and data communication. In some embodiments, the electronic device 102 uses eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 102 and cannot be separated from the electronic device 102.

In the transmission mechanism defined by the ISO channel of BLE, CIG is defined. CIG is a concept of a group and can include multiple CISs. A master device can send audio data to multiple slave devices through multiple CISs in the CIG. Multiple CIS correspond to multiple slave devices. And these multiple CISs can share the same CIG sync point and CIG play point. In this embodiment, multiple slave devices can determine the CIG synchronization point and the CIG playback point based on the synchronized audio clock and the synchronized Bluetooth clock, so that the determined CIG synchronization point and CIG playback point can be used to achieve the playback level of audio data Synchronize.

In addition, in order to enable the above-mentioned audio data to be transmitted between devices using Bluetooth technology, devices that use Bluetooth communication (such as the aforementioned electronic device 102 and peripheral device 101) need to follow a certain Bluetooth transmission framework. FIG. 5 is a schematic diagram of the composition of a Bluetooth transmission framework provided by an embodiment of the present application. As shown in FIG. 5, the Bluetooth transmission framework may include an application layer, a host, a host controller interface (HCI), and a controller.

Among them, the application layer may include telephone applications, multimedia applications (such as music players, video players) and other applications.

The host may include various application profiles of Bluetooth and transmission protocols. For example, hands-free profile (HFP), advanced audio transmission profile (advanced audio distribution profile, A2DP), audio/video remote control profile (audio/video remote control profile, AVRCP), general access profile (generic access profile) , GAP), Generic Attribute Profile (GATT), Audio and Video Distribution Transmission Protocol (audio/video distribution protocol, AVDTP), Audio and Video Control Transmission Protocol (audio/video control transmission protocol, AVCTP), Service Discovery Protocol (service discovery protocol, SDP), serial port emulation protocol (radio frequency communications protocols, RFCOMM), logical link control and adaptation protocol (logical link control and adaptation protocol, L2CAP). Interoperable devices need to use the same application specification to communicate. Different applications require different application specifications.

HCI is located between the host and the controller, and can provide a unified interface for the upper layer protocol to enter the link layer in the controller and a unified way to enter the baseband in the controller.

The controller may include a link layer (LL), a baseband, and a radio frequency (Bluetooth radio frequency) unit. Among them, the link layer is responsible for managing the communication between the devices, to achieve link establishment, verification, link configuration and other operations. The baseband mainly performs mutual conversion between radio frequency signals and digital or voice signals to implement baseband protocols and other low-level connection procedures. The radio frequency unit is used to transmit and receive Bluetooth signals. In some embodiments, the host may be implemented in the AP of the device. The controller can be implemented in the Bluetooth chip of the device. In other embodiments, the host and the controller may be implemented in the same processor or controller of the device, in which case HCI is optional.

In this embodiment, when the electronic device establishes a wireless connection with multiple Bluetooth devices (the multiple Bluetooth devices may be multiple peripheral devices, and the multiple Bluetooth devices may also be multiple bodies of one peripheral device), The electronic device may send a data packet containing audio data to each of the plurality of Bluetooth devices based on the wireless connection established with each Bluetooth device. The data packet may also include an audio time stamp and an initial Bluetooth time stamp of the data packet. Multiple Bluetooth devices can determine the CIG playback point of the audio data based on the audio timestamp and the initial Bluetooth timestamp in the data packet, and play the audio data according to the determined CIG playback point to achieve playback-level synchronization of the audio data.

For ease of understanding, the following embodiment combines the Bluetooth transmission framework shown in FIG. 5 and uses the electronic device as a mobile phone and the Bluetooth device as the main body of the peripheral device. For example, the peripheral device is a TWS headset. The TWS headset includes two headset bodies, which are respectively Taking earplugs and right earplugs as examples, the method for synchronizing audio data provided in this embodiment will be described in detail. The left earplug may be the first earplug in this application, the right earplug may be the second earplug in this application; or, the right earplug may be the first earplug in this application, and the left earplug may be the second earplug in this application. Earplugs.

The left and right earbuds of the TWS headset are paired with the mobile phone via Bluetooth.

Exemplarily, when the user wishes to use the TWS earphone, the lid of the earphone box of the TWS earphone can be opened. The left earplug and right earplug of the TWS headset turn on automatically, or the function key is pressed and then turned on. If the left earbud and the right earbud have not completed Bluetooth pairing before, the left earbud and the right earbud can automatically perform Bluetooth pairing, or the left earbud and right earbud can perform Bluetooth pairing after the pairing function key is pressed. If the left earbud and the right earbud have previously completed Bluetooth pairing, the Bluetooth pairing process can be omitted. After Bluetooth pairing is completed, a Bluetooth connection can be established between the left and right earbuds of the TWS headset. Then, any one of the left earplug and the right earplug of the TWS earphone (such as the right earplug) can send a paired broadcast to the outside. If the mobile phone has turned on the Bluetooth function, the mobile phone can receive the pairing broadcast and prompt the user that the relevant Bluetooth device (such as a TWS headset) has been scanned. When the TWS headset is selected as the connected device on the mobile phone, the mobile phone can be paired with the right earbud of the TWS headset via Bluetooth. Of course, if the Bluetooth pairing of the mobile phone and the right earbud of the TWS headset has been completed before, the Bluetooth pairing process can be omitted. In other words, after receiving the pairing broadcast, the mobile phone can automatically perform Bluetooth pairing with the right earbud.

After the right earbud is paired with the mobile phone via Bluetooth, the right earbud can send the Bluetooth address of the mobile phone to the left earbud through the Bluetooth connection between the left earbud and the left earbud, and notify the left earbud to send a pairing broadcast to the outside. In this way, the mobile phone can receive the pairing broadcast sent by the left earplug and perform Bluetooth pairing with the left earplug.

In some embodiments, the right earplug of the TWS headset may also send the Bluetooth address of the left earplug to the mobile phone to indicate to the mobile phone that the left earplug and the right earplug are two main bodies of the same peripheral device. When there is audio data that needs to be transmitted to the left and right earplugs of the TWS headset, the mobile phone can transmit audio data to the right and left earplugs through the two CISs in the same CIG, so that the right earplug and the left earplug can be realized. Play level synchronization of audio data.

It should be noted that the above process of pairing the left earplug and the right earplug of the TWS headset with the mobile phone via Bluetooth is only an example. In some embodiments, the mobile phone may also perform Bluetooth pairing with the left earbud of the TWS earphone, and then the left earbud sends the Bluetooth address of the mobile phone to the right earbud and notifies the right earbud to send a pairing broadcast to the outside so that the right earbud and the mobile phone can perform Bluetooth pair. In some other embodiments, the left earplug and the right earplug of the TWS earphone can separately send pairing broadcasts after being turned on, so that the left earplug and the right earplug can perform Bluetooth pairing with the mobile phone respectively. In addition, in the embodiment of the present application, the trigger condition for the paired broadcast of the left earplug or the right earplug of the TWS headset may be that the cover of the headphone box of the TWS headset is opened, or the pairing of the left earplug and the right earplug of the TWS headset Bluetooth Done, or the left earplug or right earplug of the TWS earphone is taken out of the earphone box, or the pairing function key is pressed, or other triggering conditions. The pairing function key can be set on the earphone box of the TWS earphone. For example, the earphone box of the TWS earphone is equipped with a pairing function key. When the pairing function key is pressed, the left or right earplug of the TWS earphone can be Send paired broadcasts to the outside world. The pairing function keys can also be set on the left and right earbuds of the TWS headset. For example, the pairing function keys are configured on the left and/or right earbuds of the TWS headset. When the pairing function key is pressed, the corresponding earbud sends a pairing broadcast to the outside .

After the mobile phone is paired with the left earplug and the right earplug of the TWS headset via Bluetooth, the mobile phone can establish an ACL link with the left earplug and the right earplug of the TWS headset, respectively.

For example, a mobile phone can establish an ACL link 1 with the left earbud, and a mobile phone can establish an ACL link 2 with the right earbud. Take the case where the mobile phone and the left earbud establish ACL link 1 as an example. The mobile phone can send a request to establish an ACL link to the left earplug. The left earbud answers after receiving the request to establish the ACL link. After the mobile phone receives the response from the left earplug, ACL link 1 is established. Wherein, ACL link 1 may be the first ACL link in this application, and ACL link 2 may be the second ACL link in this application.

The mobile phone negotiates parameters with the TWS headset and configures CIG parameters based on the negotiation results.

Exemplarily, the parameters of the CIG (such as CIG parameters) used to transmit audio data of different audio services may be different. In order to adapt to the transmission of audio data of different audio services, when the mobile phone determines that there is audio data to be transmitted (for example, the mobile phone determines that the audio service is turned on), the mobile phone can be connected with the left and right earplugs of the TWS headset or the left earplug of the TWS headset The earplug and one of the right earplugs perform parameter negotiation. The mobile phone can determine the CIG parameters that can be adapted to the audio service according to the results of the parameter negotiation. This CIG parameter can be used to establish the CIS in the CIG for the transmission of related audio data. For example, the above parameter negotiation may include one or more of the following: quality of service (QoS) parameter negotiation, codec parameter negotiation, and CIS parameter negotiation. Correspondingly, the CIG parameters determined by the mobile phone may include one or more of the following: QoS parameters, codec parameters, and CIS parameters.

Among them, the QoS parameters may include parameters such as delay, packet loss rate, throughput, etc. that represent transmission quality. The codec parameters may include encoding methods, compression ratios, and other parameters that affect audio quality. The CIS parameters may include an anchor point, ISO interval, CIS identification (ID), and so on. The CIG may include multiple CIG events (CIG_event). The anchor point is the start time point of the corresponding CIG event. The ISO interval is the time between two consecutive anchor points. Each CIG event belongs to an ISO interval in time.

In some embodiments, the above-mentioned parameter negotiation process may be implemented based on the ACL link established between the mobile phone and the left and right earplugs of the TWS headset. For example, taking a mobile phone to negotiate parameter with one of the left earplug and right earplug of the TWS earphone (such as the left earplug) to determine the CIG parameter as an example, the specific process of parameter negotiation may be: the mobile phone passes the ACL link 1 to the left earplug Send parameter negotiation messages. The parameter negotiation message may carry CIG parameters corresponding to the currently enabled audio service. In some embodiments, the CIG parameters corresponding to the currently enabled audio service may be predefined. The left earplug receives the parameter negotiation message sent by the mobile phone through ACL link 1. If the left earplug agrees with the CIG parameters carried in the parameter negotiation message, it can return a confirmation message to the mobile phone; if the left earplug does not agree with the parameters carried in the parameter negotiation message or agrees with some of the parameters carried in the parameter negotiation message, you can send The mobile phone returns a continue negotiation message to continue parameter negotiation with the mobile phone until the left earbud returns a confirmation message to the mobile phone. The mobile phone receives the confirmation message returned by the left earplug through ACL link 1. The mobile phone obtains the parameter negotiation result with the left earplug according to the confirmation message. In this way, the mobile phone can determine the CIG parameters that can be adapted to the audio data transmission of the audio service according to the parameter negotiation results.

After the mobile phone determines the CIG parameters that can be adapted to the audio data transmission of the audio service, the CIG parameters can be configured according to the determined CIG parameters. For example, in combination with the audio transmission framework shown in FIG. 5 above, the CIG parameter configuration process may be: the host of the mobile phone sets the CIG parameter. For example, the host of the mobile phone can send the HCI command "Low Power CIG Parameter Settings (LE Set CIG parameters)" to the LL of the mobile phone through the HCI. After receiving the HCI command, the LL of the mobile phone may return a response message to the host of the mobile phone, such as "command complete". The host of the mobile phone initiated the creation of CIS. For example, the host of the mobile phone can send the HCI command "LE createCIS" to the LL of the mobile phone through the HCI. After receiving the HCI command, the LL of the mobile phone can return a response message to the host of the mobile phone, such as "HCI command status (HCI command status)". At this point, the configuration of CIG parameters is complete.

The mobile phone establishes CIS with the left and right earplugs of the TWS earphone through the ACL link between the left and right earplugs.

After the CIG parameter configuration is completed, the mobile phone can establish the CIS with the left and right earplugs of the TWS headset through the ACL link with the left and right earplugs according to the configured CIG parameters. For example, a mobile phone can establish CIS 1 with the left earbud and CIS 2 with the right earbud. The CIS1 and the CIS2 are included in the same CIG. Among them, CIS 1 may be the first CIS in this application, and CIS 2 may be the second CIS in this application.

Among them, the process of establishing the CIS 1 between the mobile phone and the left earplug and the CIS 2 between the mobile phone and the right earplug can be seen in FIG. 6. Combining the audio transmission framework shown in Figure 5, take the mobile phone and the left earbud to establish CIS as an example. The process of establishing the CIS1 between the mobile phone and the left earbud may include:

Step 601: The LL of the mobile phone can send a CIS establishment request message to the LL of the left earplug through the ACL link 1, such as an air interface request message LL_CIS_REQ. The CIS establishment request message can be used to request the LL of the left earplug to create a CIS.

Step 602: The LL of the left earplug can send a CIS establishment response message to the LL of the mobile phone through the ACL link 1, such as an air interface response message LL_CIS_RSP. The CIS establishment response message may be used to indicate to the mobile phone that the left earplug agrees to the establishment of CIS.

In some embodiments, after receiving the CIS establishment request message sent by the LL of the mobile phone, the LL of the left earplug may initiate a CIS creation request to the host of the left earplug. For example, the LL of the left earplug can send the HCI command "low power consumption CIS request (LE CIS request)" to the host of the left earplug through HCI. The host of the left earplug can accept the CIS creation request of the left earplug LL. For example, the host of the left earplug can send the HCI command "low power CIS acceptance (LE accept CIS)" to the LL of the left earplug through HCI. After receiving the HCI command, the LL of the left earplug may send the CIS establishment response message to the LL of the mobile phone. After receiving the HCI command, the LL of the left earplug may return a response message to the host of the left earplug, such as “command status”.

Step 603: The LL of the mobile phone can send a CIS establishment message to the LL of the left earplug through the ACL link 1, such as an air interface notification message LL_CIS_IND. The CIS establishment message can be used to notify the LL of the left earplug that the CIS has completed establishment.

In addition, the LL of the mobile phone can also send the HCI instruction "LE CIS establishment (LE) CIS establishment" to the host of the mobile phone through the HCI to notify the host of the mobile phone that the CIS is completed. The LL of the left earplug can also send the HCI command "LE CIS establish" to the host of the left earplug through HCI to notify the host of the left earplug to complete the establishment. In this way, the establishment of CIS1 between the mobile phone and the left earbud is completed. Similarly, as shown in FIG. 6, the mobile phone can perform step 604, step 605, and step 606 with the right earplug (the description of step 604-step 606 is the same as the description type of the above step 601-step 603, which will not be described in detail here) , In order to complete the establishment of CIS2 between the mobile phone and the right earbud. At this point, the two CIS between the mobile phone and the left and right earplugs of the TWS headset are established.

After the establishment of the CIS between the mobile phone and the left and right earplugs of the TWS headset (that is, the above CIS 1 and CIS 2, the CIS 1 and CIS 2 belong to the same CIG), the mobile phone can send the TWS headset through CIS 1 and CIS 2 respectively The left earbud and right earbud transmit audio data. The left earplug and right earplug of the TWS headset can receive audio data from the mobile phone through CIS1 and CIS2, respectively. The left earplug and the right earplug of the TWS headset can also determine the CIG synchronization point and the CIG playback point based on the synchronized audio clock and the synchronized Bluetooth clock, and realize the playback level synchronization of the audio data according to the determined CIG synchronization point and the CIG playback point.

In the following, in conjunction with the audio transmission framework shown in FIG. 5, the process of sending audio data to the left earplug and the right earplug of the TWS earphone and the process of synchronizing the playback level of the audio data by the left earplug and the right earplug of the TWS earphone will be described in detail. As shown in FIG. 6, the method may further include:

Step 607: The mobile phone obtains a data packet, and the data packet includes audio data to be transmitted.

In some embodiments, as shown in FIG. 6, step 607 may specifically be: the host of the mobile phone may encode the audio data to be transmitted. The host of the mobile phone can transmit the encoded audio data to the LL of the mobile phone through HCI. After the LL of the mobile phone receives the data, the controller where the LL of the mobile phone is located may add one or more fields in the frame header, frame tail, preamble and other fields to the data. In this way, the phone gets the data packet.

Step 608: The mobile phone sends the above data packets to the left earplug and the right earplug through the CIS between the left earplug and the right earplug of the TWS earphone, respectively.

For example, after the mobile phone obtains the data packet, the LL of the mobile phone can send the data packet to the LL of the left earbud of the TWS headset through CIS 1 and the LL of the right earbud of the TWS headset through CIS 2.

Step 609: The left earplug of the TWS earphone determines the CIG play point of the audio data in the data packet.

Step 610: The right earplug of the TWS earphone determines the CIG play point of the audio data in the data packet.

In some embodiments, according to the transmission mechanism defined by the ISO channel of BLE, the CIG playback point of audio data may be determined according to the CIG synchronization point and playback delay (presentation delay), for example, as shown in formula (1). Among them, Render_m_n is used to represent the CIG play point of the audio data. SYNC_m_n represents the CIG synchronization point. Presentation_delay indicates the playback delay. m indicates that the data packet containing the audio data is the m-th data packet on the Bluetooth clock. n indicates that the packet containing the audio data is the nth packet on the audio clock. m and n may be the same or different.

Render_m_n=SYNC_m_n+Presentation_delay formula (1)

In addition, in order to achieve playback-level synchronization of audio data in the left and right earplugs of the TWS earphone, the synchronization of the audio clock and the synchronization of the Bluetooth clock need to be considered when determining the CIG synchronization point.

In this embodiment, the audio clock may be represented by an audio time stamp (ATS), and the Bluetooth clock may be represented by a Bluetooth time stamp (BTS). For example, BTS_m is the Bluetooth timestamp of the data packet containing the audio data. ATS _n is the audio time stamp of the data packet containing the audio data. For ATS _n and BTS _m , in this embodiment, a common clock can be virtualized as a reference time for reference. Then, ATS _n can be determined according to the following formula:

Wherein, Timeoffset _ATS is the starting offset of the audio clock relative to the common clock. sample_num(i) is the number of sampling points of the data packet. f _s is the sampling frequency.

BTS _m can be determined according to the following formula:

Among them, Timeoffset _BTS is the starting offset of the Bluetooth clock relative to the common clock. Iso_interval(i) is the time between two consecutive anchor points. The CIG to which the CIS established between the mobile phone and the left earplug and the right earplug of the TWS headset can include multiple CIG events (CIG_event), and the start time point of each CIG event is an anchor point. The time between two consecutive anchor points can also be understood as the time between the starting time points of two consecutive CIG events.

If it is assumed that the start points of the Bluetooth clock and the audio clock are aligned, that is to say, Timeoffset _BTS =0 and Timeoffset _ATS =0 can be assumed. With reference to the schematic diagram of the transmission principle of audio data shown in FIG. 7, it can be known that the CIG synchronization point, that is, SYNC_m_n can be determined according to the audio timestamp and transmission delay of a data packet containing audio data with the Bluetooth clock as a reference, for example, such as Formula (2) shows. Among them, ATS_m_n is the audio timestamp of the data packet containing audio data under the premise of taking the Bluetooth clock as a reference. transport_delay is the transmission delay.

SYNC_m_n=ATS_m_n+transport_delay formula (2)

Among them, considering that the starting point of data transmission may not be BTS=0, that is to say, the data is allowed to be transmitted from Init_BTS, that is, after the CIS is established on the left and right earbuds of the mobile phone and the TWS headset, the first data packet is from nit_BTS starts transmission, therefore, it can be agreed that ATS_m_n can be determined according to the following formula (3).

ATS_m_n=Init_BTS+ATS_n formula (3)

Among them, Init_BTS (In the embodiment, Init_BTS may be called the initial Bluetooth timestamp) is the Bluetooth timestamp of the first data packet (ATS=0 of the first data packet on the audio clock), ATS_n is on the audio clock The audio timestamp of the nth packet.

According to the above formula (1), formula (2) and formula (3) can be obtained: the CIG playback point of the audio data, that is, Render_m_n can be determined according to the following formula (4). According to formula (4), it is necessary to consider the synchronization of the audio clock and the synchronization of the Bluetooth clock when determining the above-mentioned CIG play point.

Render_m_n=Init_BTS+transport_delay+ATS_n+Presentation_delay

Formula (4)

It can be seen from the above formula (4) that the left and right earplugs of the TWS earphones need to achieve playback-level synchronization of audio data, that is, to determine Render_m_n, the following parameters need to be obtained separately:

Parameter (1): transmission delay transport_delay.

Parameter (2): Presentation delay delay_delay.

Parameter (3): Bluetooth timestamp Init_BTS of the first data packet.

And parameter (4): audio timestamp ATS_n of the data packet containing the audio data.

For the above parameter (1) and the above parameter (2):

In some embodiments, the transmission delay transport_delay and the playback delay Presentation_delay can be sent from the mobile phone to the left and right earplugs of the TWS headset when the mobile phone establishes a CIS with the left and right earplugs of the TWS headset. For example, referring to FIG. 6, the mobile phone may send the transmission delay transport_delay and the playback delay Presentation_delay to the left earplug in the process of establishing the CIS 1 with the left earplug, such as the air interface request message LL_CIS_REQ carried in step 601 to the left earplug. The mobile phone may send the transmission delay transport_delay and the playback delay Presentation_delay to the right earplug in the process of establishing CIS 2 with the right earplug, for example, the air interface request message LL_CIS_REQ carried in step 604 is sent to the right earplug.

For the above parameter (3) and the above parameter (4):

In some embodiments, Init_BTS and ATS_n may be carried by the mobile phone in data packets transmitted to the left and right earplugs of the TWS headset. For example, in step 607, after encoding the audio data to be transmitted, the host of the mobile phone may add a packet header to the audio data after the encoding process. For example, the encoded audio data after adding the packet header may be called an encoded data packet, or called a load (playload). In this embodiment, the packet header may include ATS_n and Init_BTS. Taking the data packet as the nth data packet on the audio clock as an example, the audio time stamp of the data packet can be represented by ATS_n. Then, the host of the mobile phone can transmit the encoded data packet to the LL of the mobile phone. After the LL of the mobile phone receives the coded data packet, the controller where the LL is located may add one or more fields in the above frame header, frame end, preamble, etc. to the coded data packet to obtain the data packet. For example, the structure of the data packet may be as shown in FIG. 8. The data packet includes, in sequence, a preamble, a frame header, a packet header, audio data, and a frame tail.

In some embodiments, the above Init_BTS can be expressed as an ISO event counter (ISO event counter). In some embodiments, in combination with the above calculation formula of ATS _n , assuming that the start points of the Bluetooth clock and the audio clock are aligned, that is, assuming Timeoffset _BTS =0 and Timeoffset _ATS =0, the above ATS_n may be modulated by pulse code (pulse code modulation, PCM) sample (sample) to count, or, can also be counted in units of Bluetooth clock. Taking ATS_n taking PCM sample count as an example, the audio timestamp ATS represents the number of samples, and the value of ATS of each data packet is the sum of the number of sampling points of all previous data packets. If the count starts at 0, that is, if the ATS of the first data packet (ie ATS_1) is 0, the ATS of the second data packet (ie ATS_2) is the number of samples of the first data, and the ATS of the third data packet ( That is, ATS_3) is the sum of the sampling points of the first packet and the second packet. By analogy, the ATS of the nth data packet (that is, ATS_n) is the sum of the sampling points of the first n-1 data packets. The mobile phone can obtain the ATS (such as ATS_n) of the currently transmitted data according to the PCM sample count, and carry the left and right earplugs sent to the TWS headset in the data packet. According to the obtained ATS (such as ATS_n) and the sampling frequency f _s (such as 16ksps, 32ksps, 44.1ksps, 48ksps, 96ksps), the left and right earplugs of the TWS earphone can calculate the audio time of each data packet.

In some other embodiments, the Init_BTS may not be carried by the mobile phone in a data packet to notify the left and right earplugs of the TWS headset. Instead, the left and right earplugs of the TWS headset are obtained based on the ISO event count (ISO event count) maintained by the TWS headset. In some embodiments, the mobile phone may be responsible for the maintenance of the ISO event, and may notify the left earplug and the right earplug of the TWS headset through an air interface message to ensure the synchronization of the ISO events of the left earplug and the right earplug of the TWS headset. Exemplarily, when the ISO establishes a CIS between the mobile phone and the left earplug and the right earplug of the TWS headset, the mobile phone sends the left earplug and the right earplug of the TWS headset through an air interface message, respectively. For example, referring to FIG. 6, the mobile phone may send the ISO event to the left earplug during the establishment of the CIS 1 with the left earplug, such as the air interface request message LL_CIS_IND carried in step 603 to the left earplug. The mobile phone may send the ISO event to the right earplug during the establishment of the CIS 2 with the right earplug, for example, the air interface request message LL_CIS_IND carried in step 606 is sent to the right earplug. In addition, the mobile phone can also send the indication value to the left and right earplugs of the TWS headset. After receiving the ISO event and the indication value, the left and right earplugs of the TWS headset can also maintain the ISO event according to the indication value. To continue to ensure the synchronization of the ISO and event of the left earplug and right earplug of the TWS headset. The left and right earbuds of the TWS headset can obtain the Bluetooth timestamp of the first data packet on the Bluetooth clock according to the maintained ISO event, that is, Init_BTS.

In addition, on the premise that the audio clock and the Bluetooth clock are based on a unified common clock, according to the above formula (4), it can be seen that the playback time (ie, CIG playback point) of the audio data contained in any data packet is unique and ATS_n, transport_delay and Presentation_delay are related. In other words, regardless of the Bluetooth timestamp of which Bluetooth clock the data packet was transmitted, the time when the receiving device (such as the left and right earbuds of the TWS headset) received the data packet and the time when the data packet was played is the Bluetooth time at the time of transmission The stamp is irrelevant. ATS_n is decoupled from BTS_m when actually transmitting audio data, that is, decoupling of audio timestamp and Bluetooth timestamp is achieved.

In other embodiments, due to jitter during actual transmission, the CIG play point of the audio data in the data packet determined by the left and right earplugs of the TWS earphone may be earlier than the current actual time. Therefore, the audio data The continuity of playback requires consideration. The time offset between the Bluetooth timestamp of the audio data packet and the audio timestamp, as represented by timeoffset_m_n (timeoffset_m_n=BTS _m -ATS _n ), must meet the following conditions:

Condition 1: timeoffset_m_n≤transport_delay

Condition 2: timeoffset_m_n≤Presentation_delay

Condition 3: timeoffset_m_n≤transport_delay+Presentation_delay

In the case of ensuring that timeoffset_m_n satisfies the above three conditions, the continuity of audio data at the playback level can be guaranteed.

In conjunction with the above description, the left earplug of the TWS earphone determines the CIG playback point of the audio data in the data packet as an example. As shown in FIG. 6, step 609 may specifically include:

After receiving the data packet, the LL of the left earplug can send the received data packet to the host of the left earplug through the HCI. The host of the left earplug can parse the data packet to obtain Init_BTS and ATS_n carried in the packet header. In this way, the host of the left earplug can calculate the CIG synchronization point of the data packet based on the Init_BTS and ATS_n carried in the data packet, and the transport_delay carried in the air interface request message LL_CIS_REQ. Then, according to the determined CIG synchronization point and the Presentation_delay carried in the air interface request message LL_CIS_REQ, the CIG playback point of the audio data can be determined.

Similarly, the right earplug can also obtain the CIG playback point of the audio data after the above processing.

Step 611: The left earplug of the TWS earphone plays the audio data in the data packet according to the determined CIG play point.

Step 612: The right earplug of the TWS earphone plays the audio data in the data packet according to the determined CIG play point.

After the left earplug and the right earplug of the TWS earphone respectively determine the CIG play point, the audio data of the received data packets can be played at the determined CIG play point. For example, after the host of the left earplug and the right earplug of the TWS earphone determines the CIG playback point, the CIG playback point and audio data can be transmitted to the DSP of the left earplug and the right earplug, and the DSP decodes the audio data to the CIG Play at the playback point.

Using this technical solution, the left earplug and right earplug of the TWS earphone can determine the CIG synchronization point and CIG playback point of audio data according to the synchronized Bluetooth clock and synchronized audio clock, and use the determined CIG playback point to realize the playback of audio data Level synchronization. In addition, in this embodiment, the Bluetooth clock and the audio clock are decoupled and synchronized independently, and no hardware clock synchronization of the Bluetooth clock and the audio clock is required. For the Bluetooth clock, it is only necessary to carry the BTS of the first data packet in each data packet, that is, Init_BTS, and there is no need to obtain the BTS of each data packet to reduce the load. Corresponding to the audio clock, each data packet is independently labeled. Therefore, while achieving playback-level synchronization of audio data, it can also support independent playback of each data packet. In this way, the technical solution of this embodiment can be applied to scenarios where single earplugs are used alone, both earplugs are used simultaneously, and single ear and binaural switching is performed.

In some other embodiments of the present application, a peripheral device is also provided. For example, the peripheral device is a Bluetooth headset, and the Bluetooth headset may include at least one earplug. As shown in FIG. 9, the earbuds of the Bluetooth headset may include: one or more processors 901; a memory 902; a communication interface 903; a receiver 904; a microphone 905; and one or more computer programs 906. Or multiple communication buses 907 are connected. The one or more computer programs 906 are stored in the above-mentioned memory 902, and are configured to be executed by the one or more processors 901. The one or more computer programs 906 include instructions, and the above instructions may be used to execute Each step performed by the earplug (such as the left earplug and the right earplug) in the corresponding embodiment of FIG. 6. Of course, the earbuds of the Bluetooth headset shown in FIG. 9 may also include other devices such as sensors and the like, which are not limited in the embodiments of the present application. When the earplug shown in FIG. 9 further includes other devices such as sensors, it may be the earplug shown in FIG. 2.

Other embodiments of the present application also provide an electronic device. As shown in FIG. 10, the electronic device may include: a touch screen 1001, wherein the touch screen 1001 may include a touch-sensitive surface 1006 and a display screen 1007; one or more The processor 1002; the memory 1003; and one or more computer programs 1004. The above devices may be connected through one or more communication buses 1005. The one or more computer programs 1004 are stored in the above-mentioned memory 1003 and are configured to be executed by the one or more processors 1002. The one or more computer programs 1004 include instructions, which can be used to execute Each step performed by the electronic device in the corresponding embodiment of FIG. 6. Of course, the electronic device shown in FIG. 10 may also include other devices such as a sensor module, an audio module, and a SIM card interface, which is not limited in the embodiments of the present application. When the electronic device shown in FIG. 10 further includes other devices such as a sensor module, an audio module, and a SIM card interface, it may be the electronic device shown in FIG. 4.

This embodiment also provides a computer-readable storage medium including instructions. When the above instructions run on an electronic device, the electronic device causes the electronic device to perform the relevant method steps in FIG. 6 to implement the above The method in the examples.

This embodiment also provides a computer-readable storage medium including instructions. When the above instructions run on the earphone of the Bluetooth headset, the earphone of the Bluetooth headset executes the relevant method steps in FIG. 6 To implement the method in the above embodiment.

This embodiment also provides a computer program product containing instructions. When the computer program product runs on an electronic device, the electronic device is caused to execute relevant method steps as shown in FIG. 6 to implement the method in the above embodiment.

This embodiment also provides a computer program product containing instructions. When the computer program product runs on the earphone of the Bluetooth headset, the earphone of the Bluetooth headset executes the relevant method steps shown in FIG. 6 to implement the above embodiment The method.

This embodiment also provides a control device. The control device includes a processor and a memory. The memory is used to store computer program code. The computer program code includes computer instructions. When the processor executes the computer instructions At this time, the control device executes the relevant method steps as shown in FIG. 6 to implement the method in the above embodiment. The control device may be an integrated circuit IC or a system-on-chip SOC. The integrated circuit may be a general-purpose integrated circuit, a field programmable gate array FPGA, or an application-specific integrated circuit ASIC.

This embodiment also provides a device for synchronizing audio data. The device has a function of realizing the earplug behavior of an electronic device or a Bluetooth headset in the above method. The function can be realized by hardware, or can also be realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

Among them, the electronic device, the earbud of the Bluetooth headset, the computer storage medium, the computer program product, the control device, or the audio data synchronization device provided in this embodiment are all used to execute the corresponding method provided above, therefore, it can achieve For the beneficial effects, please refer to the beneficial effects in the corresponding methods provided above, which will not be repeated here.

Through the description of the above embodiments, those skilled in the art can clearly understand that, for convenience and conciseness of description, only the above-mentioned division of each functional module is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated according to needs Completed by different functional modules, that is, dividing the internal structure of the device into different functional modules to complete all or part of the functions described above. For the specific working processes of the system, device and unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this embodiment, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple units or components may be The combination can either be integrated into another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical, or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each of the embodiments of this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The above integrated unit can be implemented in the form of hardware or software function unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment essentially or part of the contribution to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium In it, several instructions are included to enable a computer device (which may be a personal computer, server, or network device, etc.) or processor to perform all or part of the steps of the methods described in the various embodiments. The foregoing storage media include: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk and other media that can store program codes.

The above is only the specific implementation of this application, but the scope of protection of this application is not limited to this, any change or replacement within the technical scope disclosed in this application should be covered within the scope of protection of this application . Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A method for synchronizing audio data is characterized by being applied to an electronic device, which establishes a wireless link with a first earplug and a second earplug, respectively. The method includes:

The electronic device sends a data packet to the first earplug through a first connection-based isochronous stream (CIG) based connection-based isochronous audio stream (CIS);

The electronic device sends the data packet to the second earplug through the second CIS of the CIG;

Wherein, the data packet includes: audio data, an audio timestamp of the data packet, and an initial Bluetooth timestamp, the initial Bluetooth timestamp establishes the first CIS for the electronic device and the first earbud, Bluetooth timestamp of the first data packet transmitted after establishing the second CIS with the second earplug; the audio timestamp and the initial Bluetooth timestamp are used for the first earplug and the second earplug A CIG play point is determined, and the CIG play point is a time point at which the first earplug and the second earplug play the audio data in the data packet.
The method according to claim 1, wherein before the electronic device sends the data packet to the first earplug and the data packet to the second earplug, the method further comprises:

The electronic device establishes the first CIS of the CIG with the first earplug, and establishes the second CIS of the CIG with the second earplug.
The method according to claim 2, wherein the electronic device establishes the first CIS of the CIG with the first earplug, and establishes the second CIS of the CIG with the second earplug , Including:

The electronic device sends a CIS establishment request message to the first earplug, receives a CIS establishment response message sent by the first earplug, and sends a CIS establishment message to the first earplug;

The electronic device sends a CIS establishment request message to the second earplug, receives a CIS establishment response message sent by the second earplug, and sends a CIS establishment message to the second earplug;

Wherein, the CIS establishment request message includes a transmission delay and a playback delay, and the transmission delay and the playback delay are used for the first earplug and the second earplug to determine the CIG play point.
The method according to claim 2 or 3, wherein the first CIS that establishes the CIG in the electronic device and the first earplug and the CIG that establishes the CIG with the second earplug Before the second CIS, the method further includes:

The electronic device is paired with the first earplug and the second earplug, respectively;

The electronic device establishes a first asynchronous connection-oriented (ACL) link with the first earplug;

The electronic device establishes a second ACL link with the second earplug;

The electronic device establishing the first CIS of the CIG with the first earplug and the second CIS establishing the CIG with the second earplug specifically include:

The electronic device establishes the first CIS of the CIG with the first earplug through the first ACL link, and establishes the CIG of the CIG with the second earplug through the second ACL link Second CIS.
The method according to any one of claims 1 to 4, wherein the time offset between the audio time stamp of the data packet and the Bluetooth time stamp of the data packet satisfies at least one of the following conditions: Less than or equal to transmission delay, less than or equal to playback delay, less than or equal to the sum of transmission delay and playback delay.
A method for synchronizing audio data is characterized by being applied to a first earplug and a second earplug that establish a wireless connection with an electronic device. The method includes:

The first earplug receives a data packet from the electronic device through a first connection-based isochronous stream (CIG)-based isochronous audio stream (CIS); the second earplug passes through the CIG The second CIS receives the data packet from the electronic device; the data packet includes: audio data, an audio time stamp of the data packet, and an initial Bluetooth time stamp, the initial Bluetooth time stamp is the electronic The device establishes the first CIS with the first earplug and the Bluetooth timestamp of the first data packet transmitted after establishing the second CIS with the second earplug;

The first earplug and the second earplug determine the CIG play point according to the audio timestamp, the initial Bluetooth timestamp, the transmission delay, and the playback delay, respectively;

The first earplug and the second earplug respectively play the audio data in the data packet at the CIG play point.
The method of claim 6, wherein before the first earplug receives the data packet from the electronic device; before the second earplug receives the data packet from the electronic device, the The method also includes:

The first earplug and the electronic device establish the first CIS of the CIG, and the second earplug and the electronic device establish the second CIS of the CIG.
The method according to claim 6, wherein the first earplug and the electronic device establish the first CIS of the CIG, and the second earplug and the electronic device establish the location of the CIG The second CIS, including:

The first earplug receives a CIS establishment request message from the electronic device, sends a CIS establishment response message to the electronic device, and receives a CIS establishment message from the electronic device;

The second earplug receives a CIS establishment request message from the electronic device, sends a CIS establishment response message to the electronic device, and receives a CIS establishment message from the electronic device;

Wherein, the CIS establishment request message includes the transmission delay and the playback delay.
The method according to claim 7 or 8, wherein the first CIS of the CIG is established between the first earplug and the electronic device, and the second earplug and the electronic device establish the Before the second CIS of CIG, the method further includes:

The first earplug and the second earplug are paired with the electronic device respectively;

The first earplug establishes a first asynchronous connection-oriented (ACL) link with the electronic device;

The second earplug establishes a second ACL link with the electronic device;

The first earplug and the electronic device establish the first CIS of the CIG, and the second earplug and the electronic device establish the second CIS of the CIG, specifically including:

The first earplug establishes the first CIS of the CIG with the electronic device through the first ACL link, and the second earplug establishes the first CIS with the electronic device through the second ACL link The second CIS of CIG.
The method according to any one of claims 6-9, wherein the time offset between the audio time stamp of the data packet and the Bluetooth time stamp of the data packet satisfies at least one of the following conditions: Less than or equal to transmission delay, less than or equal to playback delay, less than or equal to the sum of transmission delay and playback delay.
The method according to any one of claims 6-10, wherein the first earplug and the second earplug are based on the audio timestamp, the initial Bluetooth timestamp, transmission delay, and playback, respectively The delay determines the CIG play point, which includes:

The first earbud determines the CIG synchronization point according to the audio timestamp, the initial Bluetooth timestamp and the transmission delay, and the first earbud determines the CIG according to the determined CIG synchronization point and the playback delay Play point

The second earbud determines the CIG synchronization point according to the audio timestamp, the initial Bluetooth timestamp and the transmission delay, and the first earbud determines the CIG according to the determined CIG synchronization point and the playback delay Play point.
The method according to any one of claims 6-11, wherein the determining the CIG play point according to the audio time stamp, the initial Bluetooth time stamp, the transmission delay and the playback delay specifically includes:

Determine the CIG play point according to Render_m_n=Init_BTS+transport_delay+ATS_n+Presentation_delay;

Where Render_m_n is the CIG play point, Init_BTS is the initial Bluetooth timestamp, ATS_n is the audio timestamp, transport_delay is the transmission delay, Presentation_delay is the playback delay, and m indicates that the packet is included Is the m-th data packet on the Bluetooth clock, and n indicates that the data packet is the n-th data packet on the audio clock.
An electronic device, characterized by comprising: one or more processors, memory, wireless communication module and mobile communication module; wherein:

The memory, the wireless communication module, and the mobile communication module are coupled to the one or more processors. The memory is used to store computer program code. The computer program code includes computer instructions. When the one or When multiple processors execute the computer instructions, the electronic device executes the audio data synchronization method according to any one of claims 1-5.
A Bluetooth headset, characterized in that the Bluetooth headset includes at least one earplug;

The earplug includes: a processor, a memory, a wireless communication module, a receiver, and a microphone; the memory, the wireless communication module, the receiver, and the microphone are coupled to the processor, and the memory is used to store a computer program Code, the computer program code includes computer instructions;

When the processor executes the computer instruction, the earplug executes the audio data synchronization method according to any one of claims 6-12.
A Bluetooth audio system, characterized in that the Bluetooth audio system includes: an electronic device, a first earplug and a second earplug; wherein:

The electronic device is configured to send a data packet to the first earplug through a first connection-based isochronous stream (CIS) based connection isochronous stream (CIG), and through a second CIS of the CIG, Sending the data packet to the second earbud; the data packet includes: audio data, an audio timestamp of the data packet, and an initial Bluetooth timestamp, the initial Bluetooth timestamp is the electronic device and the The first earbud establishes the first CIS, and the Bluetooth timestamp of the first data packet transmitted after establishing the second CIS with the second earbud;

The first earplug is used to receive the data packet, determine a CIG playback point based on the audio timestamp, the initial Bluetooth timestamp, transmission delay and playback delay, and play the data packet at the CIG playback point Audio data in

The second earplug is used to receive the data packet, determine a CIG playback point based on the audio timestamp, the initial Bluetooth timestamp, transmission delay and playback delay, and play the data packet at the CIG playback point Audio data in.
A Bluetooth chip, characterized in that the Bluetooth chip includes a memory, a processor and a radio frequency module, the memory is used to store computer program code, and the computer program code includes computer instructions; wherein:

The processor executes computer instructions stored in the memory to control the radio frequency module to send a data packet to the first earbud through a first connection-based isochronous stream (CIG) based connection-based isochronous audio stream (CIS), Sending the data packet to the second earplug through the second CIS of the CIG;

The data packet includes: audio data, an audio timestamp of the data packet, and an initial Bluetooth timestamp, the initial Bluetooth timestamp establishes the first CIS for the electronic device and the first earbud, and all The Bluetooth timestamp of the first data packet transmitted after the second earbud establishes the second CIS; the audio timestamp and the initial Bluetooth timestamp are used for the first earbud and the second earbud to determine the CIG Play point, the CIG play point is a time point at which the first earplug and the second earplug play the audio data in the data packet.
A Bluetooth chip, characterized in that the Bluetooth chip includes a memory, a processor and a radio frequency module, the memory is used to store computer program code, and the computer program code includes computer instructions; wherein:

The processor executes computer instructions stored in the memory to control the radio frequency module to receive a data packet from an electronic device through a connection-based isochronous stream group (CIG) connection-based isochronous audio stream (CIS); The data packet includes: audio data, an audio timestamp of the data packet, and an initial Bluetooth timestamp, the initial Bluetooth timestamp is the Bluetooth timestamp of the first data packet transmitted after the CIS is established; according to the audio The timestamp, the initial Bluetooth timestamp, the transmission delay and the playback delay determine the CIG playback point; so that the microphone including the earbuds of the Bluetooth chip plays the audio data in the data packet at the CIG playback point.
A computer storage medium, characterized by comprising computer instructions, when the computer instructions run on an electronic device, the electronic device is caused to perform the audio data synchronization method according to any one of claims 1-5 .
A computer storage medium, characterized by comprising computer instructions, when the computer instructions run on the earphone of the Bluetooth headset, the earphone of the Bluetooth headset executes the audio according to any one of claims 6-12 Data synchronization method.