KR20220050641A - Electronic device and method for recording audio singnal using wireless microphone device in the same - Google Patents

Abstract

According to various embodiments, an electronic device includes a wireless communication module, a memory, and a processor, wherein the memory may include instructions for allowing the processor to: connect a wireless audio input/output device from which a first channel device and a second channel device are separated; change communication setting to a multi recording communication method so that an audio signal transmitted from the wireless audio input/output device is classified as a first channel packet and a second channel packet in a multi recording mode using the wireless audio input/output device; receive an audio signal based on a sound acquired from a microphone of the wireless audio input/output device; classifying the received audio signal as a first channel packet and a second channel packet; and execute a multi recording function by generating a first channel audio signal based on the first channel packet and a second channel audio signal based on the second channel packet. The electronic device can support multi recording (e.g., stereo recording and binaural recording) functions using the wireless audio input/output device.

Description

ELECTRONIC DEVICE AND METHOD FOR RECORDING AUDIO SINGNAL USING WIRELESS MICROPHONE DEVICE IN THE SAME

Various embodiments relate to an electronic device and an audio recording method using a wireless audio input/output device in the electronic device.

Recently, with the development of technology, various wearable devices have been developed and released along with electronic devices. Recently, in the case of audio input/output devices such as earphones or headsets, the use of devices using a wireless connection method (eg, wireless earbuds, wireless audio input/output devices (eg, true wireless stereo) devices) has increased from a wired connection method. is a trend that

As an example, the wireless audio input/output device may include a speaker or a microphone. The electronic device may output an audio signal or acquire an audio signal using a wireless audio input/output device connected to the electronic device.

Electronic devices provide various functions to users, and there is a demand from users for recording sound using the electronic device. However, when recording using the built-in microphone of an electronic device, it may be difficult to acquire a voice depending on the distance between the talker's utterance point (eg, the talker's mouth) and the electronic device. This may cause a problem in that it cannot be recorded accurately. For this reason, a separate microphone device with excellent performance may be required.

According to various embodiments, a method for overcoming the limitation of the built-in microphone of an electronic device and recording high-quality or realistic sound by recording sound using microphones included in the wireless audio input/output device without separate equipment would like to propose

According to various embodiments, an electronic device includes a wireless communication module, a memory, and a processor, wherein the memory is connected to the processor, the first channel device and the second channel device are separated from the wireless audio input/output device, and the In the multi-recording mode using the wireless audio input/output device, the communication setting is changed to a multi-recording communication method changed so that the audio signal transmitted from the wireless audio input/output device is divided into a first channel packet and a second channel packet, and the wireless audio input/output device Receives an audio signal based on the sound obtained from the microphone, divides the received audio signal into a first channel packet or a second channel packet, and generates a first channel audio signal based on the first channel packet, It may include instructions for generating a second channel audio signal based on the second channel packet to execute a multi-recording function.

According to various embodiments, there is provided a wireless audio input/output device, comprising: a first wireless audio device including a first microphone and a first communication module; and a second wireless audio device including a second microphone and a second communication module; The first wireless audio device is wirelessly connected to an electronic device through a first communication link, the first wireless audio device and the second wireless audio device are connected to each other via a second communication link, and the electronic device and a first wireless audio device connected through a first communication link controls to activate the first microphone and the second microphone in response to receiving a multi-recording activation signal from the electronic device, and A second wireless audio device and a changed multi-recording communication method so that packets are alternately transmitted to the electronic device, the communication setting is changed, the audio signal obtained from the first microphone is transmitted using predetermined first slots, and the electronic device and the second wireless audio device to which the communication link is not connected may transmit the audio signal obtained from the second microphone by sniffing the first communication link using predetermined first slots.

According to various embodiments of the present disclosure, in an audio recording method using a wireless audio input/output device in an electronic device, an operation of connecting a first channel device and a second channel device to separate wireless audio input/output devices; An operation of changing a communication setting to a multi-recording communication method changed so that the audio signal transmitted from the wireless audio input/output device is divided into a first channel packet and a second channel packet based on the reception of the recording request signal and the wireless audio input/output An operation of receiving an audio signal based on a sound obtained from a microphone of the device, an operation of dividing the received audio signal into a first channel packet or a second channel packet, and an operation of generating a first channel audio signal based on the first channel packet and generating a second channel audio signal based on the second channel packet to execute a multi-recording function.

The electronic device according to various embodiments may support a multi-recording (eg, stereo recording, binaural recording) function using a wireless audio input/output device in which channels are separated to the left or right or connected in a plurality of sets.

The electronic device and the wireless audio input/output device according to various embodiments support a recording function that was not possible in the standard Bluetooth audio transmission method, so that it is possible to record a realistic sound or a realistic sound that a user hears with both ears and provide it to the user. there is.

1 is a block diagram of an electronic device in a network environment according to various embodiments of the present disclosure;
2 is a diagram illustrating an electronic device and a wireless audio input/output device according to various embodiments of the present disclosure.
3 illustrates a configuration of an electronic device and an ear wearable device according to various embodiments of the present disclosure.
4 illustrates an audio recording method using a wireless audio input/output device in an electronic device according to various embodiments of the present disclosure.
5 illustrates packet timing of a standard transmission scheme of a SCO link according to an embodiment.
6A and 6B illustrate packet timing of a multi-recording transmission scheme of an SCO link between an ear wearable device and an electronic device according to an embodiment.
7 illustrates packet timing of a standard transmission scheme of an ACL link according to an embodiment.
8 illustrates packet timing of a multi-recording transmission method of an ACL link between an ear wearable device and an electronic device according to an embodiment.
9 specifically illustrates a transmission period of an ACL link according to an embodiment.

Hereinafter, various embodiments of the present document may be described with reference to the accompanying drawings.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments.

Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 may be included. In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in the volatile memory 132 , and may process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 may use less power than the main processor 121 or may be set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). there is. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ). The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used in a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 or an external electronic device (eg, a sound output module 155 ) directly or wirelessly connected to the electronic device 101 . A sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to one embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 includes a direct (eg, wired) communication link (or communication channel) between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108); transmission link), and wireless communication through the established communication link. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 includes various technologies for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, underside) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or part of the operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

The electronic device 101 according to various embodiments may have various types of devices. The electronic device 101 may include, for example, a portable communication device (eg, a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device 101 according to various embodiments of the present document is not limited to the aforementioned devices.

2 is a diagram illustrating an electronic device and a wireless audio input/output device according to various embodiments of the present disclosure.

Referring to FIG. 2 , according to an embodiment, the electronic device 101 may be wirelessly connected to the wireless audio input/output device. The wireless audio input/output device may include an ear wearable device, a wireless earphone, a wireless headset, an ear bud, and a true wireless stereo (TWS) device.

For example, in the embodiment of FIG. 2 , the wireless audio input/output device is illustrated as an ear wearable device 210 in which the first ear device 210a and the second ear device 210b are configured as a pair, but the present invention is not limited thereto. The wireless audio input/output device may be one accessory device (eg, a wireless headset) each having a microphone at a first position (eg, toward the right ear) and a second position (eg, toward the left ear). The wireless audio input/output device is connected to the electronic device 101 through a short-range wireless communication network, and may be a plurality of Internet of Things (IOT) devices including a microphone.

According to an embodiment, the electronic device 101 communicates with the ear wearable device 210 (eg, the first ear device 210a and the second ear device 210b) through a short-range wireless communication network (eg, Bluetooth communication). can be connected

According to an embodiment, the electronic device 101 connects with any one device operating as a master among the first ear device 210a and the second ear device 210b (eg, single pairing). )) can be For example, the first ear device 210a (eg, a main device) operating as a master maintains the electronic device 101 and the first communication link (or first transmission link) 220 while maintaining a Bluetooth low energy) method or a Bluetooth legacy method, the second ear device 210b (eg, an auxiliary device) may be connected with the second communication link (or second transmission link) 230 . For example, the first communication link may be connected in a BT manner, and the second communication link may be connected in a BLE manner. The electronic device 101 may confirm that the first ear device 210a and the second ear device 210b connected through the second communication link 230 are connected as a pair. For convenience of description, the first ear device 210a directly connected to the electronic device 101 through wireless communication is defined as a primary device (eg, primary equipment (PE)), and a second ear device not directly connected to the electronic device 101 is defined. The device 210b may be defined as an auxiliary device (eg, secondary equipment (SE)).

According to an embodiment, the first ear device 210a and the second ear device 210b receive a control signal from the electronic device 101 or from the first ear device 210a or the second ear device 210b. A function may be performed according to the generated control signal. The control signal received from the electronic device 101 may be a signal related to at least one of a play function, a call function, a stop function, a volume control function, and a recording function related to audio input/output. .

According to an embodiment, the first ear device 210a and the second ear device 210b provide an audio signal (eg, an audio packet, a voice signal, and audio data) from the electronic device 101 . ) and outputting it as sound through the speaker. The first ear device 210a and the second ear device 210b receive an external sound through a microphone, convert the received sound into an audio signal (or audio packet), and deliver the function to the electronic device 101 . can do.

According to an embodiment, the first ear device 210a (eg, the main device) includes first communication link information (eg, BT address (BD_ADDR), FHS (frequency hop synchronization) packet information, Link Key information, serial number) Alternatively, the state information of the electronic device 101 may be shared with the second ear device 210b through the second communication link 230 .

According to an embodiment, while the first ear device 210a (eg, the primary device) communicates with the electronic device 101 via the first communication link 220 , the second ear device 210b (eg, the secondary device) device) may monitor the first communication link 220 using the first communication link information.

According to an embodiment, the second ear device 210b (eg, an auxiliary device) accesses or sniffs the first communication link 220 , thereby connecting the first ear device 210a from the electronic device 101 ( For example, it is possible to obtain an audio signal transmitted to the main device). For example, a sniffing (or snooping) operation uses information about a communication link (or communication channel) between other electronic devices to access a communication link, and acquire information transmitted and received through the communication link can mean doing For example, the second ear device 210b (eg, an auxiliary device) may acquire an audio signal transmitted through the first communication link 220 through a sniffing operation, and sends the An audio signal may also be transmitted.

According to an embodiment, the electronic device 101 receives an audio signal (eg, a packetized bit stream, a packet) (eg, 240, 245) from the first ear device 210a or the second ear device 210b. can do. The electronic device 101 may recognize an audio signal transmitted from the first ear device 210a or the second ear device 210b as an audio signal transmitted through one ear wearable device 210 .

According to an embodiment, the first communication link 220 may include a Bluetooth communication link, but is not limited thereto. In the Bluetooth communication link, packets (eg, audio signals) may be transmitted and received through at least one of an asynchronous method (eg, an asynchronous connection-oriented (ACL) link) and a synchronous method (eg, a synchronous connection-oriented (SCO) link). .

According to an embodiment, the first communication link 220 may define a hands-free profile (HFP) or an advanced audio distribution profile (A2DP) according to a usage purpose. HFP can transmit packets using the SCO link. A2DP can transmit packets using the SCO link. For example, in a synchronous connection-oriented (SCO) link, a SCO packet may be transmitted through a time slot reserved at regular time intervals for voice data transmission. The SCO link can support point-to-point communication in a symmetrical and circuit switched manner for voice signals.

For another example, an ACL (Asynchronous connection-oriented) link may transmit an ACL packet without using a reserved time slot for burst data transmission. The ACL link may retransmit packets with dropped or erroneous packets. The ACL link may support point-to-multipoint communication in a symmetric or asymmetric and packet-switched manner. An ACL link may support both symmetric and asymmetric transmission.

According to an embodiment, the electronic device 101 and the ear wearable device 210 transmit and receive packets using a standard transmission method (or standard transmission protocol) based on the SCO link or the ACL link, or the first channel and the second channel Packets can be transmitted/received using the multi-recording transmission method (or multi-recording transmission protocol) changed to distinguish this. The standard transmission method may be set as a default in the electronic device 101 and the ear wearable device 210 . When the electronic device 101 and the ear wearable device 210 operate in the multi-recording mode, communication settings may be changed in the multi-recording transmission method. The multi-recording transmission method may be a communication method in which a transmission sequence is changed to be recognized as a first channel packet and a second channel packet for each frame section.

3 illustrates a configuration of an electronic device and an ear wearable device according to various embodiments of the present disclosure.

Referring to FIG. 3 , according to an exemplary embodiment, the ear wearable device ( 210 in FIG. 2 ) includes a first ear device 210a and a second ear device for a user's left ear and right ear. device 210b.

According to one embodiment, the first ear device 210a and the second ear device 210b include microphones 311 and 321, speakers 312 and 322, communication circuits 313 and 323, encoders 314 and 324 and controllers 315 and 325, respectively. can do it all Although the first ear device 210a and the second ear device 210b are not shown, sensors (eg, a wear detection sensor, a proximity sensor, a biometric sensor, a grip sensor, a temperature sensor, a gyro sensor, a geomagnetic sensor), a memory and a battery.

Additionally, the encoders 314 and 324 may be replaced with the audio module of FIG. 1 , and may include at least some of the components included in the audio module.

According to an embodiment, the microphones 311 and 321 may receive (or acquire/acquire) a sound from an external (eg, a user or a peripheral device). The microphones 311 and 321 may acquire external sound and process (eg, convert) it into an electrical signal.

According to an embodiment, the speakers 312 and 322 may output an audio signal obtained through the microphones 311 and 321 or a communication circuit as sound. The speakers 312 and 322 may convert an audio signal (or audio data, or an audio packet) received through the communication circuits 313 and 323 into sound and output the converted sound.

According to one embodiment, the encoders 314 and 324 may be audio encoders. The encoders 314 and 324 may encode the sound obtained through the microphones 311 and 321 and generate an audio packet based on the encoded audio bitstream. Audio packets generated through the encoders 314 and 324 may be transmitted to the electronic device 101 through the communication circuits 313 and 323, respectively. The audio packet may include at least one of audio bitstream information, channel information, and sync information.

According to an embodiment, the encoders 314 and 324 encode one audio frame to generate one audio packet, and may count a frame index value whenever one audio frame is encoded. The frame index value (or frame number) may be sequentially increased (eg, frame_idx=0, 1, 2, 3, 4, 5,,,,) whenever every audio frame is encoded.

According to an embodiment, when encoding is performed for sounds each acquired by the first ear device 210a and the second ear device 210b based on the same time, an audio packet having the same frame index value is generated for each frame can be For example, the first ear device 210a generates packet_1 having an index value of 1 in the frame 1 section based on a sound acquired through the first microphone (eg 311 ), and the second ear device 210b Each packet_1 may be generated in the frame 1 section based on the sound acquired through the second microphone (eg, 321 ). The sync information may include a frame index value.

According to some embodiments, the encoders 314 and 324 may generate an audio packet by inserting channel information. When the first ear device 210a has the first channel identification code, the second ear device 210b is It may have a channel identification code. For example, the encoder (eg 314) of the first ear device inserts the first channel identification code into the audio packet (eg packet_1), and the encoder (eg 324) of the second ear device inserts the second channel identification code can be inserted into an audio packet (eg, packet_1). The channel information may be inserted into at least a part of the header packet.

According to an embodiment, the electronic device 101 may generate a first channel audio signal by dividing channel information included in an audio packet, and may generate a second channel audio signal.

According to an embodiment, the first ear device 210a and the second ear device 210b may further include synchronization control units 316 and 326 . The sync control units 316 and 326 may adjust the sync to synchronize the two audio signals acquired from the first microphone 311 of the first ear device 210a and the second microphone 321 of the second ear device 210b. there is. For example, the first ear device 210a and the second ear device 210b may be connected through a second communication link, and synchronize sync information to determine the timing of the encoding operation and the signals input from the microphones 311 and 321 . It can be controlled to synchronize (or adjust).

According to some embodiments, the sync control units 316 and 326 or operations thereof may be skipped. For example, since an audio packet (eg, SCO packet) through a synchronous SCO link supports synchronization between SCO links, the operation of the sync controllers 316 and 326 may be omitted. When an audio packet is transmitted through an asynchronous ACL link, a synchronization operation may be requested through the sync control units 316 and 326 .

According to an embodiment, the communication circuits 313 and 323 are Bluetooth, low energy Bluetooth, Wi-Fi, adaptive network topology (ANT+), long term evolution (LTE), 5th generation mobile telecommunication (5G), and/or NB. -Connected to each other or the electronic device 101 through a narrowband internet of things (IoT), or may be connected to an access point (AP, access point), or other networks in some embodiments.

According to an embodiment, any one of the communication circuits 313 and 323 may establish a first wireless communication link with the electronic device 101 and wirelessly communicate through the established first communication link. The communication circuit 313 of the first ear device 210a and the second communication circuit 323 of the second ear device 210b establish a second communication link with each other and perform communication through the established second communication link. can

According to an embodiment, the controllers 315 and 325 may control the overall operation of the ear wearable device. The controllers 315 and 325 may control other components (eg, hardware or software components) and operations of the ear wearable device 210 connected to the controllers 315 and 325, and may perform various data processing or calculations. there is. For example, the controllers 315 and 325 determine the wearing state of the first ear device 210a and the second ear device 210b based on sensor data (or sensing information) obtained from the sensor, and You can control the communication connection. The controllers 315 and 325 may control the communication circuits 313 and 323 to transmit and receive a voice signal or an audio signal (or audio data) using a communication link.

According to an embodiment, a data transmission profile of a standard standard may be set as a default for the communication module 340 of the electronic device 103 and the communication circuits 313 and 323 of the ear wearable device 210 during Bluetooth pairing. When the electronic device 103 and the ear wearable device 210 are connected to the SCO link, data is transmitted and received in a sequence according to the standard specification of the SCO method, and when connected to the ACL link, data is transmitted and received in a sequence according to the standard specification of the ACL method. can

The controllers 315 and 325 may check an operation mode with the electronic device 101 and change a data transmission profile according to the operation mode.

According to an embodiment, the controllers 315 and 325 change the operation modes of the first ear device 210a and the second ear device 210b to the multi-recording mode ( or recording mode). The controllers 315 and 325 may activate the microphones 311 and 321 in the multi-recording mode, control to acquire external sound, and encode and packetize the acquired sound. The controllers 315 and 325 may control a communication circuit to transmit packetized sound, that is, an audio signal, to the electronic device 101 through a communication link connected to the electronic device 101 .

According to an embodiment, when operating in the multi-recording mode, the controllers 315 and 325 change the data transmission protocol to a protocol of the multi-recording communication method that is changed so that an audio signal other than the standard method is divided into a first channel packet and a second channel packet. can For example, in the transmitting-side device (eg, the ear wearable device 210 ), in the multi-recording communication protocol, the first ear device 210a and the second ear device 210b alternately transmit packets in a predetermined reserved slot, respectively. It may have a sequence in which packets are transmitted to the electronic device 101 , or packets are alternately transmitted to the electronic device 101 every interval interval.

When operating in the multi-recording mode, the controller 315 of the first ear device 210a transmits the first audio packet generated from the signal obtained from the first microphone 311 through the first communication circuit 313 to the electronic device 101 ) can be transmitted. The controller 325 of the second ear device 210b may transmit a second audio packet generated from a signal obtained from the second microphone 321 to the electronic device 101 through the second communication circuit 323 . .

The electronic device 101 according to an embodiment may include a communication module 340 , an audio processor 341 , a processor 342 , and a memory 343 , and the configuration of the electronic device 101 of FIG. 1 and / or may include at least some of the functions.

The communication module 340 may receive an audio signal (eg, audio packets) from the ear wearable device 210 . The communication module 340 may be connected to at least one of the first ear device 210a and the second ear device 210b through a first communication link. The communication module 340 may receive a signal from each of the first ear device 210a and the second ear device 210b. Among the first ear device 210a and the second ear device 210b, the main device transmits a signal to the electronic device 101 through the first communication link, and the auxiliary device uses the first communication link through a sniffing operation to send a signal to the electronic device. It is possible to transmit a signal to the device 101 .

The audio processor 341 may include a sync adjustment module 3411 , a packet loss compensation module 3412 , a decoder 3413 , and a post-processing module 3414 . The sync control module 3411 , the decoder 3413 , the packet loss compensation module 3412 , and the post-processing module 3414 may be implemented as software, but may be implemented as a separate hardware configuration (eg, an audio chip). .

The sync adjustment module 3411 may control to synchronize (or adjust) the timing of the decoding operation so that audio signals encoded at the same time can be decoded together. For example, the sync control module 3411 may transmit an audio bitstream having the same value to the audio decoder based on a frame index value included in an audio signal (eg, an audio packet).

According to some embodiments, the sync control module 3411 or an operation thereof may be omitted. For example, since an audio packet (eg, SCO packet) transmitted through a synchronous transmission link (eg, SCO link) supports synchronization between transport links, the operation of the sync control module 3411 may be omitted.

The decoder 3413 may be an audio decoder. The decoder 3413 decodes the audio signal (or packetized audio bitstream) transmitted from the ear wearable device 210 (eg, the first ear device 210a and the second ear device 210b) to generate the audio signal. can create The decoder 3413 may distinguish and decode the audio signal transmitted from the first ear device 210a and the audio signal transmitted from the second ear device 210b. The audio signal input to the decoder may be a packetized bit stream encoded by the first ear device 210a and the second ear device 210b at the same time through sync adjustment (or synchronization), respectively.

For example, the processor 342 or the audio processor 341 may convert the audio signal (or audio packets) received from the ear wearable device 210 to a first channel signal (or first channel signal) transmitted from the first ear device 210a. 1 channel packets) and a second channel signal (or second channel packets) transmitted from the second ear device 210b may be distinguished. The audio processor 341 may separately obtain the first channel packets and the second channel packets from the audio signal received from the ear wearable device 210 .

According to an embodiment, the decoder 3413 is configured for each reserved time slot (eg, slot1 or slot1~2) in the first ear device 210a among packets (eg, SCO packets) received through the SCO link. A first channel audio signal may be generated based on transmitted packets (eg, first channel packets). The decoder 3413 is configured to receive a second channel based on packets (eg, second channel packets) transmitted in every reserved time slot (eg, slot2 or slot3 to 4) of the second ear device 210b among the received packets. A two-channel audio signal can be generated.

According to an embodiment, the decoder 3413 is configured to perform a reserved time interval (eg, a first interval of an interval time) in the first ear device 210a among packets (eg, ACL packets) received through the ACL link. A first channel audio signal may be generated based on packets (eg, first channel packets) transmitted every time. The decoder 3413 configures the second channel audio signal based on packets (eg, second channel packets) transmitted for each reserved time interval (eg, the second interval of the interval time) in the second ear device 210b. can create When the ACL packets are received through the ACL link, the processor 342 may store the first channel packets in the first buffer and store the second channel packets in the second buffer. The decoder 3413 may decode the first channel packets stored in the first buffer and decode the second channel packets stored in the second buffer according to the set schedule.

The packet loss compensation module 3412 may perform an operation of compensating for an audio signal (or audio packet) lost in wireless transmission through a wireless communication link (or wireless transmission link). The packet loss compensation module 3412 may identify a missing audio frame based on an index value included in the audio packet, and generate an audio signal related to the missing frame. According to an embodiment, the packet loss compensation module 3412 may retransmit the missed audio frame through the communication link.

According to some embodiments, whether a packet is missing may be checked through, for example, whether an ack signal is received in a Bluetooth communication protocol.

The post-processing module 3414 performs processing for sound quality improvement and binaural effect based on the multi-audio signal (or stereo audio signal, or the first channel audio signal and the second channel audio signal) decoded by the decoder 3413. can be done For example, the post-processing module may perform at least one of bandwidth extension (BWE), equalizer (EQ), and sound effect (eg, surround effect) processing. For example, the post-processing module 3414 may be configured to have a binaural sound using a first channel audio signal and a second channel audio signal generated based on sound recorded by the first ear device and the second ear device, respectively. A recording signal can be generated.

The processor 342 may control the overall operation of the electronic device 101 . The processor 342 may control other components (eg, hardware or software components) and operations of the components of the electronic device 101 connected to the processor 342 , and may perform various data processing or operations. . The processor 342 may include at least some of the operations and functions of the processor 120 illustrated in FIG. 1 .

As at least part of data processing or operation, the processor 342 may store commands or data for controlling the overall operation of the electronic device 101 in the memory 343 , and process the commands or data stored in the memory 343 . there is.

The processor 342 may check the operation mode with the wearable device 210 and change the data transmission profile of the communication link according to the operation mode. The processor 342 receives a recording function activation request signal (eg, a user input for selecting a recording function (or multi-recording function) using the ear wearable device 210) through an input device (not shown) (eg, a touch sensor). can receive The processor 342 may transmit a recording function activation signal to the ear wearable device 210 based on receiving a user input for selecting a recording function using the ear wearable device 210 .

The processor 342 is a multi-recording communication protocol (eg, multi-recording transmission) changed so that, in the multi-recording mode (or recording mode), an audio signal other than a standard method is divided into a first channel packet and a second channel packet, the data transmission protocol protocol) can be changed. In the receiving device (eg, the electronic device 101), in the protocol of the multi-recording communication method, packets are alternately received from the first ear device 210a and the second ear device 210b in a predetermined reserved slot, and during the interval time It may have a sequence in which packets are alternately received from the first ear device 210a and the second ear device 210b at every predetermined interval.

The processor 342 controls the audio processor 341 , that is, the sync adjustment module 3411 , the packet loss compensation module 3412 , the decoder 3413 , and the post-processing module 3414 to obtain from the ear wearable device 210 . A multi-recording operation may be performed based on the sound of one microphone 311 and 321, and a recording signal having a binaural sound may be generated.

4 illustrates an audio recording method using a wireless audio input/output device in an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 4 , the processor (eg, the processor 120 of FIG. 2 and the processor 342 of FIG. 3 ) of the electronic device 101 according to an embodiment performs wireless audio input/output through a communication module in operation 410 . It may be communicatively connected to a device (eg, the ear wearable device 210 of FIG. 2 ). The electronic device 101 and the wireless audio input/output device (eg, the ear wearable device 210) may form a communication link (or transmission link) based on Bluetooth communication (eg, Bluetooth pairing). For example, , the communication link may be at least one of an asynchronous connection-oriented (ACL) link and a synchronous connection-oriented (SCO) link.

In operation 420 , the processors 120 and 342 may determine whether the operation mode of the electronic device 101 is operating in the multi-recording mode. For example, the processors 120 and 342 multi-record the operation mode of the electronic device 101 in response to a recording function activation signal using a microphone of an external device (eg, a wireless audio input/output device) together with the microphone of the electronic device 501 . mode can be switched.

According to an embodiment, the electronic device 101 may execute a multi-recording function (eg, image capturing) by a user selecting a Bluetooth microphone use item (or function). According to an embodiment, the electronic device 101 may display devices (eg, Bluetooth devices) including a microphone that can be activated through a user interface (UI). The user can switch to the multi-recording mode by selecting devices to be used for recording by referring to the information displayed on the UI. For another example, the electronic device 101 may display the activated device in response to the multi-recording function activation signal through the UI.

In operation 425, the processors 120 and 342 transmit the multi-recording activation signal to the wireless audio input/output device (eg, the ear wearable device 210) to the communication module (eg, the communication module 190 of FIG. 1 and the communication module of FIG. 3 ) (340)) can be controlled.

If the electronic device 101 is in an operation mode (or normal mode) other than the multi-recording mode, the processors 120 and 342 may transmit and receive BT packets (or data) in a Bluetooth standard communication method in operation 430 . The other operation mode (or normal mode) may include at least one of a call mode and an audio playback mode, but is not limited thereto.

In operation 440, the processors 120 and 342 transmit multi-recording recognized as the first channel packet and the second channel packet from the audio signal received from the wireless audio input/output device (eg, the ear wearable device 210) when switching to the multi-recording mode. You can change the communication settings by method (or multi-recording transmission protocol). For example, in the multi-recording transmission method, a sequence determined to divide packets received from a wireless audio input/output device (eg, the ear wearable device 210 ) recognized as one device into first channel packets or second channel packets is performed. can have

In operation 450, the processors 120 and 342 receive an audio signal, for example, audio packets (eg, a packetized bitstream) from a wireless audio input/output device (eg, the ear wearable device 210) based on the changed multi-recording transmission method. can receive Audio packets may be SCO packets or ACL packets according to the communication link type.

According to an exemplary embodiment, when the electronic device 101 and the wireless audio input/output device (eg, the ear wearable device 210) are connected by an SCO link, the electronic device 101 is connected to the wireless audio input/output device (eg, the ear wearable device) 210)), the sound obtained from the microphone can be received as SCO packets.

According to an exemplary embodiment, when the electronic device 101 and the wireless audio input/output device (eg, the ear wearable device 210) are connected by an ACL link, the electronic device 101 is connected to the wireless audio input/output device (eg, the ear wearable device) 210)), the sound obtained from the microphone can be received as ACL packets.

In operation 460, the processors 120 and 342 may obtain (or check) the first channel packet and the second channel packet by dividing the audio packet received from the wireless audio input/output device (eg, the ear wearable device 210). .

In operation 470, the processors 120 and 342 may generate a first audio signal based on the first channel packet and generate a second audio signal based on the second channel packet to perform multi-recording. The processors 120 and 342 may generate a recording signal having a binaural sound based on the microphones 311 and 321 sounds obtained from the ear wearable device 210 through multi-recording.

Hereinafter, the Bluetooth standard communication method (standard communication protocol) and the multi-recording transmission method (or multi-recording transmission protocol) will be described for each communication link type.

5 illustrates packet timing of a standard transmission scheme of a SCO link according to an embodiment.

Referring to FIG. 5 , according to an embodiment, a standard transmission method of a synchronous connection-oriented (SCO) link may transmit/receive a packet through a time slot reserved at regular time intervals (eg, periods). The receiving device and the transmitting device may exchange packet transmission and ack signals through a master slot and a slave slot every frame period. One packet is transmitted for each frame, and an index value may be assigned to each frame. For example, packet 0 510 may be transmitted in slot 1 of the frame 0 section, and packet 1 511 may be transmitted in slot 1 of the frame 1 interval. When receiving the packet, the receiving device may transmit ack signals 520 and 521 in response thereto. When the ack signal is not received, the transmitting device may retransmit the same packet to the receiving device in the retransmission period R (eg, slot 3-6). The number of retransmissions may be different according to settings. For example, in the case of the frame 1 period, the ack signal is received for packet 1, whereas in the frame 2 period, the empty slot 522-1, that is, the ack signal is not received for the packet 2 512, so the same packet 2 (512) is retransmitted through slots 3 and 5, and it can be confirmed that the ack signal 522 is received through slot 6.

According to various embodiments, the electronic device 101 and the wearable device 210 transmit the first channel packet and the second channel packet according to the slot position rather than the standard transmission method through the synchronous connection-oriented (SCO) link. Communication can be performed based on the recognized multi-recording transmission method (or multi-recording transmission protocol). Hereinafter, a multi-recording transmission scheme of a synchronous connection-oriented (SCO) link will be described with reference to FIGS. 6A and 6B.

6A and 6B illustrate packet timing of a multi-recording transmission scheme of an SCO link between an ear wearable device and an electronic device according to an embodiment.

6A and 6B , according to various embodiments, an electronic device 601 (eg, the electronic device 101 of FIG. 1 , the electronic device 101 of FIG. 2 , the electronic device 101 of FIG. 3 ) The wire wearable device 602 (eg, the ear wearable device 210 of FIG. 2 and the ear wearable device 210 of FIG. 3 ) may communicate in a multi-recording mode using a multi-recording transmission method based on the SCO link. The electronic device 601 may be a receiving device, and the ear wearable device 610 (the first ear device 602a and the second ear device 602b) may be a transmitting device. For example, the first ear device 602a may be a primary device directly paired with the electronic device 601 , and the second ear device 602b may be an auxiliary device.

The SCO link may have designated time slots per frame period. For example, the time slots shown in FIGS. 6A and 6B are illustrated as having 10 slots, but the present invention is not limited thereto, and the slots are sequentially, slot1, slot2, slot3, slot4, ?? It may be referred to as slot10.

The first ear device 602a and the second ear device 602b may transmit the SCO packet to the electronic device 601 according to the timing sequence shown in FIG. 6A or 6B , respectively.

The first ear device 602a and the second ear device 602b may transmit the SCO packet using a predetermined slot by assigning a time index value according to the frame order.

The first ear device 602a transmits the SCO packet additionally including the first channel information to the electronic device 601 , and the second ear device 602b transmits the packet additionally including the second channel information to the electronic device 601 . can be transmitted

Packet timing may proceed in a frame increase direction according to the passage of time.

For example, looking at the packet timing shown in FIG. 6A , the ear wearable device 602, which is the transmitting device, allocates the first ear device 602a and the second ear device 602b for each slot, respectively, to the electronic device ( 601) to transmit the SCO packet. The first ear device 602a divides the sound acquired from the microphone (eg, the first microphone 311 of FIG. 3 ) into an SCO packet form to form a master slot (eg, slot1, slot3, slot5??). may be used to transmit to the electronic device 601 . The second ear device 602b divides the sound obtained from the microphone (eg, the second microphone 321 of FIG. 3 ) into an SCO packet form and divides it into a slave slot (eg, slot2. slot14, slot6??) in every transmission period. may be used to transmit to the electronic device 101 .

The first ear device 602a and the second ear device 602b may assign the same index number to packets transmitted in the same transmission period. For example, if it is assumed that the packet transmitted in frame 0 from the first ear device 602a is P0 610 , it may be assumed that the packet transmitted in frame 0 from the second ear device 602b is S0 620 . there is. Thereafter, it may be transmitted to P1 ( 611 ), P2 ( 612 )..Pn and S1 ( 621 ), S2 ( 622 )..Sn every frame period.

The first ear device 602a and the second ear device 602b may repeatedly transmit the same packets every transmission period. The retransmitted packets may use slots in the retransmission period, and the loss due to packet drop may be minimized by repeatedly transmitting the same packet a specified number of times (eg, twice or three times). For example, in frame 0, a packet identical to P0 may be retransmitted (610-1) through slots 3 and 5, and a packet identical to S0 may be retransmitted (620-1) through slots 4 and slot 6.

In FIG. 6A , the electronic device 601, which is a receiving device, classifies SCO packets transmitted for each frame slot1, slot3, and slot5 into a first channel packet, and for each frame, slot2. The SCO packets transmitted for each slot4 and slot6 can be divided into second channel packets. The electronic device 601 generates the first audio signal 640 by sequentially considering the index values of the first channel packets, and generates the second audio signal 645 by sequentially considering the index values of the second channel packets. Also, the same index value binds frame sections to obtain a multi-recording signal (or a stereo recording signal).

As another example, looking at the packet timing shown in FIG. 6B , in the case of the ear wearable device 602 as the transmitting device, the first ear device 602a transmits a P0 (610) packet using slot1 and slot2, The second ear device 602b may transmit the S0 620 packet using slot3 and lot4. Thereafter, the first ear device 602a may retransmit (610-1) the P0 packet through slot5 and lot6 for retransmission. The second ear device 602b may retransmit the SO packet through slot7 and lot8 (620-1). As with the packet timing of frame 0, the first ear device 602a transmits packets P1 (611) and P2 (612) based on slot1, slot2, slot5, and slot 6 for each frame, and the second ear device 602b may transmit packets S1 621 and S2 622 based on slot3, slot4, slot7, and slot8.

In FIG. 6B , the electronic device 601, which is a receiving device, classifies SCO packets transmitted at every slot1, slot2, slot5, and slot 6 for each frame into a first channel packet, and SCO transmitted at every slot3, slot4, slot7, and slot8 for each frame. The packet may be divided into a second channel packet. The electronic device 601 generates the first audio signal 640 by sequentially considering the index values of the first channel packets, and generates the second audio signal 645 by sequentially considering the index values of the second channel packets. It is possible to obtain a multi-recording signal (or a stereo recording signal) by bundling frame sections with the same index value.

7 illustrates packet timing of a standard transmission scheme of an ACL link according to an embodiment.

Referring to FIG. 7 , according to one embodiment, in the standard transmission method of the ACL link, the receiving device transmits an ack signal to the transmitting device when packet reception is completed, and the transmitting device sends the next frame order based on the response to the ack signal of the packet may be transmitted to the receiving device. In the standard transmission method of the ACL link, the timing and period of transmitting the packet may not be constant. For example, after the transmitting device transmits the P0 packet 710 having a size of slot 123, it may check that the ack signal 720 has been received from the receiving device, and may transmit the next frame packet, P1 711 .

After receiving the ack signal 721 for the P1 (711) packet, the transmitter transmits the P2 (712), but when there is no ack signal for this (722-1), the same packet 712 as the P2 (712) 1) can be transmitted until the ack 722 for P2 712 is received.

However, in the standard transmission method of the ACL link, since a reserved time slot for packet transmission is not determined, a problem in which two devices transmit packets at the same time may occur.

According to various embodiments, the electronic device 101 and the ear wearable device 210 alternately set the first channel packet and the second channel packet according to an interval determined for each arbitrarily set interval interval rather than a standard transmission method through the ACL link. Communication can be performed based on the multi-recording transmission method (or multi-recording transmission protocol) recognized as Hereinafter, a multi-recording transmission scheme on the ACL link will be described with reference to FIGS. 8 and 9 .

8 illustrates packet timing of a multi-recording transmission method of an ACL link between an ear wearable device and an electronic device according to an embodiment, and FIG. 9 specifically shows a transmission period of the ACL link according to an embodiment.

Referring to FIG. 8 , according to various embodiments, an electronic device 601 (eg, the electronic device 101 of FIG. 1 , the electronic device 101 of FIG. 2 , the electronic device 101 of FIG. 3 ) and an ear wearable The device 602 (eg, the ear wearable device 210 of FIG. 2 , the ear wearable device 210 of FIG. 3 ) may communicate in a multi-recording mode using a multi-recording transmission method based on an ACL link. The electronic device 801 may be a receiving device, and the ear wearable device 802 (the first ear device 802a and the second ear device 802b) may be a transmitting device. For example, the first ear device 802a may be a primary device directly paired with the electronic device 801 , and the second ear device 802b may be an auxiliary device.

The multi-recording transmission method in the ACL link sets a transmission period as much as a time interval (interval_T) for transmitting one packet, and sets the time interval (interval_T) for the first ear device 802a and the second ear device 802b. It can be assigned at different times. For example, as shown in FIG. 9 , when a transmission period is assumed by interval_T, the first ear device is divided into 1/2 of the interval_T interval, that is, in the first time interval, that is, the first interval (sub_interval 1), the ACL device is A packet (eg, P0 910 ) may be transmitted, and the second ear device may be configured to transmit an ACL packet (eg, S0 920 ) in a second time interval, that is, a second interval sub_interval2 .

The first ear device 802a and the second ear device 802b may assign a time index value according to a frame order and transmit the ACL packet according to an allocated time within a transmission period.

The first ear device 802a transmits the ACL packet additionally including the first channel information to the electronic device 801 , and the second ear device 802b transmits the packet additionally including the second channel information to the electronic device 801 . can

For example, looking at the packet timing shown in FIG. 8 , the ear wearable device 802 as the transmitting device transmits the ACL packet P0 only in the first interval (sub_interval 1) to the first ear device 802a every transmission period. (810), P1 (811), P2 (812)), and the second ear device 802b transmits ACL packets (S0 820, S1 821, S2 ( 822 )) may be transmitted.

The electronic device 601 may transmit ack signals 830,840,831,841,832,842 to the ear wearable device whenever ACL packet reception to the ear wearable device 802 is completed.

The first ear device 802a and the second ear device 802b may transmit the retransmission packet only in a predetermined allocation period when a packet is missing, for example, when an ack signal is not received from the electronic device. For example, since the ack signal is not received for the P1 811 packet, the P1 811 packet may be retransmitted every first interval (sub_interval 1) of the next period.

The electronic device 801, which is a receiving device, classifies the ACL packet acquired in the first interval (sub_interval 1) into the first channel packet for each transmission period, and the ACL packet acquired in the second interval (sub_interval 2) is the second Can be divided into 2-channel packets

The electronic device 801 may store the first channel packet in the first buffer 850 and store the second channel packet in the second buffer 855 . For example, in the case of the ACL method, since the packet reception timing is not constant, the electronic device 801 may store the ACL packet in a buffer and then perform a decoding operation.

The electronic device 801 sequentially considers the index values of the first channel packets stored in the first buffer 850 , and generates the first audio signal 870 through the audio encoder 860 , and stores the first audio signal 870 in the second buffer 855 . The second audio signal 875 may be generated by sequentially considering the index values of the stored second channel packets, and a multi-recording signal (or stereo recording signal) may be obtained by bundling frame sections with the same index value.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 120 ) of the device (eg, the electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term refers to the case where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to an embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store ^TM ) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly between smartphones (eg: smartphones) and online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

101,601,801: electronic device
210: ear wearable device
120,342: Processor
315,324: encoder
3413: decoder

Claims (19)

In an electronic device,
wireless communication module;
Memory; and
including a processor;
The memory is the processor,
The first channel device and the second channel device are connected to the separated wireless audio input/output device,
In the multi-recording mode using the wireless audio input/output device, the communication setting is changed to a multi-recording communication method that is changed so that the audio signal transmitted from the wireless audio input/output device is divided into a first channel packet and a second channel packet,
receiving an audio signal based on the sound obtained from the microphone of the wireless audio input/output device;
classifying the received audio signal into a first channel packet or a second channel packet;
An electronic device storing instructions for generating a first channel audio signal based on the first channel packet and generating a second channel audio signal based on the second channel packet to execute a multi-recording function. According to claim 1,
The memory, the processor,
control so that a Bluetooth wireless communication link is formed with the wireless audio input/output device through the communication module;
The Bluetooth communication link comprises at least one of a synchronous connection-oriented (SCO) link and an asynchronous connection-oriented (ACL) link. According to claim 1,
The memory, the processor,
The electronic device further comprising instructions for transmitting a multi-recording activation signal to the wireless audio input/output device based on a multi-recording request input using the wireless audio input/output device. 4. The method of claim 3,
The memory is the processor,
The electronic device further comprising instructions for setting a standard communication method as a default when pairing with the wireless audio input/output device and changing to the multi-recording communication method based on the multi-recording request input. 4. The method of claim 3,
The memory is the processor,
Checking the channel information included in the audio signal,
instructions for classifying an audio signal including a first channel identification code among the audio signals into the first channel packet, and classifying an audio signal including a second channel identification code among the audio signals into the second channel packet Further comprising an electronic device. 3. The method of claim 2,
If the audio signal obtained from the wireless audio input/output device is based on the SCO link, if it is a SCO packet,
The changed multi-recording communication method is
Without a response signal to the packet, the SCO packet received through the slots of the first position reserved for each frame period is recognized as the first channel packet, and received through the slots of the second position alternated with the first position. The SCO packet is configured to be recognized as the second channel packet. 3. The method of claim 2,
When the audio signal obtained from the wireless audio input/output device is based on an ACL link, if it is an ACL packet,
The changed multi-recording communication method is,
For each arbitrarily set interval interval, the ACL packet received through the slots of 1/2 of the interval interval is recognized as the first channel packet, and the ACL packet received through the slots of the 2/2 interval of the interval interval is The electronic device characterized in that it is set to be recognized as the second channel packet. 8. The method of claim 7,
The memory, the processor,
ACL packets received through the slots of the 1/2 section of the audio signal are stored in a first buffer, and the ACL packets received through the slots of the 2/2 section are controlled to be stored in a second buffer,
The electronic device further comprising instructions for generating the first channel audio signal based on the ACL packet stored in the first buffer and generating the second channel audio signal based on the ACL packet stored in the second buffer. 8. The method of claim 7,
The memory, the processor,
Further instructions for generating a multi-recording signal composed of the first audio signal and the second audio signal by grouping frame sections with the same index value based on the index value of the first channel packet and the index value of the second channel packet including electronic devices. A wireless audio input/output device comprising:
a first wireless audio device including a first microphone and a first communication module; and
a second wireless audio device including a second microphone and a second communication module;
The first wireless audio device is wirelessly connected to the electronic device through a first communication link, and the first wireless audio device and the second wireless audio device are connected to each other via a second communication link,
A first wireless audio device connected to the electronic device through a first communication link,
Control to activate the first microphone and the second microphone in response to receiving a multi-recording activation signal from the electronic device, and transmit a packet to the electronic device alternately between the first wireless audio device and the second wireless audio device Change the communication settings to the changed multi-recording communication method,
Transmitting the audio signal obtained from the first microphone using predetermined first slots,
The second wireless audio device to which the communication link is not connected to the electronic device,
and transmitting the audio signal obtained from the second microphone by sniffing the first communication link using predetermined first slots. 11. The method of claim 10,
The first communication link includes at least one of a synchronous connection-oriented (SCO) link and an asynchronous connection-oriented (ACL) link. 12. The method of claim 11,
When the first communication link is a SCO link,
The first wireless audio device transmits the SCO packet generated from the sound acquired through the first microphone to the electronic device through slots at a first location reserved for each frame section,
and the second wireless audio device transmits the SCO packet generated from the sound acquired through the second microphone to the electronic device through slots at a second location reserved for each frame section. 12. The method of claim 11,
When the first communication link is an ACL link,
The first wireless audio device transmits an ACL packet to the electronic device using slots of 1/2 of the interval for each arbitrarily set interval interval;
and the second wireless audio device transmits the ACL packet to the electronic device using slots of the 2/2 section of the interval section. 11. The method of claim 10,
The first wireless audio device and the second wireless audio device include:
A wireless audio input/output device, characterized in that each audio signal is generated to have the same frame index value based on the same time. 11. The method of claim 10,
The first wireless audio device
generating an audio signal obtained through the first microphone including a first channel identification code;
The second wireless audio device,
Wireless audio input/output device comprising a second channel identification code and generating an audio signal obtained through the second microphone In an audio recording method using a wireless audio input/output device in an electronic device
connecting the first channel device and the second channel device to the separated wireless audio input/output device;
Based on the reception of the multi-recording request signal using the wireless audio input/output device, the communication setting is performed in a multi-recording communication method that is changed to divide the audio signal transmitted from the wireless audio input/output device into a first channel packet and a second channel packet. changing action;
receiving an audio signal based on sound acquired from a microphone of the wireless audio input/output device;
dividing the received audio signal into a first channel packet or a second channel packet; and
and generating a first channel audio signal based on the first channel packet and generating a second channel audio signal based on the second channel packet to execute a multi-recording function. 17. The method of claim 16,
The operation of connecting to the wireless audio input/output device includes:
Method according to claim 1, wherein the wireless audio input/output device is connected through Bluetooth communication through at least one of a synchronous connection-oriented (SCO) link and an Asynchronous connection-oriented (ACL) link. 17. The method of claim 16,
The operation of dividing the received audio signal into a first channel packet or a second channel packet includes:
When the audio signal obtained from the wireless audio input/output device is based on the SCO link and is a SCO packet, the SCO packet received through slots at the first position reserved for each frame section is recognized as the first channel packet, and the A method of recognizing an SCO packet received through slots in a second position alternately positioned with the first position as the second channel packet. 17. The method of claim 16,
The operation of dividing the received audio signal into a first channel packet or a second channel packet includes:
When the audio signal obtained from the wireless audio input/output device is based on an ACL link, if the ACL packet is an ACL packet, the ACL packet received through slots of 1/2 of the interval interval for each arbitrarily set interval interval is the first channel packet , and the ACL packet received through the slots of the 2/2 period of the interval period is recognized as the second channel packet.

Images