WO2024051638A1 - Procédé d'étalonnage de champ sonore, dispositif électronique et système - Google Patents

Procédé d'étalonnage de champ sonore, dispositif électronique et système Download PDF

Info

Publication number
WO2024051638A1
WO2024051638A1 PCT/CN2023/116692 CN2023116692W WO2024051638A1 WO 2024051638 A1 WO2024051638 A1 WO 2024051638A1 CN 2023116692 W CN2023116692 W CN 2023116692W WO 2024051638 A1 WO2024051638 A1 WO 2024051638A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
recorded
played
preset
calibration
Prior art date
Application number
PCT/CN2023/116692
Other languages
English (en)
Chinese (zh)
Inventor
代哲
胡浩
蔡超
周永强
高志伟
沈晓东
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2024051638A1 publication Critical patent/WO2024051638A1/fr

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
    • G10L21/0324—Details of processing therefor
    • G10L21/034—Automatic adjustment
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/21—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being power information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control

Definitions

  • This application relates to the field of terminal technology, and in particular to sound field calibration methods, electronic equipment and systems.
  • the acoustic environment in which the audio playback device is located is also a key factor affecting the sound effect.
  • an audio playback device plays audio, it generates a sound field in the space. Due to environmental interference, noise effects and other factors, the sound field is different from the target sound field, resulting in poor auditory effects and poor user experience.
  • This application provides a sound field calibration method, electronic equipment and system.
  • the method calculates calibration parameters based on the recorded audio recorded by the recording equipment during the large-screen device playing the preset audio.
  • the calibration parameters are used to calibrate the audio to be played, and can Effectively calibrate the sound field when playing audio to improve user experience.
  • embodiments of the present application provide a sound field calibration method, which is applied to large-screen devices.
  • the method includes:
  • the large-screen device sends recording instructions to the recording device; the recording instructions are used to instruct the recording device to record and obtain the recording audio of the preset audio; the recording audio includes the audio recorded by the recording device while the large-screen device plays the preset audio;
  • the large-screen device receives the recording audio sent by the recording device
  • the large-screen device determines the calibration parameters based on the recorded audio; the calibration parameters are used to calibrate the audio to be played;
  • the large screen device driver plays the calibrated audio to be played.
  • the above-mentioned driven playback refers to playing through at least one sound source;
  • the at least one sound source can be a sound source on a large-screen device, such as a speaker, etc., or it can be an external playback device of the large-screen device, such as a speaker, etc.
  • the sound source for the large-screen device to play the preset audio and the sound source for the large-screen device to play the audio to be played can be the same sound source, or they can be different sound sources; the sound source for the large-screen device to play the preset audio is the same as the sound source for the large-screen device to play the audio to be played.
  • the number of sound sources for large-screen devices to play audio to be played can be the same or different.
  • the audio is recorded through the recording device; further, the calibration parameters are calculated based on the recorded audio.
  • the accuracy of the calibration parameters is higher, and the user listens to the calibrated
  • the audio effect of the audio to be played is better, which can effectively improve the user experience.
  • the large-screen device drives the playback of preset audio in response to user instructions, including: the large-screen device plays the preset audio based on the metadata of the preset audio;
  • the large-screen device driver plays the calibrated audio to be played, including: the large-screen device plays the calibrated audio to be played based on the metadata of the audio to be played.
  • the metadata is audio-related data in addition to audio information.
  • the metadata may include mapping relationships between multiple sound channels and at least one sound source and corresponding loudness and gain of the multiple sound channels. Metadata is used to process audio or playback settings, which can improve the audio playback effect.
  • Embodiments of the present application can play the preset audio based on the metadata of the preset audio, and play the calibrated audio to be played based on the metadata of the audio to be played, which can effectively improve the audio playback effect and improve the user's listening experience; at the same time, it can improve the calibration accuracy.
  • both the preset audio and the audio to be played include audio of multiple channels
  • the metadata of the preset audio and the metadata of the audio to be played both include multiple channels and The mapping relationship of at least one sound source;
  • the multiple channels include the first channel and the second channel;
  • the large-screen device drives the playback of preset audio, including: the large-screen device is based on the first channel in the metadata of the preset audio.
  • the corresponding sound source plays the audio of the first channel in the preset audio;
  • the large-screen device driver plays the calibrated audio to be played, including: the large-screen device plays the audio of the second channel of the preset audio based on the sound source corresponding to the second channel in the metadata of the audio to be played.
  • both the preset audio and the audio to be played include audio of multiple channels
  • the metadata of the preset audio and the audio to be played includes the loudness and corresponding values of the multiple channels respectively.
  • Gain Multiple channels including third and fourth channels;
  • the loudness of the third channel audio in the preset audio played by the large-screen device is obtained by adjusting the loudness corresponding to the third channel in the preset audio metadata by adjusting the gain corresponding to the third channel in the preset audio metadata;
  • the loudness of the fourth channel of the calibrated audio to be played on a large-screen device is adjusted by adjusting the third channel of the preset audio metadata through the gain and calibration parameters corresponding to the fourth channel in the metadata of the audio to be played. The corresponding loudness is obtained.
  • the preset audio includes calibration audio
  • the recorded audio includes recorded calibration audio
  • the recorded calibration audio is used to calibrate the audio to be played.
  • the preset audio includes instrumented audio
  • the recorded audio includes recorded instrumented audio
  • the recorded instrumented audio is used to calibrate the audio to be played.
  • the preset audio includes noise audio
  • the recorded audio includes recorded noise audio
  • the recorded noise audio is used to calibrate the audio to be played.
  • adding noise audio to the preset audio can improve the anti-interference ability of the preset audio and improve the accuracy of sound field calibration.
  • the calibration parameters are obtained based on spatial parameters, and the spatial parameters are used to indicate the audio characteristics of the space where the preset audio is played.
  • Embodiments of the present application can calculate calibration parameters based on spatial parameters, taking into account the impact of user space on audio playback, and can effectively improve the accuracy of sound field calibration.
  • the recorded audio includes recorded instrumentation audio and recorded calibration audio
  • the large-screen device determines calibration parameters based on the recorded audio, including:
  • the large-screen device calculates spatial parameters based on reverberation time and recorded calibrated audio.
  • the large-screen device determines the calibration parameters based on the recorded audio including:
  • the large-screen device calculates the sweet spot frequency response energy of the recorded calibrated audio in the user's room based on the reverberation time and spatial parameters;
  • the large-screen device determines the calibration parameters based on the difference between the sweet spot frequency response energy and the target frequency response energy.
  • the position of the recording device may be at a standing wave node of certain frequencies. If the frequency response capability of the recorded audio is directly used to calculate the calibration parameters, a large error will occur. . Therefore, in the embodiment of this application, the spatial parameters are first calculated, and then the sweet spot frequency response energy is calculated based on the spatial parameters. Using the sweet spot frequency response energy to calculate the calibration parameters can reduce errors and improve the accuracy of sound field calibration.
  • each frame of the preset audio includes noise audio
  • the preset audio includes instrumented audio
  • the recorded audio includes recorded noise audio and recorded instrumented audio
  • large screen device Based on the recorded audio, calibration parameters are determined, including:
  • the large-screen device When the large-screen device recognizes the instrumented audio from the recorded audio, it determines the starting node of the reverberation attenuation of the recorded instrumented audio;
  • the large-screen device When the large-screen device recognizes noise audio from the recorded audio, it determines the end node of the reverberation attenuation of the recorded instrumented audio;
  • the large-screen device determines the reverberation time based on the recorded audio between the start node and the end node.
  • the recording device is located at the position where the user uses the large-screen device.
  • placing the recording device at the position where users usually use large-screen devices can make the calibrated auditory effect of the audio to be played more in line with the needs of human ears and improve user experience.
  • the large-screen device calculates spatial parameters based on the reverberation time and recorded calibration audio, including:
  • the large-screen device determines the microphone frequency response energy and direct sound energy based on the recorded calibration audio
  • Large-screen equipment determines spatial parameters based on reverberation sound energy and reverberation time.
  • the recorded calibration audio includes audio in multiple frequency bands
  • the large-screen device determines the spatial parameters based on the reverberation sound energy and reverberation time, including:
  • the large-screen device determines the spatial parameters corresponding to the audio of multiple frequency bands based on the reverberation sound energy corresponding to the audio of each frequency band and the reverberation time corresponding to the audio of each frequency band;
  • the large-screen device determines the spatial parameters based on the spatial parameters corresponding to the audio in multiple frequency bands.
  • embodiments of the present application provide a sound field calibration method, which is applied to recording equipment.
  • the method includes:
  • the recording device receives recording instructions from the large-screen device
  • the recording device responds to the recording instruction, records audio, and obtains the recording audio; the recording audio includes the audio recorded by the recording device during the large-screen device driver's playback of the preset audio; the recording audio is used to determine the calibration parameters; the calibration parameters are used to calibrate the content to be played Audio; large-screen devices are used to drive and play the calibrated audio to be played.
  • the preset audio includes calibration audio
  • the recorded audio includes recorded calibration audio
  • the recorded calibration audio is used to calibrate the audio to be played.
  • the preset audio includes instrumented audio
  • the recorded audio includes recorded instrumented audio
  • the recorded instrumented audio is used to calibrate the audio to be played.
  • the preset audio includes noise audio
  • the recorded audio includes recorded noise audio
  • the recorded noise audio is used to calibrate the audio to be played.
  • a sound field calibration device which includes:
  • the playback unit is used to drive the playback of preset audio in response to user instructions
  • the communication unit is used to send recording instructions to the recording equipment; the recording instructions are used to instruct the recording equipment to record and obtain the recording audio of the preset audio; the recording audio includes the audio recorded by the recording equipment during the playback unit of playing the preset audio;
  • the communication unit is used to receive the recording audio sent by the recording device
  • the determination unit is used to determine the calibration parameters based on the recorded audio; the calibration parameters are used to calibrate the audio to be played;
  • the playback unit is used to drive and play the calibrated audio to be played.
  • the playback unit is configured to: play the preset audio based on the metadata of the preset audio;
  • the playback unit is used for: playing the calibrated audio to be played based on the metadata of the audio to be played.
  • both the preset audio and the audio to be played include audio of multiple channels
  • the metadata of the preset audio and the metadata of the audio to be played both include multiple channels and The mapping relationship of at least one sound source; the multiple channels include the first channel and the second channel;
  • a playback unit configured to play the audio of the first channel of the preset audio based on the sound source corresponding to the first channel in the metadata of the preset audio;
  • the playback unit is configured to play the audio of the second channel of the preset audio based on the sound source corresponding to the second channel in the metadata of the audio to be played.
  • both the preset audio and the audio to be played include audio of multiple channels
  • the metadata of the preset audio and the audio to be played includes the loudness and corresponding values of the multiple channels respectively.
  • Gain Multiple channels including third and fourth channels;
  • the loudness of the audio of the third channel in the preset audio played by the playback unit is obtained by adjusting the loudness corresponding to the third channel in the metadata of the preset audio through the gain corresponding to the third channel in the metadata of the preset audio;
  • the loudness of the fourth channel audio in the calibrated audio to be played by the playback unit is adjusted by adjusting the gain and calibration parameters corresponding to the fourth channel in the metadata of the audio to be played and the corresponding third channel in the metadata of the preset audio. The loudness obtained.
  • the preset audio includes calibration audio
  • the recorded audio includes recorded calibration audio
  • the recorded calibration audio is used to calibrate the audio to be played.
  • the preset audio includes instrumented audio
  • the recorded audio includes recorded instrumented audio
  • the recorded instrumented audio is used to calibrate the audio to be played.
  • the preset audio includes noise audio
  • the recorded audio includes recorded noise audio
  • the recorded noise audio is used to calibrate the audio to be played.
  • the calibration parameters are obtained based on spatial parameters, and the spatial parameters are used to indicate the audio characteristics of the space where the preset audio is played.
  • the recorded audio includes recorded calibration audio
  • the determining unit is configured to calculate spatial parameters based on the reverberation time and the recorded calibration audio.
  • the determination unit is configured to calculate the sweet spot frequency response energy of the recorded calibration audio in the user's room based on the reverberation time and spatial parameters;
  • Calibration parameters are determined based on the difference between the sweet spot frequency response energy and the target frequency response energy.
  • each frame of the preset audio includes noise audio
  • the preset audio includes instrumented audio
  • the recorded audio includes recorded noise audio and recorded instrumented audio
  • the recording device is located at the position where the user uses the sound field calibration device.
  • a sound field calibration device which includes:
  • Communication unit used to establish communication connections with large-screen devices
  • Communication unit used to receive recording instructions from large-screen devices
  • the recording unit is used to record audio in response to the recording instruction to obtain the recorded audio; the recorded audio includes the audio recorded by the sound field calibration device during the process of playing the preset audio by the large-screen device driver; the recorded audio is used to determine the calibration parameters; the calibration parameters are used It is used to calibrate the audio to be played; the large screen device is used to drive the calibrated audio to be played.
  • the preset audio includes calibration audio
  • the recorded audio includes recorded calibration audio
  • the recorded calibration audio is used to calibrate the audio to be played.
  • the preset audio includes instrumented audio
  • the recorded audio includes recorded instrumented audio
  • the recorded instrumented audio is used to calibrate the audio to be played.
  • the preset audio includes noise audio
  • the recorded audio includes recorded noise audio
  • the recorded noise audio is used to calibrate the audio to be played.
  • inventions of the present application provide a sound field calibration system.
  • the system includes a large screen device and a recording device.
  • the large screen device is used to perform the above first aspect or any possible implementation of the first aspect.
  • the sound field calibration method in the recording device is used to perform the sound field calibration method in the above second aspect or any possible implementation of the second aspect.
  • embodiments of the present application provide an electronic device, including one or more functional modules.
  • the one or more functional modules can be used to perform any of the above aspects or any possible implementation of any aspect. Sound field calibration method in .
  • the present application provides a computer storage medium, including computer instructions.
  • the communication device causes the communication device to perform any of the above aspects or any possible implementation of any aspect. Sound field calibration method.
  • the present application provides a computer program product.
  • the computer program product When the computer program product is run on a computer, it causes the computer to execute the sound field calibration method in any of the above aspects or any possible implementation of any aspect.
  • this application provides a chip, including: a processor and an interface.
  • the processor and the interface cooperate with each other so that the chip executes any of the above aspects or any possible implementation of any aspect. sound field calibration method.
  • the electronic device provided by the sixth aspect the computer-readable storage medium provided by the seventh aspect, the computer program product provided by the eighth aspect, and the chip provided by the ninth aspect are all used to execute the tasks provided by the embodiments of the present application. method. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects in the corresponding methods, and will not be described again here.
  • Figure 1 is a schematic diagram of a sound field calibration system provided by an embodiment of the present application.
  • Figure 2 is a schematic diagram of the hardware structure of the electronic device 100 provided by the embodiment of the present application.
  • Figure 3 is a software structure block diagram of an electronic device 100 provided by an embodiment of the present application.
  • Figure 4 is a schematic diagram of the hardware structure of the second device 20 provided by this embodiment of the present application.
  • Figure 5 is a schematic diagram of a home scene provided by an embodiment of the present application.
  • Figure 6 is a schematic diagram of software interaction between a smart screen and a remote control provided by an embodiment of the present application
  • Figure 7A is a flow chart of a sound field calibration method provided by an embodiment of the present application.
  • Figure 7B is a flow chart of another sound field calibration method provided by an embodiment of the present application.
  • 8A to 8C are some interface diagrams illustrating the sound field calibration process according to the embodiment of the present application.
  • Figure 9 is a schematic diagram of a preset audio provided in an exemplary embodiment of the present application.
  • Figure 10 is a schematic diagram of recorded audio corresponding to a preset audio provided by an embodiment of the present application.
  • Figure 11 is a schematic flow chart of a smart screen playing audio to be played provided by this application.
  • Reverberation After the indoor sound source stops emitting sound, the sound wave continues to be reflected or scattered multiple times due to room boundaries or obstacles. Sound that reaches the receiving point directly from the sound source is called direct sound, and sound that reaches the receiving point after one or more reflections from objects or walls is called reverberated sound.
  • Reverberation time is generally used to describe the degree of attenuation of indoor sound. That is to say, reverberation is the sound "residual" phenomenon caused by late reflections from various interfaces that still exist in the sound field after the sound source stops emitting sound. The length of this residual phenomenon can be characterized by reverberation time.
  • Reverberation time refers to the time when indoor sound reaches a stable state. After the sound source stops emitting sound, the residual sound is repeatedly reflected in the room and absorbed by the sound-absorbing material. Its average sound energy density decays from the original value to one millionth (or sound energy density). time required to reduce 60dB).
  • the reverberation time of a room is determined by its sound absorption and volume. A room with a large volume and small sound absorption usually has a long reverberation time, while a room with strong sound absorption and a small volume usually has a long reverberation time. , usually the reverberation time is short. Commonly used formulas for calculating reverberation time include Sabine equation and Airen equation.
  • PCM Pulse Code Modulation
  • Pulse code modulation is to transform an analog signal that is continuous in time and has continuous values into a digital signal that is discrete in time and has discrete values.
  • Pulse code modulation is a process in which the analog signal is first sampled, and then the sample amplitude is quantized and encoded. Among them, sampling is to periodically scan the analog signal to turn the time-continuous signal into a time-discrete signal. Sampling must follow the Nyquist sampling theorem.
  • Quantization means discretizing the amplitude of the instantaneous value obtained by sampling, that is, using a set of prescribed levels, and representing the instantaneous sampled value with the closest level value, usually in binary.
  • a quantized pulse amplitude modulation signal is obtained, which only has a limited number of values. Coding is to use a set of binary codes to represent each quantized value with a fixed level. However, in fact, quantization is completed simultaneously during the encoding process, so the encoding process is also called analog/digital conversion, which can be recorded as A/D.
  • Frequency response refers to the phenomenon that when an audio signal with a constant voltage output is connected to an audio playback device, the sound pressure generated by the audio playback device increases or attenuates as the frequency changes, and the phase changes with the frequency. , this change relationship (change amount) related to sound pressure and phase and frequency is called frequency response, and the unit is decibel (Db).
  • frequency response can be described by a frequency response curve.
  • the ordinate of the frequency response curve represents power (unit is decibel), and the abscissa represents frequency (unit is Hz).
  • the scale of the abscissa can be logarithmic. , the ordinate is linear.
  • the audio playback device can make the actual frequency response of the played audio in the user space an ideal frequency response, then the actual sound field of the audio is the ideal sound field, and the optimal user hearing effect can be achieved.
  • the actual frequency response of the audio in the user space due to factors such as environmental interference and noise effects in different user spaces, there is often a certain difference between the actual frequency response of the audio in the user space and the ideal frequency response, resulting in poor actual sound field effects and poor user listening experience.
  • the audio playback device (such as a large-screen device) can calculate the sweet spot frequency response energy based on spatial parameters, where the sweet point refers to the position with the best sense of hearing; through the sweet point frequency response energy and The target frequency response energy determines the calibration parameters, and the audio to be played is processed through the calibration parameters (ie, audio equalization) to achieve the ideal sound field effect.
  • audio equalization is to enhance or weaken components in certain frequency bands of the audio signal to make the audio signal more pleasant to the ear or closer to people's listening habits. For example, if the listener wants to increase the experience of the guitar, increasing the energy in the 100-300Hz frequency band through the audio equalization process can allow the user to hear a fuller tone.
  • the frequency response curve of an audio playback device is flat, that is, the audio signals of different frequencies played by the audio playback device have the same loudness.
  • the frequency response curve of the recorded audio signal It may not be straight.
  • Audio equalization can process the recorded audio signal based on calibration parameters to reduce the impact of environmental and other factors on the audio signal. For example, environmental factors cause the loudness of the audio signal in frequency band N heard by the user to increase by ndb compared with the initial loudness of the audio played by the audio playback device.
  • the initial loudness is the ideal loudness, where n is a positive number, then the audio playback device can adjust frequency band N
  • the audio signal is processed so that the loudness of the audio signal in frequency band N played by the audio playback device is ndb lower than the initial loudness, so that the loudness of the audio signal in frequency band N heard by the user is the initial loudness, thereby achieving audio balance and reducing environmental factors to be played back Audio effects.
  • the audio playback device performs equalization processing on the audio to be played through calibration parameters, so that the effect of the audio signal heard by the user is closer to the desired audio effect.
  • the system may include: a first device 10 and a second device 20.
  • the first device 10 and the second device 20 may establish a communication connection and communicate based on the communication connection.
  • the first device 10 can send a recording instruction to the second device 20, and the second device 20 starts recording after receiving the recording instruction, and sends the recorded audio to the first device 10 after the recording ends.
  • the first device 10 is an electronic device with the function of playing audio, such as a large screen device and a speaker, etc.
  • the second device 20 is a device with a recording function, such as a remote control or a mobile phone.
  • the first device 10 and the second device 20 may be the following electronic devices.
  • the electronic devices include but are not limited to large-screen devices (such as smart screens), smart phones, tablets, personal digital assistants (PDAs), wireless devices, etc.
  • Wearable electronic devices with communication functions such as smart watches, smart glasses), augmented reality (AR) devices, virtual reality (VR) devices, etc.
  • Exemplary embodiments of electronic devices include, but are not limited to, equipped with Portable electronic devices with Linux or other operating systems.
  • the above-mentioned electronic device can also be other portable electronic devices, such as a laptop computer (Laptop). It should also be understood that in some other embodiments, the above-mentioned electronic device may not be a portable electronic device, but a desktop computer or the like.
  • the communication connection between the first device 10 and the second device 20 may be a wired connection or a wireless connection.
  • the wireless connection can be a high-fidelity wireless communication (wireless fidelity, Wi-Fi) connection, Bluetooth connection, infrared connection, Near Field Communication (NFC) connection, ZigBee connection and other short-range connections.
  • the first device 10 and the second device 20 can log in to the same user account (such as a Huawei account) and then connect through the server.
  • the sound field calibration system may also include one or more peripheral devices connected to the smart screen.
  • the peripheral devices may be speakers or other electronic devices.
  • the smart screen can control peripheral devices to play audio.
  • FIG. 2 shows a schematic diagram of the hardware structure of the electronic device 100.
  • electronic devices such as smart screens, mobile phones, and remote controls may have all or part of the structure of the electronic device 100 shown in FIG. 2 .
  • electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have a different component configuration.
  • the various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
  • the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2.
  • Mobile communication module 150 wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194 and Subscriber identification module (SIM) card interface 195, etc.
  • SIM Subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figures, or some components may be combined, some components may be separated, or some components may be arranged differently.
  • the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), Baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • Baseband processor Baseband processor
  • neural network processor neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate operation control signals based on the instruction operation code and timing signals to complete the control of fetching and executing instructions.
  • the processor 110 may also be provided with a memory for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have been recently used or recycled by processor 110 . If the processor 110 needs to use the instructions or data again, it can be called directly from the memory. Repeated access is avoided and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.
  • processor 110 may include one or more interfaces.
  • Interfaces may include integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiver and transmitter (universal asynchronous receiver/transmitter (UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and /or universal serial bus (USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • UART universal asynchronous receiver and transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (derail clock line, SCL).
  • processor 110 may include multiple sets of I2C buses.
  • the processor 110 can separately couple the touch sensor 180K, charger, flash, camera 193, etc. through different I2C bus interfaces.
  • the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to implement the touch function of the electronic device 100 .
  • the I2S interface can be used for audio communication.
  • processor 110 may include multiple sets of I2S buses.
  • the processor 110 can be coupled with the audio module 170 through the I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface to implement the function of answering calls through a Bluetooth headset.
  • the PCM interface can also be used for audio communications to sample, quantize and encode analog signals.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to implement the function of answering calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is generally used to connect the processor 110 and the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc.
  • the processor 110 and the camera 193 communicate through the CSI interface to implement the shooting function of the electronic device 100 .
  • the processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100 .
  • the GPIO interface can be configured through software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface can be used to connect the processor 110 with the camera 193, display screen 194, wireless communication module 160, audio module 170, sensor module 180, etc.
  • the GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the SIM interface can be used to communicate with the SIM card interface 195 to implement the function of transmitting data to the SIM card or reading data in the SIM card.
  • the USB interface 130 is an interface that complies with the USB standard specification, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through them. This interface can also be used to connect other electronic devices, such as AR devices, etc.
  • the interface connection relationships between the modules illustrated in the embodiments of the present application are only schematic illustrations and do not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger can be a wireless charger or a wired charger.
  • the power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110.
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, wireless communication module 160, etc.
  • the wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: Antenna 1 can be reused as a diversity antenna for a wireless LAN. In other embodiments, antennas may be used in conjunction with tuning switches.
  • the mobile communication module 150 can provide solutions for wireless communication including 2G/3G/4G/5G applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc.
  • the mobile communication module 150 can receive electromagnetic waves through the antenna 1, perform filtering, amplification and other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves through the antenna 1 for radiation.
  • at least part of the functional modules of the mobile communication module 150 may be disposed in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be provided in the same device.
  • a modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal.
  • the demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194.
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent of the processor 110 and may be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (bluetooth, BT), and global navigation satellites.
  • WLAN wireless local area networks
  • System global navigation satellite system, GNSS
  • frequency modulation frequency modulation, FM
  • near field communication technology near field communication, NFC
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi) -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is an image processing microprocessor and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • the display screen 194 is used to display images, videos, etc.
  • Display 194 includes a display panel.
  • the display panel can use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed, quantum dot light emitting diode (QLED), etc.
  • the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
  • the electronic device 100 may be implemented by an ISP, a camera 193, a video codec, a GPU, a display 194, an application processor, etc. Shooting function.
  • the ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera sensor through the lens, the optical signal is converted into an electrical signal, and the camera sensor passes the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193.
  • Camera 193 is used to capture still images or video.
  • the object passes through the lens to produce an optical image that is projected onto the photosensitive element.
  • the photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to convert it into a digital image signal.
  • ISP outputs digital image signals to DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other format image signals.
  • the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
  • Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
  • Video codecs are used to compress or decompress digital video.
  • Electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
  • MPEG moving picture experts group
  • MPEG2 MPEG2, MPEG3, MPEG4, etc.
  • NPU is a neural network (NN) computing processor.
  • NN neural network
  • Intelligent cognitive applications of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, etc.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. Such as saving music, videos, etc. files in external memory card.
  • Internal memory 121 may be used to store computer executable program code, which includes instructions.
  • the processor 110 executes instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 100 .
  • the internal memory 121 may include a program storage area and a data storage area.
  • the stored program area can store the operating system, at least one application required for the function (such as face recognition function, fingerprint recognition function, mobile payment function, etc.).
  • the storage data area can store data created during the use of the electronic device 100 (such as face information template data, fingerprint information templates, etc.).
  • the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
  • the electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playback, recording, etc.
  • the audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .
  • Speaker 170A also called “speaker” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to hands-free calls.
  • Receiver 170B also called “earpiece” is used to convert audio electrical signals into sound signals.
  • the electronic device 100 answers a call or a voice message, the voice can be heard by bringing the receiver 170B close to the human ear.
  • Microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 170C with the human mouth and input the sound signal to the microphone 170C.
  • the electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which in addition to collecting sound signals, may also implement a noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions, etc.
  • the headphone interface 170D is used to connect wired headphones.
  • the headphone interface 170D may be a USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA Cellular Telecommunications Industry Association of the USA
  • the pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals.
  • pressure sensor 180A may be disposed on display screen 194 .
  • pressure sensors 180A such as resistive pressure sensors, inductive pressure sensors, Capacitive pressure sensor, etc.
  • a capacitive pressure sensor may include at least two parallel plates of conductive material.
  • touch operations acting on the same touch location but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold is applied to the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.
  • the gyro sensor 180B may be used to determine the motion posture of the electronic device 100 .
  • the angular velocity of electronic device 100 about three axes ie, x, y, and z axes
  • the gyro sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyro sensor 180B detects the angle at which the electronic device 100 shakes, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shake of the electronic device 100 through reverse movement to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenes.
  • Air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
  • Magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 may utilize the magnetic sensor 180D to detect opening and closing of the flip holster.
  • the electronic device 100 may detect the opening and closing of the flip according to the magnetic sensor 180D. Then, based on the detected opening and closing status of the leather case or the opening and closing status of the flip cover, features such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices and be used in horizontal and vertical screen switching, pedometer and other applications.
  • Distance sensor 180F for measuring distance.
  • Electronic device 100 can measure distance via infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may utilize the distance sensor 180F to measure distance to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode.
  • the light emitting diode may be an infrared light emitting diode.
  • the electronic device 100 emits infrared light outwardly through the light emitting diode.
  • Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
  • the electronic device 100 can use the proximity light sensor 180G to detect when the user holds the electronic device 100 close to the ear for talking, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in holster mode, and pocket mode automatically unlocks and locks the screen.
  • the ambient light sensor 180L is used to sense ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent accidental touching.
  • Fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access to application locks, fingerprint photography, fingerprint answering of incoming calls, etc.
  • Temperature sensor 180J is used to detect temperature.
  • the electronic device 100 utilizes the temperature detected by the temperature sensor 180J to execute the temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the low temperature from causing the electronic device 100 to shut down abnormally. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • Touch sensor 180K also called “touch panel”.
  • the touch sensor 180K can be disposed on the display screen 194.
  • the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near the touch sensor 180K.
  • the touch sensor can pass the detected touch operation to the application processor to determine the touch event type.
  • Visual output related to the touch operation may be provided through display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a location different from that of the display screen 194 .
  • the buttons 190 include a power button, a volume button, etc.
  • Key 190 may be a mechanical key. It can also be a touch button.
  • the electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
  • the motor 191 can generate vibration prompts.
  • the motor 191 can be used for vibration prompts for incoming calls and can also be used for touch vibration feedback.
  • touch operations for different applications can correspond to different vibration feedback effects.
  • the motor 191 can also respond to different vibration feedback effects for touch operations in different areas of the display screen 194 .
  • Different application scenarios for example: time reminder, Receiving messages, alarm clocks, games, etc.
  • the touch vibration feedback effect can also be customized.
  • the indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, or may be used to synthesize requests, missed calls, notifications, etc.
  • the SIM card interface 195 is used to connect a SIM card.
  • the SIM card can be connected to or separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 .
  • the electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different.
  • the SIM card interface 195 is also compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.
  • the electronic device 100 may be the first device 10.
  • the first device 10 may convert digital audio information into analog audio signals and output them through the audio module 170, such as parsing a preset audio file, through one or more speakers 170A. Play preset audio.
  • the first device can also establish a communication connection with other audio playback devices (such as speakers) through the wireless communication module, and play preset audio or audio to be played through other audio playback devices.
  • FIG. 3 is a software structure block diagram of the electronic device 100 provided by the embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has clear roles and division of labor.
  • the layers communicate through software interfaces.
  • the system is divided into four layers, from top to bottom: application layer, application framework layer, runtime and system library, and kernel layer.
  • the application layer can include a series of application packages.
  • the application package can include a sound field calibration application, camera, gallery, calendar, call, map, WLAN, music, video and other applications (also called applications).
  • the application framework layer provides an application programming interface (API) and programming framework for applications in the application layer.
  • API application programming interface
  • the application framework layer includes some predefined functions.
  • the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, music manager, etc.
  • the audio manager is used to play audio.
  • the audio manager can obtain audio data.
  • a window manager is used to manage window programs.
  • the window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.
  • Content providers are used to store and retrieve data and make this data accessible to applications.
  • Said data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
  • the view system includes visual controls, such as controls that display text, controls that display pictures, etc.
  • a view system can be used to build applications.
  • the display interface can be composed of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide communication functions of the electronic device 100 .
  • call status management including connected, hung up, etc.
  • the resource manager provides various resources to applications, such as localized strings, icons, pictures, layout files, video files, etc.
  • the notification manager allows applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also be notifications that appear in the status bar at the top of the system in the form of charts or scroll bar text, such as notifications from applications running in the background, or notifications that appear on the screen in the form of a conversation interface. For example, text information is prompted in the status bar, a beep sounds, the electronic device vibrates, the indicator light flashes, etc.
  • Runtime includes core libraries and virtual machines. Runtime is responsible for the scheduling and management of the system.
  • the core library contains two parts: one part is the functional functions that the programming language (for example, Java language) needs to call, and the other part is the core library of the system.
  • one part is the functional functions that the programming language (for example, Java language) needs to call
  • the other part is the core library of the system.
  • System libraries can include multiple functional modules. For example: surface manager (surface manager), media libraries (Media Libraries), three-dimensional graphics processing libraries (for example: OpenGL ES), two-dimensional graphics engines (for example: SGL), etc. Also includes AR engine.
  • the AR engine is used to implement augmented reality functions such as two-dimensional/three-dimensional coordinate conversion and motion tracking.
  • the surface manager is used to manage the display subsystem and provides the fusion of two-dimensional (2-Dimensional, 2D) and three-dimensional (3-Dimensional, 3D) layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, composition and layer processing, etc.
  • 2D Graphics Engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer at least includes display driver, camera driver, audio driver, sensor driver, and virtual card driver.
  • the following exemplifies the workflow of the software and hardware of the electronic device 100 in conjunction with the audio playback scenario.
  • the corresponding hardware interrupt is sent to the kernel layer.
  • the kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, and other information). Raw input events are stored at the kernel level.
  • the application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation and the control corresponding to the click operation as the sound field calibration application icon control as an example, the sound field calibration application calls the interface of the application framework layer to start the sound field calibration application, and then starts the earpiece by calling the kernel layer. Drive to play the preset audio through the speaker 170A.
  • the embodiment of this application uses the electronic device 200 as an example to introduce the hardware structure and software structure of the second device 20 .
  • the second device 20 may include: a processor 222 , a memory 223 , a wireless communication module 224 , a power switch 225 , a display screen 229 , and an audio module 230 .
  • the second device 20 may also include a wired LAN communication processing module 226, a high definition multimedia interface (HDMI) communication processing module 227, a USB communication processing module 228, and the like.
  • HDMI high definition multimedia interface
  • Processor 222 may be used to read and execute computer-readable instructions.
  • the processor 222 may mainly include a controller, arithmetic unit, and a register.
  • the controller is mainly responsible for decoding instructions and issuing control signals for operations corresponding to the instructions.
  • the arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations. It can also perform address operations and conversions.
  • Registers are mainly responsible for storing register operands and intermediate operation results temporarily stored during instruction execution.
  • the hardware architecture of the processor 222 may be an application specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture or an NP architecture, etc.
  • ASIC application specific integrated circuit
  • Wireless communication module 224 may include a WLAN communication processing module.
  • the wireless communication module 224 may also include a Bluetooth (BT) communication processing module, an NFC processing module, a cellular mobile communication processing module (not shown), and the like.
  • BT Bluetooth
  • NFC NFC
  • cellular mobile communication processing module not shown
  • the wireless communication module 224 may be used to establish a communication connection with the first device 10 .
  • the WLAN communication processing module can be used to establish a Wi-Fi direct communication connection with the first device 10
  • the Bluetooth (BT) communication processing module can be used to establish a Bluetooth communication connection with the first device 10
  • the NFC processing module can be used to establish a Bluetooth communication connection with the first device 10. 10 Establish NFC connection and more.
  • the wireless communication module 224 can also be used to establish a communication connection with the first device 10 and receive the recording instruction sent by the first device 10 based on the communication connection.
  • the communication connection established between the wireless communication module 224 and the first device 10 can transmit data based on the HTTP protocol. This application does not impose any restrictions on the type of communication connection and data transmission protocol between devices.
  • Memory 223 is coupled to processor 222 for storing various software programs and/or sets of instructions.
  • the memory 223 may include high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices.
  • the memory 223 can store operating systems, such as uCOS, VxWorks, RTLinux and other embedded operating systems.
  • the memory 223 may also store a communication program that may be used to communicate with the first device 10, one or more servers, or additional devices.
  • the power switch 225 may be used to control power supply to the second device 20 .
  • the wired LAN communication processing module 226 can be used to communicate with other devices in the same LAN through the wired LAN, and can also be used to connect to the WAN through the wired LAN and communicate with devices in the WAN.
  • the HDMI communication processing module 227 may be used to communicate with other devices through an HDMI interface (not shown).
  • USB communication processing module 228 may be used to communicate with other devices through a USB interface (not shown).
  • the display screen 229 can be used to display interfaces and the like.
  • the display screen 229 can be an LCD, OLED, AMOLED, FLED, QLED or other display screen.
  • Audio module 230 can be used to output audio signals through the audio output interface, so that the second device 20 can support audio playback.
  • the audio module 230 may also be used to receive audio data through the audio input interface.
  • the audio module 230 may include, but is not limited to, a microphone, a speaker, a receiver, and the like.
  • the second device 20 can collect sound signals through a microphone, such as when the first device 10 plays preset audio. Collect the sound signals to get the recorded audio.
  • the second device 20 may also include a serial interface such as an RS-232 interface.
  • This serial interface can be connected to other devices, such as speakers and other audio external devices, so that the display and audio external devices can cooperate to play audio and video.
  • the structure illustrated in FIG. 3 does not constitute a limitation on the only implementation manner of the second device 20 .
  • the second device 20 may include more or less components than shown in the figures, or combine some components, or split some components, or arrange different components.
  • the components illustrated may be implemented in hardware, software, or a combination of software and hardware.
  • the second device 20 may include the hardware included in the first device 10 shown in FIG. 2 described above.
  • the various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.
  • the software system of the second device 20 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture, etc.
  • the software system of the second device 20 may include but is not limited to Linux or other operating systems. For Huawei’s Hongmeng system.
  • the second device 20 is an Android system, which is divided into four layers. From top to bottom, they are the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.
  • the application layer can include screen projection management applications for device connection and screen display
  • the application framework layer can include cross-device connection managers, event managers, window managers and display managers, etc.
  • system libraries can include Media library and event data, etc.
  • the kernel layer is used for device discovery, device authentication and device connection, etc. For details of each part, please refer to the relevant descriptions in Figure 3, and will not be repeated here.
  • the sound field calibration method provided by the embodiment of the present application is introduced in detail below.
  • the sound field calibration method provided by the embodiment of the present application can be applied to a variety of scenes, such as home scenes, concert halls, conference rooms, etc., and is not limited here.
  • Figure 5 illustrates a possible home scenario.
  • the smart screen in Figure 5 is the first device 10 in Figure 1
  • the remote control in Figure 5 is the second device 20 in Figure 1 .
  • the home scene includes smart screens, remote controls, sofas and users.
  • the screen of the smart screen is facing the user on the sofa.
  • the user can sit on the sofa in the living room to watch the smart screen, and hold the remote control to control the smart screen to play audio or video.
  • the user can sit on the sofa, hold the remote control in hand, and input the user operation to start the sound field calibration on the remote control.
  • the remote control sends the user instruction instructing the sound field calibration to the smart screen;
  • the smart screen sends a recording command to the remote control and plays the preset audio; after receiving the recording command, the remote control starts recording the preset audio, and then sends the recorded recording audio to the smart screen; the remote control Send the recorded audio to the smart screen;
  • the smart screen obtains calibration parameters based on the recorded audio, which are used to process the audio so that the actual sound field of the audio is close to the target sound field of the audio; the smart screen can save the calibration parameters for subsequent playback For other audio, use the calibration parameters to process the audio and play the processed audio.
  • the remote control can be placed at the position of the sofa during the sound field calibration process. It is understandable that The auditory effect of the recorded audio recorded by the remote control at this position is close to the user's auditory effect.
  • Figure 6 exemplarily shows a schematic diagram of software interaction between the smart screen and the remote control.
  • a sound field calibration application can be installed on the smart screen; when the smart screen receives a user instruction to start sound field calibration, the sound field calibration application sends commands to the audio track interface (AudioTrack) and recording interface (AudioRecorder); AudioTrack Call the audio hardware abstraction layer (AudioHal) of the smart screen, and drive the speakers or speakers of the smart screen to play preset audio through the sound architecture (Advanced Linux Sound Architecture, Alsa); the AudioRecorder downloads the audio hardware abstraction layer (AudioHal) of the remote control. Send a command to drive the remote control to start recording through the MIC of the remote control. Furthermore, after the recording is completed, the remote control sends the recorded recording file to the smart screen via Bluetooth.
  • AudioTrack Audio track interface
  • AudioRecorder AudioTrack Call the audio hardware abstraction layer (AudioHal) of the smart screen, and drive the speakers or speakers of the smart screen to play preset audio through the sound architecture (Advanced Linux Sound Architecture, Alsa)
  • the smart screen inputs the recording file into the calibration calculation module.
  • the calibration calculation module is used to process the recording file with PCM to obtain the recorded audio and calculate the calibration parameters based on the recorded audio.
  • the smart screen sends the calculated calibration parameters to the sound field calibration application.
  • the sound field calibration application in Figure 6 is the sound field calibration application in the application layer shown in Figure 3; the music manager in the application software framework layer shown in Figure 3 can include AudioTrack, AudioRecorder and AudioHal; the audio driver in the kernel layer shown in Figure 3 may include Alsa shown in Figure 6.
  • Figure 7A shows a flow chart of a sound field calibration method. As shown in Figure 7A, the method may include the following steps:
  • S601 The large-screen device establishes a communication connection with the recording device.
  • the communication connection established between the large screen device and the recording device may be the aforementioned wired connection or wireless connection.
  • the large-screen device can be a smart screen
  • the recording device can be a remote control.
  • S602 The large-screen device responds to user instructions and drives to play preset audio.
  • the preset audio may include calibration audio.
  • the preset audio may also include at least one of noise audio and instrumentation audio.
  • noise audio For specific content of preset audio, please refer to the detailed description below.
  • the large-screen device can play preset audio through one or more sound sources.
  • the sound source may be a sound source on a large screen device, such as a speaker, or a playback device external to the large screen device, such as a speaker.
  • large-screen devices can also play preset audio based on the metadata of the preset audio.
  • the metadata is audio-related data in addition to audio information.
  • the metadata may include mapping relationships between multiple sound channels and at least one sound source and corresponding loudness and gain of the multiple sound channels. Metadata is used to process audio or playback settings, which can improve the audio playback effect.
  • S603 The large-screen device sends a recording instruction to the recording device.
  • the recording instruction is used to instruct the large-screen device to record.
  • the recording device performs recording in response to the recording instruction, and obtains the recorded audio of the preset audio; the recorded audio includes the audio recorded by the recording device during the process of playing the preset audio on the large-screen device.
  • the preset audio includes calibration audio, noise audio, and instrumented audio
  • the recorded audio includes recorded calibration audio, recorded noise audio, and recorded instrumented audio
  • the recording device sends recording audio to the large-screen device.
  • the recording device may send the recorded audio to the large-screen device based on the above communication connection after completing the recording.
  • the large-screen device determines the calibration parameters based on the recorded audio; the calibration parameters are used to calibrate the audio to be played.
  • steps S706 to S709 exemplarily illustrate a method for a large-screen device to determine calibration audio based on recorded audio.
  • steps S706 to S709 exemplarily illustrate a method for a large-screen device to determine calibration audio based on recorded audio.
  • S607 The large-screen device driver plays the calibrated audio to be played.
  • the large-screen device when receiving a play instruction for the audio to be played, calibrates the audio to be played based on the calibration parameters; and then plays the calibrated audio to be played.
  • the large-screen device can also play the calibrated audio to be played based on the metadata of the audio to be played.
  • the large-screen device can play the calibrated audio to be played through one or more sound sources.
  • the sound source for the large-screen device to play the preset audio and the sound source for the large-screen device to play the audio to be played can be the same sound source, or they can be different sound sources; the sound source for the large-screen device to play the preset audio is the same as the sound source for the large-screen device to play the audio to be played.
  • the number of sound sources for large-screen devices to play audio to be played can be the same or different.
  • FIG. 7B shows a flow chart of another sound field calibration method.
  • the smart screen and remote control are located in the user's room.
  • the method may include the following steps:
  • S701 The smart screen establishes a communication connection with the remote control.
  • the communication connection established between the smart screen and the remote control can be the aforementioned wired connection or wireless connection.
  • wireless connection For example, Bluetooth communication.
  • the smart screen when it detects a user operation indicating calibration of the sound field, it may send a recording instruction to the remote control in response to the user operation.
  • the smart screen can receive user instructions instructing to calibrate the sound field.
  • the smart screen can display a sound field calibration application. Accordingly, the user can click on the sound field calibration application on the smart screen; furthermore, the smart screen detects that the sound field calibration application is When calibrating the touch operation of the application, a recording instruction is sent to the remote control in response to the touch operation.
  • the remote control can receive a user instruction instructing to calibrate the sound field.
  • the user can touch a button on the remote control.
  • the remote control detects a touch operation on the button, it responds to the touch operation.
  • Touch operation to send a user command to start calibrating the sound field to the remote control; then, the smart screen responds to the user command and sends a recording command to the remote control.
  • the smart screen plays preset audio, which includes instrumentation audio and calibration audio.
  • the instrumented (flag) audio is used to calculate the reverberation time; the calibration audio includes audio of multiple frequency bands and is used to obtain the calibration parameters of each frequency band.
  • the instrumented audio may be any one of audio in a specific format, audio with a specific gain value, audio with a specific audio signal amplitude, and audio with a specific loudness.
  • the preset audio may also include noise audio, which is used to determine the reverberation decay end node of the instrumented audio.
  • each frame of the preset audio may include noise audio.
  • the noise audio is used to determine the reverberation decay end node of the instrumented audio, and the reverberation decay end node is used to calculate the reverberation time;
  • the instrumented audio is used to calculate the reverberation time, and the reverberation time is used to calculate the spatial parameters;
  • Calibration audio is used to obtain calibration parameters based on spatial parameters.
  • playing the noise audio first is to avoid the recording time of the remote control being later than the playback time of the smart screen and missing the recording of the instrumented audio; if the noise audio is still played after the instrumented audio is played, the smart screen can identify it in the recorded audio. Determine the end of the reverberation attenuation of the instrumented audio after recording the noise audio, improving the accuracy of judging the end node of the reverberation attenuation.
  • each frame of the preset audio may include noise audio.
  • noise audio is included between the calibration audio and the instrumented audio to avoid the reverberation sound of the calibration audio from affecting the judgment of the reverb attenuation start node of the instrumented audio; the noise audio is set when starting to play the noise audio and after the instrumented audio.
  • the smart screen when it detects a user operation instructing to start calibrating the sound field, in response to the user operation, it first sends a recording instruction to the remote control; after the recording instruction is sent, it plays the preset audio after a certain period of time.
  • the smart screen sends a recording command to the remote control at 9:00, and the smart screen can start playing the preset audio at 9:02.
  • the smart screen can also start playing the preset audio at the moment it starts sending recording instructions to the remote control.
  • the smart screen sends a recording instruction to the remote control at 9:00 and starts playing the preset audio at 9:00.
  • the transmission time of the recording command is generally very short, even if the smart screen starts playing the preset audio when sending the recording command to the remote control, as long as the remote control starts recording when it receives the recording command, then It can be considered that the playback time of the smart screen and the recording time of the remote control are the same time, that is, the remote control can record the preset audio played by the smart screen.
  • the user controls the smart screen through the remote controller, that is, the remote controller sends user instructions to the smart screen in response to user operations, and the smart screen is installed with a sound field calibration application; then the smart screen can respond to the control instructions sent by the remote controller.
  • displaying the user interface 81 as shown in Figure 8A which displays the sound field calibration application icon 811 and other application icons; when the smart screen receives the confirmation instruction for the sound field calibration application icon 811, it responds to the confirmation instruction
  • the user interface 82 shown in Figure 8B is displayed.
  • the user interface 82 displays a play control 821 and a return control 822.
  • the return control 822 is used to display the user interface 81; furthermore, when the smart screen receives a confirmation instruction for the play control 821 , in response to the confirmation command, send a recording command to the remote control and start playing the preset audio.
  • the user interface for the smart screen to play the preset audio can be shown in Figure 8C.
  • Figure 8C exemplarily shows the user interface 83 during the smart screen playing the preset audio.
  • the user interface 83 includes text prompts and a return control 831 located on the interface.
  • progress bar 832 wherein, the text prompt can be "Playing the preset audio"; the return control 831 is used to display the user interface 82; the progress bar 832 is used to display the playback progress and the duration of the preset audio, Figure 8C exemplarily shows the user interface when the preset audio with a duration of 2 minutes is played to 0:22 seconds.
  • the preset audio may also include noise audio, which is used to determine the reverberation decay end node of the instrumented audio.
  • the energy of the noise audio and the energy of the instrumented audio can differ by a preset threshold, and the preset threshold can be larger to distinguish the noise audio and the instrumented audio.
  • the noise audio can be low-frequency data such as white noise
  • the inserted audio can be low-frequency data such as white noise.
  • Pile audio can be mid- to high-frequency data.
  • the noise audio is audio data that simulates the background noise of the environment, and the noise audio may exist in each audio frame of the preset audio or in a specified audio frame.
  • FIG. 9 is a schematic diagram of a preset audio provided by an embodiment of the present application. It should be noted that FIG. 9 only illustrates monophonic audio data in the preset audio.
  • the preset audio may be multi-channel audio. For convenience of description, monoaudio will be used as an example in the following. .
  • the preset audio consists of three parts of audio: noise audio, instrumentation audio and calibration audio.
  • the black straight line represents the noise audio
  • the diagonal area represents the instrumentation audio
  • the vertical line area represents the calibration. Audio.
  • the noise audio is present in every audio frame of the preset audio, that is, the noise audio is also included in the audio frames of the instrumented audio and the calibration audio. It should be noted that due to the low energy of noise data, in the situation shown in Figure 9, the audio in which instrumented audio and noise audio overlap can also be considered as instrumented audio, and the audio in which calibration audio and noise audio overlap is calibration audio. .
  • the smart screen can also play preset audio based on the meta data of the calibration audio.
  • metadata can include multi-channel audio loudness and its gain factor, spatial location data, etc.
  • the spatial position data is the spatial mapping relationship between the multi-channel data and the sound source (Professional Audio, PA).
  • the remote control records in response to the recording instruction and obtains the recording audio; wherein the recording audio includes recording when playing the preset audio audio.
  • the remote control can start recording after receiving a recording instruction from the smart screen.
  • the remote control can record based on the instruction information. For example, if the recording instruction includes first instruction information indicating the recording time and second instruction information indicating the end time, then the remote control can start recording at the recording time indicated by the first instruction information and end the recording at the end time indicated by the second instruction information. .
  • the remote control is usually located at a location that users commonly use when using the smart screen.
  • the smart screen is placed in the living room, and the user usually sits on the sofa to use the smart screen. Therefore, the user can sit on the sofa and hold the remote control to start sound field calibration of the smart screen. It is understandable that if the remote control is located at a location that the user commonly uses when using the smart screen, the recorded audio can better reflect the user's hearing effect.
  • the remote control sends recording audio to the smart screen.
  • the recording audio includes recorded instrumentation audio and recorded calibration audio.
  • the remote control can send the recorded audio to the smart screen after the recording ends.
  • the preset audio only includes instrumented audio and calibration audio
  • the recorded audio may only include recorded instrumented audio and recorded calibration audio.
  • the recorded instrumented audio includes the main part of the instrumented audio and the residual part of the instrumented audio.
  • the main part of the instrumented audio is the audio recorded when the remote control plays noise audio on the smart screen; the residual part of the instrumented audio It is the reverberation sound of the instrumented audio recorded by the remote control after the smart screen stops playing the instrumented audio.
  • the recorded calibration audio is the audio recorded by the remote control when the smart screen plays the calibration audio.
  • the preset audio includes instrumented audio, calibration audio and noise audio
  • the recorded audio may include recorded instrumented audio, recorded calibration audio and recorded audio corresponding to the noise audio.
  • you can add noise audio to the preset audio and the recorded audio also includes the recorded audio corresponding to the noise audio.
  • the preset audio is shown in Figure 9. Each audio frame of the preset audio includes noise audio, and each frame of the recorded audio also includes the recorded audio corresponding to the noise audio.
  • S706 The smart screen determines the reverberation time based on the recorded instrumented audio.
  • the smart screen can first identify the recorded instrumented audio from the recorded audio; and then determine the reverberation time based on the recorded instrumented audio.
  • the recorded audio shown in Figure 10 can be the audio recorded by the remote control when the smart screen plays the preset audio shown in Figure 9.
  • Figure 10 exemplarily shows three arrows, which are used to indicate the recorded audio corresponding to the noise audio;
  • Figure 10 shows two rectangular boxes, from left to right, the first rectangular box and the second rectangular box, the first The audio in the rectangular box is the main part of the instrumented audio, the audio with amplitude attenuation on the right side of the first rectangular box is the residual part of the instrumented audio, and the audio in the second rectangular box is the recorded calibration audio. It can be seen that the recorded instrumentation audio is preceded and followed by the recorded audio corresponding to the noise audio.
  • the duration of the instrumented audio is shorter, and the duration of the main part of the instrumented audio is shorter. Therefore, it can be considered that the moment when the instrumented audio starts to be recorded is the starting point of the reverberation attenuation of the instrumented audio.
  • the smart screen can determine the moment when the instrumented audio starts playing from the recorded audio based on the characteristics of the instrumented audio, that is, the starting point of the reverberation attenuation, where,
  • the characteristics of the piled audio can be any one of the format, gain value, audio signal amplitude or loudness of the piled audio; furthermore, the smart screen detects the audio corresponding to the noise audio for the audio recorded after the starting point of the reverberation attenuation.
  • the audio channel corresponding to the noise audio is detected to determine the end point of the reverberation attenuation; the smart screen determines the audio from the reverberation attenuation start node to the reverberation attenuation end node as the recorded instrumented audio. It should be noted that the purpose of adding noise audio to the preset audio is to more accurately determine whether the residual sound of the instrumented audio has finished attenuating.
  • the smart screen can calculate the energy attenuation slope based on the energy of the recorded instrumented audio, which is used to represent the attenuation speed of the audio energy; and calculate the reverberation time based on the slope.
  • the reverberation time may be the time required for the energy at the reverberation attenuation starting node to attenuate by several decibels.
  • the reverberation time T 60 (f) is the time required for the energy at the reverberation attenuation starting node to attenuate 60 dB.
  • the smart screen determines the spatial parameters of the user's room based on the reverberation time and recorded calibration audio.
  • the spatial parameters are used to characterize the audio characteristics of the space.
  • the smart screen first obtains the recorded calibration audio from the recorded audio; then, based on the recorded calibration audio, determines the Mic frequency response energy Es(f) and direct sound energy Em(f) of the recorded calibration audio. ; Subtract the direct sound energy Em(f) from the Mic frequency response energy Es(f) to obtain the reverberation sound energy; then determine the spatial parameter V of the user's room based on the reverberation sound energy and reverberation time T 60 (f).
  • the reverberation sound energy is inversely proportional to the spatial parameter V of the user's room, and the reverberation sound energy is directly proportional to the reverberation time T 60 (f) of the user's room.
  • the following calculation process illustrates the relationship between the direct proportional relationship.
  • the method for the smart screen to obtain the recorded calibration audio from the recorded audio can be that the time when the smart screen plays the preset audio is the same time as the time when the remote control starts recording the audio. Then, when the smart screen receives the recorded audio, it can The recorded calibration audio is determined in the recorded audio based on the playback period of the calibration audio.
  • the frequency response energy of the calibrated audio may be mid-to-high frequency energy. It should be noted that compared to low-frequency energy, mid- and high-frequency energy excites more room modes. Broadband mid- and high-frequency input can make the steady-state reverberation energy of the room more uniform. Therefore, the embodiment of the present application uses mid- and high-frequency calibrated audio. To calculate spatial parameters, the calculated spatial parameters can be more accurate.
  • W(f), ⁇ , and c are all known numbers
  • W(f) is the radiation energy spectrum function of the whole machine
  • ⁇ is the air density
  • c is the speed of sound in the air.
  • r is the distance from the sound source to the receiving point
  • Q(f) is the directivity coefficient
  • R(f) is the room constant.
  • the reverb sound part It can be calculated from the reverberation time T 60 (f).
  • the specific formula is as follows:
  • V is the spatial parameter.
  • Dp(f) is used to represent the direct sound part, that is
  • the smart screen after the smart screen obtains the recorded audio corresponding to the calibration audio, it can calculate the spatial parameter set through formula (3).
  • the smart screen can first determine the Mic frequency response energy Es(f) and direct sound energy Em(f) based on the recorded calibration audio; then subtract Em(f) from Es(f) to obtain the reverberant sound energy; and Since the reverberation sound energy can be calculated from the reverberation time and spatial parameters, the spatial parameters can be calculated based on the reverberation time and reverberation sound energy.
  • the smart screen can substitute Es(f), Em(f) and the reverberation time T 60 (f) calculated in step S706 into formula (3) to obtain the following calculation formula:
  • the spatial parameter set can be obtained:
  • A is a constant
  • smart screens can be based on Determine the spatial parameter V of the user's room.
  • the smart screen can take the median from the spatial parameter set to obtain the spatial parameters of the user's room.
  • the smart screen determines the sweet spot frequency response energy of the calibrated audio in the user's room based on the reverberation time and spatial parameters.
  • the smart screen can calculate the sweet spot frequency response energy E 1 (f) of the calibrated audio in the user space based on the following formula. As follows:
  • Q(f) and r can use physical quantities measured in the laboratory.
  • the position of the smart screen Mic may be at a standing wave node at certain frequencies. If the Mic frequency response energy is directly used to calculate the calibration parameters, large errors will occur. Therefore, in the embodiment of this application, Mic frequency response energy and reverberation time are used, and then the spatial parameters are calculated by taking the median. The sweet spot frequency response energy is calculated based on the spatial parameters. Using the sweet spot frequency response energy to calculate the calibration parameters can reduce errors.
  • the smart screen determines the calibration parameters based on the difference between the sweet spot frequency response energy of the calibration audio in the user room and the target frequency response energy of the calibration audio.
  • the calibration parameters are used to make the sweet spot frequency response energy of the user's room equal to the target frequency response energy.
  • the smart screen can calculate the difference ⁇ SPL between the sweet spot frequency response energy E 1 (f) and the target frequency response energy E 0 (f) of the calibrated audio in the user's room through the following formula:
  • T 1 (f) is the reverberation time T 60 (f) calculated in step S706;
  • V 1 is the spatial parameter V calculated in step S707.
  • T 0 (f) is the reverberation time corresponding to the target frequency response energy, and V 0 is the spatial parameter corresponding to the target frequency response energy.
  • the frequency characteristics of audio playback equipment are often described by frequency response curves.
  • the ordinate of the frequency response curve represents power (unit is decibel), and the abscissa represents frequency (unit is Hertz).
  • the abscissa The scale is logarithmic, and the ordinate is linear. Therefore, in the embodiment of the present application, the gain can be converted into a linear domain amplitude gain gain.
  • the smart screen saves calibration parameters, which include gains corresponding to multiple frequency bands.
  • the smart screen after obtaining the calibration parameters of each frequency band, applies the calibration parameters of each frequency band to the audio of this frequency band when playing other audio, so that the actual sound field of the audio of this frequency band meets the target of this frequency band. soundstage.
  • the calibration parameters may also include metadata of the audio to be played, where the metadata may include multi-channel audio loudness and Its gain factor, and spatial position data, the spatial position data is the spatial mapping relationship between multi-channel data and PA.
  • the audio format of the preset audio is Audiovivid
  • the smart screen plays the preset audio based on the metadata of the preset audio in step S703.
  • the metadata includes multi-channel audio loudness and its gain factor, and spatial position data; then,
  • the smart screen plays the audio to be played in the audio format Audiovivid, it can play the audio to be played based on the calibration parameters obtained by S709 and the metadata of the preset audio. It can be understood that the calibration parameters are obtained based on the preset audio.
  • the calibration parameters have the best calibration effect on the audio to be played.
  • the following is an example of a method for playing audio to be played on a smart screen. Please refer to Figure 11.
  • the method includes the following steps:
  • the smart screen can be installed with various music software; the smart screen can display the playback interface of the music software, and when detecting the user's playback operation for the audio to be played, in response to the playback operation, from the music playback server or Get the audio to be played from storage.
  • S902 The smart screen determines whether the audio format of the audio to be played is the preset audio format.
  • the smart screen can decode the audio to be played through a decoder and determine the audio format of the audio to be played.
  • audio formats can include Dolby, Audiovivid, etc.
  • the smart screen determines that the audio format of the audio to be played is the preset audio format, it obtains calibration parameters.
  • the calibration parameters include gains corresponding to multiple frequency bands and metadata of the data to be played.
  • the smart screen when the smart screen determines that the audio format of the audio to be played is the preset audio format, it obtains the calibration parameters, and then performs step S905.
  • the preset audio format is the audio format of the preset audio
  • the gains corresponding to the multiple frequency bands are the calibration parameters calculated by the smart screen through the above steps S701 to S710 based on the preset audio
  • the metadata includes multi-channel audio loudness and its gain factors, and spatial location data.
  • the audio format of the preset audio can be Audiovivid, then the preset audio format is Audiovivid; therefore, after obtaining the audio to be played, the smart screen determines whether the audio format of the audio to be played is Audiovivid.
  • the smart screen processes the audio to be played according to the default settings of the audio equalizer in the smart screen, and plays the processed audio to be played.
  • S905 The smart screen processes the audio to be played based on the calibration parameters and obtains the processed audio.
  • the smart screen can process the loudness of the audio to be played based on the gains corresponding to multiple frequency bands, as well as the multi-channel audio loudness and its gain factor of the audio to be played, to obtain the processed loudness.
  • the processed audio The loudness is the processed loudness; based on the spatial position data of the audio to be played, the playback parameters of the audio to be played are processed to obtain the processed playback parameters.
  • the processed audio includes the adjusted playback parameters.
  • the adjusted playback parameters It is used to play the multi-channel data of the audio to be played based on the spatial position data indicating the spatial mapping relationship between the multi-channel data and the PA.
  • the following is an exemplary description of how the smart screen processes the loudness of the audio to be played based on the loudness of the multi-channel audio and its gain factor.
  • the gain factor is used to change the loudness of the audio.
  • the processed loudness can be obtained by multiplying the audio loudness of the channel by the gain factor corresponding to the channel.
  • the three channels are channel A, channel B and channel C. That is, the audio to be played includes the audio of channel A, the audio of channel B and the audio of channel B.
  • the audio loudness of channel A is adjusted to the initial loudness of the audio of channel A multiplied by the gain factor c'.
  • the smart screen adjusts the audio loudness of channel B to the initial loudness of the audio of channel B multiplied by the gain factor b'.
  • the smart screen adjusts the audio loudness of channel C to the initial loudness of the audio of channel C multiplied by the gain factor c'.
  • S906 The smart screen plays the processed audio.
  • the processed audio includes playback parameters
  • the adjusted playback parameters are used to instruct the multi-channel data of the audio to be played to be played based on the spatial position data indicating the spatial mapping relationship between the multi-channel data and the PA; Then the smart screen plays the processed audio based on the above-adjusted playback parameters, that is, the data of different channels is emitted from the sound source corresponding to the channel.
  • the multiple channels can be sky channels, left and right surround channels, etc.
  • the smart screen plays processed audio based on the above-adjusted playback parameters.
  • the smart screen can include multiple sound sources, distributed on the top, back, left and right sides of the smart screen, and Positions such as the bottom, combined with multiple channels such as sky channels, left and right surround channels, can bring a better 3D surround effect.
  • the smart screen when it obtains the processed audio, it can determine the sound source corresponding to the data of each sound channel based on the adjusted playback parameters; and play the sound channel data corresponding to the sound source through the sound source. Assuming that the adjusted playback parameters indicate that the audio of channel A corresponds to sound source a, the audio of channel B corresponds to sound source b, and the audio of channel C corresponds to sound source c, then the smart screen plays the audio of channel A through sound source a. , the audio of channel B is played through sound source b, and the audio of channel C is played through sound source c.
  • channel A is the sky channel
  • sound source a is several sound units located at the top of the smart screen
  • channel B is the left channel
  • sound source b is the sound unit located on the left side of the smart screen
  • channel C is the right sound unit.
  • the sound source is the sound unit located on the right side of the smart screen
  • the smart screen plays the audio of the sky channel through several sound units located on the top of the smart screen based on the adjusted playback parameters, and through the sound unit located on the left side of the smart screen Play left channel audio frequency, and plays the audio of the right channel through the sound unit on the right side of the smart screen.
  • the term “when” may be interpreted to mean “if" or “after” or “in response to determining" or “in response to detecting" depending on the context.
  • the phrase “when determining" or “if (stated condition or event) is detected” may be interpreted to mean “if it is determined" or “in response to determining" or “on detecting (stated condition or event)” or “in response to detecting (stated condition or event)”.
  • the computer program product includes one or more computer commands.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer commands may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer commands may be transmitted from a website, computer, server, or data center Transmission to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated.
  • the available media may be magnetic media (eg, floppy disk, hard disk, tape), optical media (eg, DVD), or semiconductor media (eg, solid state drive), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Quality & Reliability (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic System (AREA)

Abstract

La présente demande concerne un procédé d'étalonnage de champ sonore, un dispositif électronique et un système. Le procédé comprend les étapes suivantes consistant à : établir par un dispositif à grand écran une connexion de communication avec un dispositif d'enregistrement ; en réponse à une instruction d'utilisateur, commander par le dispositif à grand écran la lecture d'un audio prédéfini ; envoyer par le dispositif à grand écran une instruction d'enregistrement au dispositif d'enregistrement, l'instruction d'enregistrement étant utilisée pour ordonner au dispositif d'enregistrement d'effectuer un enregistrement, de façon à obtenir un audio enregistré de l'audio prédéfini ; recevoir par le dispositif à grand écran l'audio enregistré, qui est envoyé par le dispositif d'enregistrement ; déterminer par le dispositif à grand écran un paramètre d'étalonnage sur la base de l'audio enregistré, le paramètre d'étalonnage étant utilisé pour étalonner un audio à lire ; et commander par le dispositif à grand écran la lecture de l'audio à lire qui a été étalonné. Au moyen de la mise en œuvre de la présente demande, un paramètre d'étalonnage peut être calculé sur la base d'un audio enregistré qui est enregistré par un dispositif d'enregistrement lors du processus pendant lequel un dispositif à grand écran lit un audio prédéfini, et le paramètre d'étalonnage est utilisé pour étalonner un audio à lire, de telle sorte qu'un champ sonore pendant la lecture audio peut être efficacement étalonné, ce qui permet d'améliorer l'expérience de l'utilisateur.
PCT/CN2023/116692 2022-09-05 2023-09-04 Procédé d'étalonnage de champ sonore, dispositif électronique et système WO2024051638A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211079965.5 2022-09-05
CN202211079965.5A CN117692845A (zh) 2022-09-05 2022-09-05 声场校准方法、电子设备及系统

Publications (1)

Publication Number Publication Date
WO2024051638A1 true WO2024051638A1 (fr) 2024-03-14

Family

ID=90128877

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/116692 WO2024051638A1 (fr) 2022-09-05 2023-09-04 Procédé d'étalonnage de champ sonore, dispositif électronique et système

Country Status (2)

Country Link
CN (1) CN117692845A (fr)
WO (1) WO2024051638A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070116306A1 (en) * 2003-12-11 2007-05-24 Sony Deutschland Gmbh Dynamic sweet spot tracking
CN112235688A (zh) * 2020-09-25 2021-01-15 深圳市火乐科技发展有限公司 一种调节声场的方法和装置
CN113179401A (zh) * 2021-05-18 2021-07-27 康佳集团股份有限公司 一种声场校正方法、终端及计算机可读存储介质
CN113709629A (zh) * 2021-08-26 2021-11-26 深圳创维－Rgb电子有限公司 频响参数调节方法、装置、设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070116306A1 (en) * 2003-12-11 2007-05-24 Sony Deutschland Gmbh Dynamic sweet spot tracking
CN112235688A (zh) * 2020-09-25 2021-01-15 深圳市火乐科技发展有限公司 一种调节声场的方法和装置
CN113179401A (zh) * 2021-05-18 2021-07-27 康佳集团股份有限公司 一种声场校正方法、终端及计算机可读存储介质
CN113709629A (zh) * 2021-08-26 2021-11-26 深圳创维－Rgb电子有限公司 频响参数调节方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN117692845A (zh) 2024-03-12

Similar Documents

Publication Publication Date Title
US11880628B2 (en) Screen mirroring display method and electronic device
JP7222112B2 (ja) 歌の録音方法、音声補正方法、および電子デバイス
WO2022002166A1 (fr) Procédé et dispositif de traitement de bruit d'écouteur, et écouteur
US20230053104A1 (en) Method for implementing stereo output and terminal
WO2022002110A1 (fr) Procédé et appareil de commande de mode, et dispositif terminal
EP4044578A1 (fr) Procédé de traitement audio et dispositif électronique
WO2021000817A1 (fr) Procédé et dispositif de traitement de son ambiant
CN111314560A (zh) 一种调整声音响度的方法及通信终端
WO2022007944A1 (fr) Procédé de commande de dispositif et appareil associé
CN110989961A (zh) 一种声音处理方法及其装置
US20230343318A1 (en) Noise reduction method and noise reduction apparatus
WO2023125847A1 (fr) Procédé et système de traitement audio, et appareils associés
EP4203447A1 (fr) Procédé de traitement de son et appareil associé
CN115223579A (zh) 一种编解码器协商与切换方法
WO2023071502A1 (fr) Procédé et appareil de contrôle de volume, et dispositif électronique
WO2024051638A1 (fr) Procédé d'étalonnage de champ sonore, dispositif électronique et système
CN114040319B (zh) 一种终端设备外放音质优化方法、装置、设备和介质
WO2022089563A1 (fr) Procédé d'amélioration de son, procédé et appareil de commande d'écouteur et écouteur
CN115641870A (zh) 一种音频信号的处理方法及相关电子设备
WO2024027259A1 (fr) Procédé et appareil de traitement de signaux et procédé et appareil de commande de dispositif
WO2024046416A1 (fr) Procédé de réglage de volume, dispositif électronique et système
WO2024066933A1 (fr) Procédé et dispositif de commande de haut-parleurs
WO2024046182A1 (fr) Procédé et système de lecture audio, et appareil associé
WO2024032035A1 (fr) Procédé de sortie de signal vocal et dispositif électronique
WO2023216922A1 (fr) Procédé d'identification de sélection de dispositif cible, dispositif terminal, système et support de stockage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23862332

Country of ref document: EP

Kind code of ref document: A1