TW202324372A - Audio system with dynamic target listening spot and ambient object interference cancelation - Google Patents

Abstract

A sound system is proposed, dynamically playing optimized audio signals based on user position. A sensor circuits dynamically senses a target space to generate a field environment information. First speaker and second speaker are arranged for audio playback. A host device recognizes a user in the field environment information, and determine the user position corresponding to the target space, and adaptively designate the user position as a target listening spot. A sensor circuit contains a camera capturing a field environment image out of the target space. A recognizer circuit analyzes the field environment image to obtain from the target space, an ambient object with location, size and acoustic attribute information. The control circuit performs a channel based compensation operation on the target listening spot to generate optimized first channel audio signal and second channel audio signal.

Description

Audio system that dynamically adjusts to the target listening point and eliminates distractions from ambient objects

This application relates to audio processing technology, which is actually an audio system that can dynamically adjust the playback effect according to the changes in the sound field space.

The existing sound system includes a plurality of loudspeakers arranged around a target space to form a surround sound field environment. Each speaker can output corresponding channel audio respectively. When configuring the surround sound field environment, the installer of the sound system will usually designate a central area of the target space as the best listening point, as a basis for installing multiple speakers. When multiple speakers play multiple channels of audio at the same time, the user at the sweet spot can obtain an immersive listening effect.

However, in a real environment, the user's listening effect is easily affected by various variables. For example, in a traditional audio system, the range of the sweet spot is zone-bound. When the user moves to an area outside the sweet spot, although the multi-channel audio output by the audio system can still be heard, the listening effect of the multi-channel audio at the user's position may have been greatly reduced or completely eliminated. invalidated. In addition, the room layout, furniture position and materials in the target space are all environmental objects that may interfere with the listening effect. For example, couches, windows, and drapes absorb or reflect some of the sound energy, distorting the individual channel audio received at the sweet spot.

In other words, the traditional audio system cannot dynamically adjust the position of the sweet spot, and the user is forced to restrict movement to accommodate the position of the sweet spot, which is really inconvenient. On the other hand, the audio of each channel may be distorted and distorted by the interference of environmental objects, so that the range of the sweet spot is further limited or even disappeared. In this way, the sound field environment built at high cost is meaningless.

In view of this, how to make the audio system dynamically adjust the sweet spot as the user moves and eliminate the interference of environmental objects in the target space is a problem to be solved.

This specification provides an embodiment of an audio system, which can dynamically optimize the playback effect according to the user's position, wherein the audio system includes a sensor circuit, a first speaker and a second speaker, and a host device. The sensor circuit is configured to dynamically sense a target space to generate sound field environment information. A first speaker and a second speaker are set to play audio. The host device is coupled to the sensor circuit, the first speaker and the second speaker, and includes an identification circuit, a control circuit and an audio transmission circuit. The identification circuit is configured to identify a user from the sound field environment information and determine a user position of the user in the target space. The control circuit is coupled to the identification circuit and configured to dynamically assign the user's position as a target listening point. The audio transmission circuit is coupled to the control circuit, the first speaker and the second speaker, and is configured to transmit audio. Wherein, the sensor circuit includes a camera configured to capture a sound field environment image of the target space. The identification circuit analyzes the sound field environment image to obtain spatial configuration information and acoustic attribute information of an environmental object in the target space. The control circuit performs channel floor compensation operation according to the target listening point, spatial configuration information and acoustic property information of the environmental object to generate a first channel audio and a second channel audio optimized for the target listening point. Finally, the control circuit respectively outputs the first channel audio and the second channel audio to the corresponding first speaker and the second speaker through the audio transmission circuit.

This specification provides an embodiment of an audio system, which can dynamically optimize the playback effect according to the user's position, wherein the audio system includes a sensor circuit, a first speaker and a second speaker, and a host device. The sensor circuit is configured to dynamically sense a target space to generate sound field environment information. A first speaker and a second speaker are set to play audio. The host device is coupled to the sensor circuit, the first speaker and the second speaker, and includes an identification circuit, a control circuit and an audio transmission circuit. The identification circuit is configured to identify a user from the sound field environment information and determine a user position of the user in the target space. The control circuit is coupled to the identification circuit and configured to dynamically assign the user's position as a target listening point. The audio transmission circuit is coupled to the control circuit, the first speaker and the second speaker, and is configured to transmit audio. Wherein, the sensor circuit includes a camera configured to capture a sound field environment image of the target space. The identification circuit analyzes the sound field environment image to obtain spatial configuration information and acoustic attribute information of an environmental object in the target space. The control circuit corresponds the target space to an object base space, and correspondingly creates a compensation sound source object in the object base space according to the environmental object. A metadata of the compensation sound source object includes: the coordinate position, size, and reflection rate and absorption rate of the environmental object. The control circuit performs an object base compensation operation based on the target listening point and the relay data to offset the interference of the environmental object on the target listening point to generate a first channel audio and a second channel optimized for the target listening point channel audio. The control circuit respectively outputs the first channel audio and the second channel audio to the corresponding first speaker and the second speaker through the audio transmission circuit.

This specification provides an embodiment of an audio system, which can dynamically optimize the playback effect according to the user's position, wherein the audio system includes a sensor circuit, a first speaker and a second speaker, and a host device. The sensor circuit is configured to dynamically sense a target space to generate sound field environment information. The first speaker and the second speaker are configured to play audio. The host device is coupled to the sensor circuit, the first speaker and the second speaker, and includes an identification circuit, a control circuit, an audio transmission circuit and a man-machine interface circuit. The identifying circuit is configured to identify a user from the sound field environment information and determine a user position of the user in the target space. The control circuit is coupled to the identification circuit and configured to dynamically assign the user's position as a target listening point. The audio transmission circuit is coupled to the control circuit, the first speaker and the second speaker, and is configured to transmit audio. The human-machine interface circuit is coupled to the control circuit, and is configured to run a configuration program to obtain spatial configuration information and acoustic attribute information of an environmental object in the target space. Wherein, the control circuit performs a channel floor compensation operation according to the target listening point, the spatial configuration information and the acoustic property information of the environmental object to generate a first channel audio and a second sound optimized for the target listening point Road audio. The control circuit respectively outputs the first channel audio and the second channel audio to the corresponding first speaker and the second speaker through the audio transmission circuit.

This specification provides an embodiment of an audio system, which can dynamically optimize the playback effect according to the user's position, wherein the audio system includes a sensor circuit, a first speaker and a second speaker, and a host device. The sensor circuit is configured to dynamically sense a target space to generate sound field environment information. The first speaker and the second speaker are configured to play audio. The host device is coupled to the sensor circuit, the first speaker and the second speaker, and includes an identification circuit, a control circuit, an audio transmission circuit and a man-machine interface circuit. The identifying circuit is configured to identify a user from the sound field environment information and determine a user position of the user in the target space. The control circuit is coupled to the identification circuit and configured to dynamically assign the user's position as a target listening point. The audio transmission circuit is coupled to the control circuit, the first speaker and the second speaker, and is configured to transmit audio. The human-machine interface circuit is coupled to the control circuit, and is configured to run a configuration program to obtain spatial configuration information and acoustic attribute information of an environmental object in the target space. Wherein, the control circuit corresponds the target space to an object base space, and correspondingly creates a compensation sound source object in the object base space according to the environmental object. A metadata of the compensation sound source object includes: the coordinate position, size, and reflection rate and absorption rate of the environmental object. The control circuit performs an object base compensation operation based on the target listening point and the relay data to offset the interference of the environmental object on the target listening point to generate a first channel audio and a second channel optimized for the target listening point channel audio. The control circuit respectively outputs the first channel audio and the second channel audio to the corresponding first speaker and the second speaker through the audio transmission circuit.

One of the advantages of the above embodiment is that the audio system can dynamically track the user's location through the sensor, and continuously optimize the playback effect according to the user's location. Users do not need to settle for a fixed listening position for the best experience.

Another advantage of the above-mentioned embodiment is that the sound system can identify the environmental objects in the target space, and accordingly adjust the channel audio to counteract the interference of the environmental objects.

Other advantages of the present invention will be explained in more detail with the following description and drawings.

Embodiments of the present invention will be described below in conjunction with related figures. In the drawings, the same reference numerals indicate the same or similar elements or method flows.

FIG. 1 is a functional block diagram of an audio system 100 according to an embodiment of the present invention.

The audio system 100 is mainly composed of a host device 130 and a plurality of speakers. The host device 130 can control multiple speakers to play audio. The host device 130 can be a host computer, a barebone system, an embedded system, or a customized digital audio processing device. The host device 130 includes a communication circuit 136 , so that the host device 130 can be wired or wirelessly connected to a user equipment 150 as an input channel for audio signals or data.

The user equipment 150 can be a mobile phone, a computer, a TV stick, a game console, or other audio source providing devices, which provide music or audio streams to the host device 130 through the communication circuit 136 . Furthermore, the audio system 100 can use the communication circuit 136 to cooperate with the user equipment 150 or other multimedia equipment to form a home theater system with both video and audio functions. For example, the target space 170 may further include a projection screen, screen or display (not shown), which is controlled by the user equipment 150 to display images. For another example, the user device 150 may be a head-mounted virtual reality device. The user 180 can stand in the target space 170 and see the picture through the user equipment 150 , and the host device 130 can be controlled by the user equipment 150 to play audio synchronously with the picture. The communication circuit 136 in this embodiment may be, but not limited to, a high definition multimedia interface (High Definition Multimedia Interface; HDMI), a digital transmission interface (Sony/Philips Digital Interface Format; SPDIF), a wireless local area network module , Ethernet module, short-wave RF transceiver, or Bluetooth Low Energy (Bluetooth Low Energy; BLE) version 4 or version 5 evolution application, or Universal Serial Bus (Universal Serial Bus).

The host device 130 also includes an audio transmission circuit 135 for connecting multiple speakers, and outputting multiple channel audio respectively for the speakers to play. The manner in which the host device 130 controls multiple speakers through the audio transmission circuit 135 may be a one-way digital or analog output, or a two-way synchronous communication protocol. The connection between the audio transmission circuit 135 and each speaker can be a wired interface, a wireless interface or a hybrid application of the two. The wired interface can be, but not limited to, a composite audio-visual terminal, a digital transmission interface, or a high-definition multimedia interface. The wireless interface can be, but is not limited to, a WLAN, a shortwave radio frequency transceiver, or an evolution of Bluetooth low energy version 4 or 5. In a further derivative embodiment, since the audio transmission circuit 135 and the communication circuit 136 are functionally positioned as interfaces for connecting with external components, they may be an integrated multifunctional bidirectional transmission interface module in a derivative implementation. The audio transmission circuit 135 and the communication circuit 136 use various public standard transmission technologies to realize the connection and transmission between components, which can increase the future functional expandability of the audio system 100 and reduce the replacement cost when components are damaged.

The target space 170 in FIG. 1 can be understood as a three-dimensional space where the user 180 can use the sound system 100 . Each speaker can be arranged at a different position in the target space 170 to play a channel of audio correspondingly. The surround configuration of multiple speakers can create a surround sound field environment in a target space 170 . There are many standard specifications for the number and configuration of horns. For example, in a 5.1-channel surround sound system, two front speakers, a center speaker, two surround channel speakers, and a subwoofer are included to create a sound system that surrounds a target listening point. Create a surround sound field space, and play the sound to the target listening point together. In a 7.1-channel ambient sound system, a pair of rear surround channel speakers are further arranged behind the target listening point to provide a more three-dimensional sound field effect. In recent years, new specifications such as 5.1.2-channel and 7.2.2-channel have emerged, including more speakers and channel configurations in specific directions, which can achieve more realistic "panoramic sound", "sky sound" or " Floor Sound Effect” and other effects. To facilitate description of the technical features of the audio system 100 of this embodiment, only the first speaker 110 and the second speaker 120 are shown in FIG. 1 as representatives. Wherein, the first speaker 110 receives and plays the first channel audio 112 provided by the host device 130 , and the second speaker 120 receives and plays the second channel audio 122 provided by the host device 130 . It must be understood that, in practice, the audio system 100 of this embodiment is not limited to only two speakers, but can be applied to 2.1 channels, 4.1 channels, 5.1 channels, 7.2 channels or more Multi-channel specification configuration. Each speaker in the target space 170 may have different audio output specifications. For example, some speakers are good at outputting heavy bass, and some speakers are good at outputting mid-high. The host device 130 can plan various sound field environments with different characteristics in the target space 170 according to different speaker specifications.

The term "sound channel" referred to in the specification and scope of patent application generally refers to various physical sound channels and logical sound channels. The logical channel refers to the audio data stream transmitted within the system, while the physical channel refers to the source of the signal played by each speaker. In this embodiment, each speaker corresponds to the first-channel audio 112 and the second-channel audio 122, which belong to physical channels and can be down-mixed by one or more logical channels. And the result produced. For example, a pair of headphones has only two speakers, but you can hear the sound effects generated by multiple applications at the same time. In other words, the audio data of multiple applications can be down-mixed into two physical audio channels by the system, and played as audible sound through two speakers. Therefore, the first-channel audio 112 and the second-channel audio 122 in this embodiment are not limited to audio signals that only include a single logical channel, but may also be audio generated by mixing multiple logical channels according to a predetermined ratio. Signal.

In FIG. 1 , the first speaker 110 and the second speaker 120 are arranged on two sides of a target space 170 , and play sound to a target listening point in the target space 170 . The target listening point can be understood as the position where the playback effect of the sound system 100 is optimized. In some audio systems, the target listening point is called the listening sweet spot (Listening Sweet Spot). In most cases, the target listening point is usually located in a specific area of the target space 170 , such as the center point, on the axis, on the tangent plane, or the equivalent volume center of multiple speakers. In FIG. 1 , the target listening point of the target space 170 is represented by the first position 171 where the user 180 is located. When the user 180 moves from the first position 171 to the second position 172 along the moving track 173, since the user 180 moves away from the first speaker 110 and approaches the second speaker 120, the listening effect received by the user 180 is produced. Deviation. The traditional audio system cannot track the movement of the user 180 and accordingly adjust the listening effect received by the second position 172 , but the solution proposed in this embodiment will be described in detail later.

On the other hand, the target space 170 usually includes some environmental objects 175 , such as sofas, tables, windowsills, walls, ceilings, and floors. Depending on the material, size and location of these environmental objects 175 , they will produce different degrees of interference to the sounds played by the first speaker 110 and the second speaker 120 . For example, a fabric sofa or window sills will absorb sound, and marble floors or walls will reflect sound. In other words, the existence of the environmental object 175 will affect the first channel audio 112 and the second channel audio 122 received by the target listening point. The traditional audio system does not have the ability to identify the environmental objects 175 in the target space 170 , nor does it have the function of compensating the first channel audio 112 and the second channel audio 122 according to the size, material, and position of the environmental objects 175 . The sound system 100 of the present embodiment can calculate and eliminate the interference of all environmental objects 175 in the target space 170 on the first channel audio 112 and the second channel audio 122 . For convenience of description, only one environment object 175 is shown in FIG. 1 of this embodiment to explain the operation of the sound system 100 . However, it must be understood that the target space 170 in FIG. 1 is not limited to only one environmental object 175 . The solution to the interference of the environmental object 175 will be described in detail later.

The host device 130 of this embodiment further includes a storage circuit 131 . The storage circuit 131 may include a non-volatile memory for storing related operating systems, application software or firmware required for the operation of the host device 130 . The storage circuit 131 may also include a volatile memory, which is used as an operation memory of the control circuit 132 . The host device 130 of this embodiment further includes a control circuit 132 . The control circuit 132 can be a central processing unit, a digital signal processor, or a microcontroller. The control circuit 132 can read the pre-stored operating system, software or firmware from the storage circuit 131 to control the host device 130 , the first speaker 110 and the second speaker 120 to perform an audio playback operation. Furthermore, the host device 130 of this embodiment uses the control circuit 132 to perform a series of sound field compensation calculations to dynamically optimize the playback effect and solve the insurmountable shortcomings of traditional audio systems.

In order to dynamically optimize the playback effect at the target listening point, the audio system 100 of this embodiment includes a sensor circuit 140 configured to dynamically sense a target space 170 to generate sound field environment information. The sensor circuit 140 may be a component located outside the host device 130 and coupled to the host device 130 . The sensor circuit 140 may be a combination of one or more of the camera 610 , the infrared sensor 620 , and the wireless detector 630 . The types of the sound field environment information captured by the sensor circuit 140 can be combined in different ways according to the implementation of the sensor circuit 140 . For example, the sound field environment information may include one or a combination of images, pictures, thermal imaging, and radio wave imaging of the user and the environmental objects. In one embodiment, the sensor circuit 140 is disposed around the target space 170 . It can be understood that, although only one sensor circuit 140 is shown in FIG. 1 , in practice, the audio system 100 may include multiple sets of sensor circuits 140 respectively arranged in different positions around the target space 170 . In order to obtain more accurate sound field environment information.

In this embodiment, the host device 130 includes an identification circuit 134 coupled to the sensor circuit 140 . The identification circuit 134 can identify the key information affecting the sound field from the sound field environment information, so that the control circuit 132 can dynamically adjust the first channel audio 112 and the second channel audio played from the first speaker 110 and the second speaker 120 accordingly. Channel Audio 122. For example, the identification circuit 134 can identify a user from the sound field environment information and determine a user position of the user in the target space. Since the sound field environment information provided by the sensor circuit 140 can be combined in various types, the identification circuit 134 can also correspondingly implement different identification technical solutions. For example, when the sound field environment information is an image, the identification circuit 134 can use artificial intelligence identification technology to identify the user in the image. Through the application of artificial intelligence, after the identification circuit 134 analyzes the user in the image, it can further locate the user's head, face, and even the position of the ears. If the sensor circuit 140 can provide multiple information such as three-dimensional images with spatial depth, infrared thermal imaging, or wireless signals, it will help the identification circuit 134 to obtain more accurate identification results.

In order to calculate the degree of interference caused by the environmental object 175 to the sound field environment, the host device 130 needs spatial configuration information and acoustic attribute information of the environmental object 175 . The spatial configuration information may include the size, position, shape, and various appearance characteristics of the environmental object 175 . The acoustic property information may include material-related characteristics such as sound absorption rate, reflectance rate, and resonance frequency. In one embodiment, when identifying the sound field environment information, the identification circuit 134 may further identify the spatial configuration information of the environmental objects 175 in the target space 170 from the sound field environment information, and search for the acoustic attribute information. In order to identify environmental objects, an object database is required. In one embodiment, the storage circuit 131 in the host device 130 can also be used to store an object database. The object database may include various appearance feature information for identifying environmental objects, and various acoustic property information corresponding to each environmental object. For example, when the host device 130 needs to calculate the degree of interference caused by an environmental object 175 to the sound field environment, it can first analyze the object name of the environmental object 175 through the identification circuit 134, and then the host device 130 reads the storage circuit 131 to find the Absorptivity and reflectivity corresponding to the environmental object 175 .

In practice, the identification circuit 134 can be a processor chip made by a customer, and cooperate with the existing operating system, software or firmware in the storage circuit 131 to perform the identification function of artificial intelligence. The identification circuit 134 can also be one of the core or thread circuits of the control circuit 132 , which executes the existing artificial intelligence software product in the storage circuit 131 to realize the identification function. The identification circuit 134 can also be a memory module of a specific artificial intelligence software product, which is executed by the control circuit 132 to complete the identification function.

The human-machine interface circuit 133 in the host device 130 can allow the user to control the operation of the host device 130 . The man-machine interface circuit 133 may include a display screen, buttons, dials, or a touch screen, allowing the user to perform basic control functions of the audio system 100, such as adjusting volume, playing, and fast forward and reverse. In one embodiment, the control circuit 132 can also execute a configuration program through the man-machine interface circuit 133 for the user to set various sound field situations, or inform the host device 130 of the spatial configuration information of the environmental objects 175 in the target space 170 . For example, in the configuration procedure, the control circuit 132 utilizes the human-machine interface circuit 133 to receive object configuration data input by the user, such as the object name, type, size and location of one or more environmental objects 175 . After the control circuit 132 obtains the spatial configuration information, it searches the corresponding absorptivity and reflectivity from the object database stored in the storage circuit 131 for subsequent sound field compensation operations. In a further derivative embodiment, the human-machine interface circuit 133 may also be provided by the user equipment 150 . The user can use the user equipment 150 to operate the configuration program, and finally the user equipment 150 transmits the setting result to the control circuit 132 through the communication circuit 136 .

The host device 130 can also be connected to a remote database 160 through the communication circuit 136 . In a further embodiment, the object database originally stored by the storage circuit 131 can also be stored through the remote database 160 . When the host device 130 needs to calculate the degree of interference caused by an environmental object 175 to the sound field environment, it can first analyze the sound field environment information through the identification circuit 134 to obtain an object characteristic value, and then use the communication circuit 136 to access the remote database 160 , find out an environmental object 175 that matches the characteristic value of the object, and obtain the sound field attribute information of the environmental object 175 . The remote database 160 can be a server located in the cloud or other systems, and is connected with the host device 130 through a wired or wireless two-way network communication technology. In addition to providing a search function, the remote database 160 can also accept uploads of updated data, so as to continuously expand the content of the database. For example, the host device 130 can use Structured Query Language (SQL) to communicate with the remote database 160 .

Based on the system architecture of FIG. 1 , the audio system 100 proposed in this application can achieve at least the following technical effects. Firstly, the audio system 100 can dynamically track the user's location as the target listening point. The sound system 100 can also dynamically acquire the spatial configuration information of the environmental objects as a basis for optimizing the sound field effect. Finally, the audio system 100 dynamically compensates the speaker output according to the user's position and the spatial configuration information of the environmental objects, so as to eliminate object interference and optimize the listening effect at the target listening point. The implementation of dynamic tracking of the user's location can adopt multiple technical solutions such as cameras, infrared sensors, or wireless positioning. The implementation of obtaining the spatial configuration information of the environmental objects can be performed automatically or manually. For example, the sound system 100 can use a camera to capture images and perform artificial intelligence recognition, or allow users to manually input the environmental conditions of the scene through a configuration program. The implementation of compensating the horn output can be based on several different algorithms. For example, this specification introduces the channel base (Channel Base) algorithm and the object base (Object Base) algorithm.

FIG. 2 below illustrates an embodiment in which the sound system 100 dynamically tracks the user's position, uses a camera to obtain the configuration of the sound field environment, and uses channel floor compensation to compensate the speaker output.

FIG. 2 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

In the flow chart of FIG. 2 , the process in the column of a specific device represents the process performed by the specific device. For example, the part marked in the column of "sensor circuit" is the process performed by the sensor circuit 140; the part marked in the column of "host device" is the process performed by the host device 130; The part marked in the "speaker" column is the process performed by the first speaker 110 and/or the second speaker 120; and so on for the rest. The aforementioned logic is also applicable to other subsequent flow charts.

In the process 202 , the sensor circuit 140 dynamically senses the target space to generate sound field environment information. In this embodiment, the sound field environment information may be optical, thermal, or electromagnetic wave information in the target space 170 . For example, the sensor circuit 140 may include a camera, which continuously captures video of the target space 170 in a video mode, or periodically captures still photos of the target space 170 in a photographic mode. In another embodiment, the sensor circuit 140 may further include an infrared sensor configured to capture a thermal imaging data in the target space. The thermal imaging data generated by the infrared sensor, in addition to containing spatial depth information, is also extremely sensitive to temperature changes, so it is especially suitable for tracking the user's location. In another embodiment, the sensor circuit 140 may further include a wireless detector, which is disposed in the target space and detects a wireless signal of an electronic device. When a user holds an electronic device, the wireless detector can detect the beacon time difference or the strength of the wireless signal of the electronic device as an auxiliary means for tracking the user's location. The electronic device can be the user's own mobile phone, a special beacon generator, a head-mounted virtual reality device, a game handle, or a remote control of the audio system 100 . It can be understood that this embodiment does not limit the number of sensor circuits 140 , nor does it limit that only one sensing scheme can be used at a time. For example, the audio system 100 of this embodiment can use multiple sensor circuits 140 to work together from different locations, or use one or more cameras, infrared sensors and wireless detectors at the same time. In this way, the host device 130 can obtain more complete sound field environment information, and obtain more accurate identification results in subsequent procedures.

In the process 204 , the sensor circuit 140 transmits the sensed sound field environment information to the host device 130 . The sensor circuit 140 can continuously transmit data, such as video, or periodically return static data. The frequency at which the sensor circuit 140 transmits data can be adaptively determined according to the information volume of the sound field environment information, the tracking accuracy requirement, and the computing capability of the host device 130 . The connection between the sensor circuit 140 and the host device 130 can be through a dedicated line or through the communication circuit 136 . In a further derivative embodiment, the sensor circuit 140 can share the audio transmission circuit 135 with the speaker, so as to transmit the sound field environment information to the host device 130 through the audio transmission circuit 135 .

In the process 206 , the host device 130 determines the location of the user according to the sound field environment information received from the sensor circuit 140 . The recognition circuit 134 in the host device 130 can execute a recognition program on the sound field environment information, such as applying artificial intelligence. As the sensing schemes of the sensor circuit 140 are different, the identification algorithms of the identification circuit 134 are also correspondingly different. It can be understood that the target space 170 and the user position can be expressed in two-dimensional space or three-dimensional space. If only a single sensor circuit 140 is implemented in the audio system 100 , at least the position information in a two-dimensional space can be sensed. If the audio system 100 increases the number of sensor circuits 140 or mixes multiple sensing schemes in practice, the depth information of the three-dimensional space can be obtained to more accurately determine the position of the user or the position of the user's head. In one embodiment, the recognition circuit 134 can dynamically recognize the user's head position, face direction, or ear position according to the sound field environment image captured by the camera. In another embodiment, the identification circuit 134 can analyze the movement track of the thermal imaging data generated by the infrared sensor to dynamically determine the location of the user 180 . For another example, the identification circuit 134 can dynamically locate the coordinate value of the electronic device in the target space 170 according to the characteristics of the wireless signal detected by the wireless detector. According to the coordinate value, the control circuit 132 can further estimate the position of the user's ear.

In the process 208 , after the identification circuit 134 in the host device 130 analyzes the user's location, the control circuit 132 in the host device 130 dynamically assigns the user's location as the target listening point. For the convenience of describing the following embodiments, the target space 170 is described here as a two-dimensional coordinate space or a three-dimensional coordinate space, and the target listening point can be expressed as a coordinate value in the target space 170 . Depending on the layout of multiple speakers, the range of the target listening point can be not only a single point, but also a plane, or a three-dimensional area with length, width and height. For example, after the identification circuit 134 analyzes the user's head position or ear position, the control circuit 132 can assign the user's head position or ear position as the target listening point. The control circuit 132 will make the playback effect obtained at the target listening point not be affected by the user's movement through subsequent compensation operations. In practice, the control circuit 132 compensates the listening effect obtained by the target listening by adjusting the first-channel audio 112 and the second-channel audio 122 . It can be understood that the process 208 may be executed dynamically as the user's location changes. Therefore, the process 208 is not limited to be executed in the order shown in FIG. 2 . In other words, the target listening point can be updated in real time as the user's position changes. The specific adjustment operation will be described later.

In the process 210 , the recognition circuit 134 in the host device 130 further recognizes the sound field environment information provided by the sensor circuit 140 to obtain the spatial configuration information of the environmental objects in the target space 170 . In other words, the sound field environment information provided by the sensor circuit 140 can not only be used to determine the user's position, but also can be used to determine various environmental objects 175 existing in the target space 170 . In one embodiment, after the camera in the sensor circuit 140 captures a sound field environment image of the target space 170, the identification circuit 134 analyzes the sound field environment image, and recognizes one or more sound field environment images from the target space 170. environmental objects 175, and the spatial configuration information of these environmental objects 175. The spatial configuration information includes the size, position, shape, and appearance characteristics of the environmental objects 175 . The identification circuit 134 can also determine the acoustic attribute information of each environmental object 175 through artificial intelligence calculation or database search, such as the absorption rate and reflectance rate of sound. In a further derivative embodiment, the identification circuit 134 can also determine the application scene category of the target space 170 according to the sound field environment image. Application scenario categories can include theater, living room, bathroom, outdoor, etc. If the host device 130 knows the application scene category of the target space 170 , it can identify the environmental objects 175 in the target space 170 more quickly and reduce misjudgment. A related embodiment will be illustrated in FIG. 9 .

In the process 212, the control circuit 132 in the host device 130 can calculate the degree to which the playback effect of the speaker on the target listening point is affected by the environmental objects. The playback effect of a speaker on the target listening point can be defined as the equivalent volume (Equal loudness) or sound pressure level (Sound Pressure Level; SPL) received from the speaker at the target listening point. An equal loudness curve (Fletcher-Munson Curve) is defined in the ISO226 standard, indicating that the equivalent volume perceived by the user under different sub-bands actually corresponds to different sound pressure values. In one embodiment, the control circuit 132 can use the equal loudness curve as a standard reference for the playback effect, and calculate the received sound pressure values for the target listening point in various situations. The control circuit 132 can use the spatial configuration information and the acoustic property information of the environmental object 175 to evaluate the interference caused by the environmental object 175 to the target listening point, so as to further calculate a method for eliminating the interference. The influence of the spatial configuration information and attribute information of the environment object 175 includes many kinds of situations. For example, the larger the volume of the environmental object 175 , the larger the interference coefficient to the target listening point may be. Whether the location of the environmental object 175 blocks the user 180 and the speaker also determines the degree to which the speaker is affected. The environmental object 175 may absorb sound or bounce sound depending on the material. Therefore, the control circuit 132 needs to select corresponding parameters or formulas for different acoustic properties to calculate the degree of speaker impact.

In the process 214 , the control circuit 132 in the host device 130 adopts the channel floor compensation operation to calculate the required output compensation value of the channel audio of each speaker. When the channel floor compensation operation judges the playback effect on the target listening point, it is calculated separately based on the audio of each channel. Taking a first channel audio 112 played by a first speaker 110 among multiple speakers as an example, before the first channel audio 112 is transmitted to the target listening point through the air, it may be disturbed by an environmental object 175 and lose energy . The change of the position of the target listening point will also affect the sound pressure value generated by the first channel audio 112 at the target listening point. Through the channel floor compensation operation, the control circuit 132 can calculate the change amount of the sound pressure value of the first channel audio 112 at the target listening point. The control circuit 132 of this embodiment adds an output compensation value to the first channel audio 112 to offset the change in the sound pressure value, so that the first channel audio 112 received by the target listening point returns to the state before being affected . In other words, the output compensation value has the same numerical value as the change amount of the sound pressure value, but opposite positive and negative polarities.

In the process 216, the control circuit 132 adjusts and outputs the channel audio to the speaker according to the output compensation value. Since the adjusted channel audio has offset the effect of the displacement of the user 180 in the target space 170 and the interference caused by the environmental objects 175 , the listening effect experienced by the user 180 remains consistent. Taking the first speaker 110 and the second speaker 120 in the target space 170 as an example, the control circuit 132 calculates and adjusts the sound pressure values of different sub-bands in the first channel audio 112 and the second channel audio 122, thereby canceling The equivalent volume deviation felt by the user 180 due to movement. On the other hand, the control circuit (132) compensates the first-channel audio 112 and the second-channel audio 112 correspondingly according to the amount of change in the sound pressure value of the target listening point caused by the position, size, and acoustic attribute information of the environmental object 175. Audio 122.

In the process 218 , each speaker correspondingly receives channel audio from the host device 130 through the audio transmission circuit 135 . Taking the first speaker 110 and the second speaker 120 in the target space 170 as an example, the control circuit 132 respectively outputs the first channel audio 112 and the second channel audio 122 to the corresponding first speaker 110 and The second horn 120. Therefore, the first speaker 110 and the second speaker 120 play the adjusted first-channel audio 112 and second-channel audio 122 correspondingly, so that the target listening point where the user 180 is located can obtain an optimized listening effect. For ease of illustration, only two speakers and one environmental object 175 are shown in the embodiment of the target space 170 in FIG. 1 . However, it can be understood that, in practice, the host device 130 may include more than two speakers, and the number of environmental objects 175 is not limited to one. In a further derivative embodiment, the audio frequency range that each speaker is good at outputting may be different. For example, some speakers are high-pitched speakers, and some speakers are subwoofers. When the control circuit 132 adjusts the channel audio, it can further adjust the corresponding output first channel audio 112 and second channel audio 122 according to the characteristics of different speakers.

The following uses FIG. 3 to illustrate an embodiment in which the sound system 100 dynamically tracks the user's position, uses a camera to acquire the configuration of the sound field environment, and compensates the output of the speaker through the operation of object floor compensation.

FIG. 3 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

In the flow chart of FIG. 3 , the process in the column of a specific device represents the process performed by the specific device. For example, the part marked in the column of "sensor circuit" is the process performed by the sensor circuit 140; the part marked in the column of "host device" is the process performed by the host device 130; The part marked in the "speaker" column is the process performed by the first speaker 110 and/or the second speaker 120; and so on for the rest. The aforementioned logic is also applicable to other subsequent flow charts.

The processes 202 , 204 , 206 , 208 and 210 in FIG. 3 are the same as those in the previous embodiment, and are not repeated for space saving.

When the audio system 100 of this embodiment completes the process 210, the control circuit 132 has tracked the position of the user 180 and assigned it as the target listening point, and also obtained the spatial configuration information of one or more environmental objects 175 in the target space 170 . Then, the operation of the object floor compensation is described in the following procedure to adjust the channel audio of each speaker.

The Object Based acoustic system originated from the mixing technology of virtual reality, which can use a limited number of physical speakers to simulate the effect of moving sound source objects. Some existing software products, such as Dolby Atmos, Spatial Audio Workstation, or DSpatial Reality, are object-based acoustic systems. The user can define the moving track of the sound source object in a virtual space through a man-machine interface. The object-based system can use physical speakers to simulate the sound effects of the sound source objects in the virtual space. Users at the target listening point can truly feel the sound source object moving in space.

The object base acoustic system is based on the array operation of a large number of acoustic parameters. Each audio source object has a metadata for describing the type, location, size (length, width, height), divergence, etc. of the audio source object. After the object-based array operation, the sound represented by an audio source object will be assigned to one or more speakers to play together, and each speaker will play a part of the sound of the audio source object. In other words, the object-based array operation can use multiple speakers to simulate the spatial effect of an audio source object. The embodiment of FIG. 3 proposes an object base compensation operation based on the object base acoustic system to solve the traditional playback effect problem.

In the process 312 , the control circuit 132 in the host device 130 creates an object-based compensation sound source object according to the environment object 175 . In practice, the control circuit 132 first maps the target space 170 to an object base space of the virtual reality, and then correspondingly creates a compensation sound source object in the object base space according to the environment object 175 for generating offset The sound source effect of the environment object 175 . For the user 180 at the target listening point, the existence of the environmental object 175 can also be compared to an audio source object. In practical applications, the environmental object 175 may reflect the sound from a speaker to the target listening point. Environmental objects 175 may also block or absorb some sound, attenuating the sound emitted by a speaker to the target listening point. In other words, after the control circuit 132 of this embodiment compares the environmental object 175 to a sound source object, it can correspondingly create a negative sound source object with an opposite sound source effect in the base space of the object as a means of canceling the interference. The sound source effect described in this embodiment may be a sound pressure value, an equivalent volume, or a gain value generated for a target listening point.

In the process 314, the host device 130 substitutes the compensated audio source object into the object base compensation operation to generate channel audio. The object-based compensation operation can use the object-based array calculation module in the existing object-based acoustic products to perform a large number of array calculations related to the acoustic interaction based on the metadata of the sound source object. For example, a metadata of the compensating sound source object includes: the coordinate position, size, and reflection rate and absorption rate of the environmental object 175 . The control circuit 132 performs an object base compensation operation according to the target listening point and the relay data to cancel the interference of the environmental object 175 on the target listening point and generate the first channel audio 112 and the second sound optimized for the target listening point. Road News 122.

In one embodiment, the object floor compensation operation is performed on a plurality of frequency sub-bands respectively. Due to the characteristics of sound transmission, the impact of the sound pressure value on each sub-band on the equivalent volume is different. Taking the influence of the first channel audio 112 generated by the first speaker 110 on the environmental object 175 as an example, the control circuit 132 of this embodiment can base on the coordinate position, size, and reflectivity and absorption rate of the environmental object 175, A sound source effect passively generated by the environmental object 175 affected by the first channel audio 112 is calculated on the plurality of frequency sub-bands respectively. Then the control circuit 132 creates the compensated sound source object according to the sound source effect. In this embodiment, the compensating sound source object is correspondingly established according to the environmental object 175, and its metadata has the same coordinate position, size, and sound reflection rate and absorption rate as the environmental object 175, but the generated sound source effect is The sign is the opposite of that of the environment object 175 .

The audible range of the human ear is known to be between 20 Hertz (Hz) and 20000 Hz. In this embodiment, the audible range of the human ear can be divided into a plurality of sub-band intervals for compensation respectively. The interval size of each sub-band may be an exponential interval. For example, the index range with the base 10 can divide the sound signal into multiple sub-band ranges such as 10 Hz to 100 Hz, 100 Hz to 1000 Hz, and 1000 Hz to 10000 Hz. In other embodiments, the exponent interval may be divided into base 2 or base 4 according to the fineness requirement of playback quality. In the field of audio processing, the equalizer (Equalizer) already has a processing technology for cutting multiple sub-bands, so it will not be explained in depth here.

After the control circuit 132 obtains the negative sound source effect of the compensated sound source object, an object base compensation operation is performed, and the negative sound source effect is correspondingly mixed into the first channel audio 112 and the second sound source according to the ratio determined by the mixing operation result. channel audio 122, thereby canceling the interference of the environmental object 175 to the target listening point. The operation of object base compensation will be described in detail in the embodiments of FIG. 11 to FIG. 13 .

In the process 316 , the host device 130 outputs the first channel audio 112 and the second channel audio 122 to the first speaker 110 and the second speaker 120 according to the operation result of the process 314 . The process 316 in FIG. 3 is different from the process 216 in the embodiment in FIG. 2 . FIG. 2 is to calculate the compensation value for the existing channel audio to adjust the existing channel audio. When the map control circuit 132 performs the object floor compensation operation, it directly calculates the channel audio corresponding to each speaker according to all the relay data at one time. The object base compensation operation mixes the interference components that need to be canceled or compensated into the channel audio in the form of the compensation source. In other words, because the channel audio includes the compensation sound source emitted by the compensation sound source object, the user 180 cannot feel the influence caused by the existence of the environmental object 175 at the target listening point.

It can be seen from the process 316 that the object base compensation operation translates the target listening point and the environment object into the relay data of the object base acoustic system, and creates a compensation sound source object, which simplifies the calculation process of eliminating interference and optimizing the playback effect. It should be understood that the audio system 100 of this embodiment can use the sensor circuit 140 to track the position of the user 180 in real time or periodically to dynamically update the target listening point. The object floor compensation operation performed by the control circuit 132 can also synchronously update all the metadata related to the relative position of the target listening point in the target space 170 as the target listening point changes.

The process 218 in FIG. 3 is the same as that of the previous embodiment, and will not be described repeatedly in order to save space.

The following uses FIG. 4 to illustrate an embodiment in which the audio system 100 dynamically tracks the user's position, runs the configuration program to acquire the configuration of the sound field environment, and uses channel floor compensation to compensate the speaker output.

FIG. 4 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

In the flow chart of FIG. 4 , the process in the column of a specific device represents the process performed by the specific device. For example, the part marked in the column of "sensor circuit" is the process performed by the sensor circuit 140; the part marked in the column of "host device" is the process performed by the host device 130; The part marked in the "speaker" column is the process performed by the first speaker 110 and/or the second speaker 120; and so on for the rest. The aforementioned logic is also applicable to other subsequent flow charts.

The processes 202 , 204 , 206 , and 208 in FIG. 4 are the same as those in the previous embodiment, and will not be described again to save space.

When the audio system 100 of this embodiment completes the process 210, the control circuit 132 has tracked the position of the user 180 and assigned it as the target listening point, and also obtained the spatial configuration information of one or more environmental objects 175 in the target space 170 . The next step in the process is to use the object-based algorithm to adjust the channel audio of each speaker.

In order to eliminate the interference in the sound field environment, the audio system 100 needs to obtain spatial configuration information of various environmental objects 175 in the target space 170 .

In the process 410 , the control circuit 132 in the host device 130 can run a configuration program to obtain spatial configuration information of one or more environmental objects 175 in the target space 170 . In the previous embodiment, the host device 130 uses the sound field environment information captured by the sensor circuit 140 to automatically recognize the spatial configuration information of the environment object 175 . When running the configuration program, the host device 130 can use a man-machine interface circuit 133 to interact with the user, allowing the user to manually input the spatial configuration information of the environmental object 175 . The man-machine interface circuit 133 can provide a screen and an input method, allowing the user to define the spatial configuration information of various objects in the target space 170 in a two-dimensional plan view or a three-dimensional stereo view. The spatial configuration information of the environmental object 175 may include the relative position, size, name, and material type of the environmental object 175 in the target space 170 . In a further derivative embodiment, the user 180 can inform the host device 130 of the type of application scenario to which the current target space 170 belongs through the man-machine interface circuit 133 . In different application scenarios, such as open outdoor space, theater space, or bathroom, etc., the types of common environmental objects 175 are also different, and the sound field atmosphere felt by users is also different. Optimizing the sound field according to different application scenarios is also one of the important functions of the sound system 100 .

Different types of materials have different acoustic properties. When the host device 130 runs the configuration program, it further searches an object database according to the object name or material type input by the user to obtain the acoustic attribute information of the environmental object 175 , such as the absorption rate or reflectivity of sound. In this way, the host device 130 can calculate the degree to which the playback effect of each speaker on the target listening point is affected by the environmental object 175 according to the aforementioned spatial configuration information and acoustic attribute information in the subsequent process 212 . In a further derivative embodiment, the host device 130 can preferentially use the corresponding object database according to the application scenario category of the target space 170 to more quickly identify the environmental objects 175 in the target space 170 . A related embodiment will be illustrated in FIG. 9 .

The processes 212, 214, 216 and 218 in FIG. 4 are the same as those in the previous embodiment, and will not be repeated for space saving.

The embodiment in FIG. 4 illustrates that the sound system 100 can not only dynamically track the user's position, but also allow the user 180 to set the spatial configuration information of the environmental objects 175 in the target space 170 through a configuration program. The configurator provides a channel for manual input to make up for the insufficiency of the identification function. In addition to assisting the host device 130 to make more accurate judgments through active input, the user also has the opportunity to deliberately designate different application scenarios according to his own preferences, or deliberately set imaginary virtual sound source objects to change the playback effect. The host device 130 operates with channel floor compensation, and calculates the output compensation value corresponding to each speaker according to the spatial configuration information of the environmental object 175 in the target space 170 .

The following uses FIG. 5 to illustrate an embodiment in which the audio system 100 dynamically tracks the user's position, runs the configuration program to obtain the configuration of the sound field environment, and compensates the output of the speaker through the object base compensation operation.

FIG. 5 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

In the flow chart of FIG. 5 , the process in the column of a specific device represents the process performed by the specific device. For example, the part marked in the column of "sensor circuit" is the process performed by the sensor circuit 140; the part marked in the column of "host device" is the process performed by the host device 130; The part marked in the "speaker" column is the process performed by the first speaker 110 and/or the second speaker 120; and so on for the rest. The aforementioned logic is also applicable to other subsequent flow charts.

The processes 202 , 204 , 206 , 208 and 210 in FIG. 5 are the same as those in the previous embodiment, and will not be repeated for space saving.

Similar to the embodiment of FIG. 4 , the embodiment of FIG. 5 runs the same process 410 as that of FIG. 4 in order to eliminate interference in the sound field environment.

In the process 410 , the host device 130 runs a configuration program to obtain spatial configuration information of one or more environmental objects 175 in the target space 170 . In the embodiment of FIG. 4 , it is illustrated that the host device 130 can receive the spatial configuration information of the environmental object 175 manually input by the user through a man-machine interface circuit 133 . In a further derivative embodiment, the host device 130 can also use the communication circuit 136 to receive the spatial configuration information transmitted from the user equipment 150 or other devices. For example, the user equipment 150 may be a mobile phone running an application program to provide functions similar to the human-machine interface circuit 133 . The application allows the user to define the scope and size of the target space 170 , the position of each speaker relative to the target space 170 , the position, size, name and type of various environmental objects 175 , and even the location of the user 180 himself. The application program can also communicate with the control circuit 132 through the communication circuit 136 to perform various playback operations, such as playing, pausing, fast forwarding, adjusting volume, and the like. In addition, the user can set the application scene category of the target space 170 through the man-machine interface circuit 133 , so that the host device 130 can produce diversified playback effects on the target space 170 .

In a further derivative embodiment, the user equipment 150 connected to the host device 130 may be a virtual reality device or a game console. The user equipment 150 generates an audio signal to be played by the host device 130 . And the audio signal may include a virtual object moving in a virtual reality space, such as an airplane or a fire-breathing dragon. The user equipment 150 can transmit the metadata of these virtual objects to the host device 130 to become a part of the space configuration information of the environment objects in the target space 170 . In other words, the host device 130 can use the object-based acoustic system to treat virtual objects and physical objects equally. Through the object base compensation operation, the host device 130 can make the user feel that there is a virtual object in the target space 170 , and can also make the user not feel the interference of a physical object in the target space 170 . The implementation of the object base compensation operation is further described in the embodiments of FIGS. 11 to 13 .

When the audio system 100 of this embodiment completes the process 410, the control circuit 132 has tracked the position of the user 180 and assigned it as the target listening point, and also obtained the spatial configuration information of one or more environmental objects 175 in the target space 170 . Then in the processes 312 to 316, the host device 130 uses the object-based algorithm to adjust the channel audio of each speaker. Since the processes 312 to 316 and the process 218 are the same as those in the previous embodiment, the description will not be repeated in order to save space.

The embodiment in FIG. 5 illustrates that the audio system 100 can not only dynamically track the user's position, but also allow the user 180 to set the spatial configuration information of the environmental objects 175 in the target space 170 through a configuration program. The configuration program can be integrated with existing virtual reality technology to receive spatial configuration information of virtual objects. The audio system 100 converts physical environmental objects and virtual objects into metadata with the same format, and then applies all the metadata to the object-based array computing module of the existing object-based acoustic system to perform object-based compensation operations . In this way, the control circuit 132 does not need to develop additional computing modules for different objects, which can reduce costs and improve execution efficiency.

The following uses FIG. 6 to illustrate several implementation aspects of the sensor circuit and the compensation algorithm of the channel base.

FIG. 6 is a schematic diagram of a target space 600 according to the present invention, which is used to illustrate an embodiment of calculating an audio adjustment amount based on the position of the best listening point.

The sound system 100 of the present application uses the sensor circuit 140 to dynamically sense the target space 600 to generate sound field environment information. The sound field environment information mainly includes the location of the user 180, and may also include spatial configuration information of the environment objects. There are many options for the technical solution of dynamic sensing. For example, the sensor circuit 140 may be a combination of one or more of the camera 610, the infrared sensor 620, and the wireless detector 630, respectively arranged in different positions around the target space 600, providing a space with The in-depth sound field environment information helps the identification circuit 134 and the control circuit 132 in the host device 130 to track the position of the user 180 more efficiently. In this way, the recognition circuit 134 uses the sound field environment information provided by the sensor circuit 140 to not only recognize the position of the user 180, but also recognize the face facing direction, ear position, and even gestures or body postures. The control factors that can be applied to adjust the sound stage are thus enriched. For example, focus detection, sleep detection, gesture control, etc.

In the target space 600 of FIG. 6 , a first speaker 110 and a second speaker 120 are disposed. The channel floor compensation operation can calculate the output compensation value for each speaker separately. In a preset situation, the target listening point is located at the center of the target space 600 , that is, the first position 601 in FIG. 6 . The distance between the first position 601 and the first horn 110 and the second horn 120 is also R1. At this time, the first-channel audio 112 and the second-channel audio 122 played by the first speaker 110 and the first-channel audio 112 are also in a default state, and no compensation processing is required for the position.

When the user 180 moves from the first position 601 to the second position 602 along the moving track 173, the sensor circuit 140 detects the new position of the user 180, and assigns the target listening point of the audio system 100 as the second position. Location 602. At this time, the distance between the user 180 and the first speaker 110 is changed to R2, and the distance between the user 180 and the second speaker 120 is changed to R2'. For the user 180, the first speaker 110 becomes far away, so the received first channel audio 112 is attenuated due to the distance. On the contrary, the second speaker 120 gets closer, and the received second channel audio 122 is enhanced. In other words, the intensities of the first channel audio 112 and the second channel audio 122 received at the second location 602 are out of balance. In this embodiment, the channel-based algorithm is used to restore the listening effect received by the second location 602 to the same preset state as that of the first location 601 . In other words, the control circuit 132 compensates the first-channel audio 112 and the second-channel audio 122 outputted by the first speaker 110 and the second speaker 120 , so as to offset the listening effect deviation caused by the movement of the user 180 . The target space 600 shown in FIG. 6 is not limited to a multi-speaker environment in a horizontal configuration. In a three-dimensional sound field environment with upper speakers and lower speakers, the problem of distance deviation also occurs. For example, if the user changes from a standing position to a sitting position, the user moves away from the upper horn and approaches the lower horn.

In order to obtain a better compensation effect, this embodiment uses an equivalent volume (Equal Loudness) as a calculation standard. For example, this embodiment can calculate the sound pressure value to be compensated at the target listening point according to the equal loudness curve defined in the ISO226;2003 protocol. Each channel audio is divided into multiple sub-bands and processed separately. In addition, as the distance between the user 180 and the speaker is different, the adopted sound field formula is also different. Since the equal loudness curve defines the linear relationship between the equivalent volume and the sound pressure value, and the equivalent volume and the "gain value" in "decibels" have a linear correspondence. Therefore, it is not limited in this embodiment to adjust with any one of the equivalent volume, the sound pressure value, or the gain value as the unit.

In the acoustic system 100, the space in which sound is transmitted due to air vibration is called a sound field. Due to the existence of reflection, the sound field can be divided into many types in a closed room. (1) Near Field: When the user 180 is located relatively close to the sound source, the physical influence of the sound source (such as pressure, displacement, vibration) will enhance the sound. (2) Reverberant Field: The wave superposition effect is caused by the sound reflected by the object. (3) Free Field (Free Field): A sound field that is not disturbed by the aforementioned near sound field and reflected sound field. The above reflected sound field and free sound field can be collectively referred to as far field (Far Field).

In many audio systems today, near and far soundstages are defined in different ways. For example, assuming that R is the distance between the speaker and the user (meters), L is the surface width of the speaker (meters), and λ is the representative wavelength (meters) of a sub-band signal, the conditions for satisfying the far sound field include the following types:

R＞＞λ/2π （1）

R＞＞L （2）

R>>πL ² /2λ (3)

Take the first speaker 110 in FIG. 1 as an example. When the distance between the target listening point and the first speaker 110 is greater than the wavelength of the sub-band signal or a certain ratio of the size of the first speaker 110 , the audio system 100 determines that the sound field type is a far sound field. When the distance between the target listening point and the first speaker 110 is smaller than the wavelength of the sub-band signal or the specific ratio of the size of the first speaker 110 , it is determined that the sound field type is a near sound field. In a relatively simple implementation, the audio system 100 can define the double value (2λ) of the wavelength corresponding to the central frequency of a sub-band signal as the boundary point between the far sound field and the near sound field of the sub-band signal.

In the far sound field, the relationship between the sound pressure value change and the distance change of a sub-band signal received by the user 180 from the speaker is as follows:

SPL2 = SPL1 - 20 log ₁₀ (R2/R1) (4)

Among them, SPL2 is the sound pressure value of the sub-band signal received at the new position, SPL1 is the sound pressure value of the sub-band signal received at the original position, R2 is the distance between the new position and the speaker, and R1 is the distance between the original position and the speaker. Speaker distance.

It can be seen from formula (4) that the difference between SPL1 and SPL2 is the part that needs to be compensated for the speaker.

SPL2' = SPL2 + 20 log ₁₀ (R2/R1) = SPL1 (5)

Wherein, SPL2' is the sound pressure value of the sub-band signal received at the new position after compensation. It can be known from formula (5) that this embodiment compensates for the changed part.

In the near sound field, the relationship between the sound pressure value of the sub-band signal received by the user 180 from the speaker and the distance change is as follows:

SPL2 = SPL1 - 10 log ₁₀ (R2/R1) (6)

SPL2' = SPL2 + 20 log ₁₀ (R2/R1) = SPL1 (7)

It can be seen from formulas (6) and (7) that the sound attenuation change rate of the near sound field is slower than that of the far sound field, and other calculation logics are the same.

It is understandable that there may be exceptions to the above formula in some special cases. For example, when the user 180 moves from the first position 601 to the second position 602 and gets close to the second speaker 120, the distance between the user 180 and the second speaker 120 decreases from R1 to R2', which may make the formula (7 ) becomes a negative value. However, the sub-band signal output by the second speaker 120 cannot be a negative value, and the minimum can only be reduced to the lowest audible value for human ears. For example, make the sound pressure value of the sub-band signal output by the second speaker 120 zero. On the other hand, when the user 180 moves from the first position 601 to the second position 602 away from the first speaker 110 , the distance between the user 180 and the first speaker 110 increases from R1 to R2 . The maximum output limit of the first speaker 110 may not be able to satisfy formula (5). At this time, the sound system 100 may issue a prompt of exceeding the limit to the user 180 .

The embodiment of Fig. 6 highlights the following advantages. Through the channel-based compensation algorithm, the user's sweet spot is not affected by movement. The calculation method of the channel basis is simple and efficient, and is applicable in most object spaces 600 .

FIG. 6 has illustrated the sound compensation method according to the movement of the user 180 . The sound compensation method according to the environmental object 175 will be described below with FIG. 7 . The acoustic property information of the environmental object 175 includes reflectance and absorption of sound. In this embodiment, according to the spatial configuration information of the environmental object 175 , an appropriate calculation method is used to calculate the acoustic impact of the environmental object.

FIG. 7 is a schematic diagram of a target space 700 according to the present invention, which is used to illustrate an embodiment of calculating an audio adjustment amount according to an absorption rate of an environmental object.

FIG. 7 shows an environmental object 175 located between a first speaker 110 and a user 180 in a target space 700 . For example, environmental object 175 may be a sofa or a pillar. In this case, the environmental object 175 may attenuate the listening effect of the user 180 due to occlusion. In other words, the sound pressure received by the user 180 from the first speaker 110 will be blocked or absorbed. When the control circuit 132 interprets this layout situation through the spatial configuration information, it uses the absorption rate of the environmental object 175 to calculate the playback effect of the first speaker 110 at the target listening point (the position of the user 180) affected by the environment. The degree of influence of the object 175 is used to determine the equivalent volume, sound pressure value or gain value that the first channel audio 112 needs to output.

In one embodiment, the sound loss absorbed by the environmental object 175 can be calculated according to the sound pressure value received by the environmental object 175 from the first speaker 110:

A _t [n]=R[n]*SPL _t (8)

Wherein, n represents the number of the sub-band. That is, the first channel audio 112 output by the first speaker 110 can be divided into a plurality of sub-bands and calculated separately. _At [n] represents the gain value of the nth sub-band detected at time point t. R[n] represents the absorption rate of the nth sub-band. SPL _t represents the sound pressure value received by the environmental object 175 from the first speaker 110 at the tth time point. The time point t may represent the time difference when the sound is transmitted from the first speaker 110 to the environmental object 175 .

It can be known from the formula (8) that A _t [n] represents the gain value of a first channel audio 112 absorbed by the environmental object 175 on the nth sub-band, and also represents the nth subband of the first channel audio 112 desired output offset value. Therefore, when the first speaker 110 generates the first channel audio 112 , the control circuit 132 increases the gain value of the nth sub-band of the first channel audio 112 by the gain value _At [n].

There may be many situations where the environmental object 175 is located between the first speaker 110 and the user 180 . In this embodiment, whether the line of sight of the first speaker 110 and the user 180 is blocked is the main basis, or further, the line of sight of the first speaker 110 and the ear of the user 180 is used as the judging criterion. It can be understood that SPL _t itself is a function related to the distance and time between the environmental object 175 and the first speaker 110, and the calculated A _t [n] has a degree of influence on the user 180 that is related to the environmental object 175. A function related to the distance and time of the user 180 . After considering the arrangement of different angles and the relationship between distance and distance, a variety of non-linear correlations are involved. The present application does not limit the derivative changes of formula (8), such as adding other weight coefficients, parameters, and offset correction values depending on the actual situation. For example, a sofa may be placed between the user 180 and the first speaker 110 . Although the sofa does not block the visible line, it may affect the sound pressure received by the user 180 from the first speaker 110 . The control circuit 132 can use the interpolation method or other correction formulas according to formula (8) to make the compensation result more in line with requirements.

FIG. 8 is a schematic diagram of a target space 800 according to the present invention, which is used to illustrate an embodiment of calculating an audio adjustment amount based on reflectivity of environmental objects.

FIG. 8 shows a user 180 located between a first speaker 110 and an environmental object 175 in a target space 800 . Environmental object 175 may be a wall, ceiling or floor. In this case, the environmental object 175 will bounce the first channel audio 112 output by the first speaker 110 to the user 180 . In other words, the sound pressure received by the user 180 from the first speaker 110 will be superimposed or interfered. When the control circuit 132 interprets the layout situation through the spatial configuration information, it uses the reflectivity of the environmental object 175 to calculate the playback effect of the first speaker 110 at the target listening point (the position of the user 180) affected by the environment. The degree of influence of the object 175 is used to determine the equivalent volume, sound pressure value or gain value that the first channel audio 112 needs to output.

In this embodiment, the influence caused by the environmental object 175 can also be calculated according to formula (8), but R[n] is changed to represent the reflectivity of the environmental object 175 on the nth sub-band.

The operation result A _t [n] of the formula (8) may represent the component of the first channel audio 112 reflected by the environmental object 175 to the user 180 on the nth sub-band. Therefore, when the control circuit 132 generates the first channel audio 112 through the first speaker 110, it can appropriately reduce the gain value of the first channel audio 112, so that the total amount received by the user 180 from the first speaker 110 and the environmental object 175 The sound pressure value is maintained at the preset level value.

Similar to the embodiment in FIG. 7 , the situation in FIG. 8 where the user 180 is located between the first speaker 110 and the environmental object 175 may have many changing scenarios. In this embodiment, it is mainly based on whether the visible line of the first speaker 110 and the environmental object 175 is blocked by the user 180 . However, in practice, walls, ceilings, and floors are reflective at any angle. Therefore, the calculation formula of this embodiment is not limited to the formula (8), and other nonlinear compensation calculation methods may be further derived based on the arrangement status and the distance relationship. For example, the target space 800 can be classified into different application scenarios, such as living room, study room, bathroom, theater, or outdoor, due to characteristics such as wall, ceiling, floor material, and room size and shape. The host device 130 can first classify the application scenarios to which the target space 800 belongs, and then adopt corresponding parameters or formulas respectively.

The embodiment of Figures 7 and 8 highlights the following advantages. The effect of the environmental object 175 on the listening effect of the user 180 is eliminated through the channel floor compensation operation. The channel base compensation operation can flexibly apply different object acoustic properties according to the configuration of environmental objects, and can effectively deal with the optimization problems of various complex environments.

To sum up, the identification circuit 134 can receive the data from the sensor circuit 140 to identify the location of the user 180 in the target space 170 , so that the control circuit 132 dynamically assigns the location of the user 180 as the target listening point. The compensation made by the control circuit 132 for the movement of the target listening point has been described in the embodiment of FIG. 6 and formulas (4) to (7). The compensation made by the control circuit 132 for the disturbance of the environmental object 175 has been described in FIGS. 7 to 8 and formula (8). These two compensation operations can be performed separately and applied to the channel audio. In other words, the final output optimized channel audio includes the compensation value for the movement of the target listening point, and also includes the compensation for the interference of the environmental object 175 .

The identification circuit 134 identifies the location of the user 180 according to the sound field environment information captured by the sensor circuit 140 . The identification process may also include the identification of the application scene, so as to help speed up the subsequent operation of the control circuit 132 . The following uses FIG. 9 to illustrate the process of the host device 130 identifying objects according to the application scenario category.

FIG. 9 is a flow chart of identifying an object by the host device 130 according to an embodiment of the present invention. The acoustic properties of environmental objects that appear in different application scenarios usually have significant group correlations, and the sound field reflection coefficients caused by the surrounding environment materials or room sizes are also different. Therefore, distinguishing the types of application scenarios in advance helps the audio system 100 to improve the efficiency of sound field optimization. It can be understood that, each process in FIG. 9 is executed by the host device 130 , but is not limited to be executed by a single circuit or module therein, and may also be a coordinated operation of multiple circuits.

In the process 902 , the host device 130 acquires the application scenario category of the target space 170 . The host device 130 can obtain the application scenario category in several different ways. In one embodiment, the identification circuit 134 in the host device 130 can determine an applicable application scenario category according to the sound field environment information when identifying the sound field environment information provided by the sensor circuit 140 . In another embodiment, when the control circuit 132 in the host device 130 obtains the spatial configuration information of the environmental objects by running a configuration program through the man-machine interface circuit 133, it also obtains the application defined by the user 180 through the configuration program. Scene category. In a further derivative embodiment, the control circuit 132 in the host device 130 can obtain information related to the application scenario type from a user device 150 through the communication circuit 136 .

In the process 904, in order to speed up the query of the environmental objects and improve the accuracy, the host device 130 preferentially selects the relevant object database according to the type of the application scenario. Object databases are usually pre-built collections of data that can be provided by a variety of different channels. For example, the storage circuit 131 in the host device 130 can pre-store one or more object databases corresponding to different application scenarios. In another embodiment, the host device 130 can use the communication circuit 136 to connect to a remote database 160 . The remote database 160 may include multiple object databases corresponding to different application scenarios. Each object database contains appearance feature information and acoustic attribute information of multiple environmental objects.

After the host device 130 obtains the application scenario category in the process 902, it can preferentially select and use an object database related to the application scenario category from the storage circuit 131 or the remote database 160 for subsequent identification of environmental objects. In one embodiment, the recognition circuit 134 analyzes the sound field environment information provided by the sensor circuit 140 to obtain one or more object shape feature information, and searches the object database according to the object shape feature information to identify Output the environmental objects that match the shape characteristic information of the object, including name, absorption rate and reflectance rate. In another embodiment, the control circuit 132 executes the configuration program, and uses the man-machine interface circuit 133 to obtain the name of an environmental object. The control circuit 132 searches the object database according to the name of the environmental object 175 to obtain the corresponding absorptivity and reflectivity of the environmental object.

In a further derivative embodiment, the parameters used in the search process may be combined in multiples. For example, the identification circuit 134 can obtain external characteristics such as material, size, and shape of the environmental object 175 during the process of analyzing the environmental information of the sound field. The identification circuit 134 sends the external feature information to the object database for multi-conditional cross-comparison to obtain a list of candidate objects sorted according to matching scores. If in the process of searching the object database, the information of the application scene category is used as the search condition, it will help to narrow the possible range, speed up the identification, and improve the accuracy.

In the process 906 , the control circuit 132 searches the absorptivity and reflectivity of the environmental object from the object database selected in the process 904 . In practice, the acoustic property information of the environmental objects stored in the object database is not limited to be divided into multiple independent object databases for storage. The object database can be an associative database, which contains various fields connected together in the form of correlation coefficients. For example, the fields of the object database may include object name, application scene type, material, absorption rate, reflectance rate, and even external characteristics such as shape, color, and luster. The field value corresponding to each environmental object is not limited to a one-to-one relationship, but can be one-to-many or many-to-one. The value stored in each column is not necessarily an absolute value, but a range value or a probability value. In a further derivative implementation, the object database can be an adaptive database that can be continuously iteratively modified by machine learning. The user 180 can feed back the preferred setting value through the man-machine interface circuit 133 to train the object database.

In the process 908, the control circuit 132 adjusts the channel audio in the multiple sub-bands according to the search result and the configuration of the environmental objects. The acoustic properties of environmental objects 175 may vary significantly across different frequency bands. For example, a sofa may absorb a lot of high-frequency signals, but it does not affect the penetration of low-frequency signals. Therefore, the absorptivity or reflectivity found from the object database can be array values corresponding to multiple sub-bands, or a frequency response curve. The size of the sub-bands or the manner of partitioning may be determined according to design requirements, and is not limited in this embodiment. The control circuit 132 separately adjusts the gain values of the channel audio in the multiple sub-bands, which can be compared to the concept of an equalizer or a filter in practice. In other words, the control circuit 132 can implement an equalizer for each speaker in the audio system 100, and customize the equalizer according to the output compensation value calculated in the foregoing embodiment, so that the corresponding channel audio can be adjusted. . A further embodiment for calculating the output compensation value will be illustrated in FIG. 10 .

In the process 910 , the control circuit 132 outputs the adjusted channel audio to the corresponding speaker through the audio transmission circuit 135 . The implementation of the audio transmission circuit 135 has been introduced in FIG. 1 , and will not be repeated here.

The embodiment of Fig. 9 highlights the following advantages. The operation of object recognition can refer to the application scenario category (automatic recognition or manual input) to increase the recognition efficiency. The object database adopts a scalable architecture, and the recognition ability can be continuously enhanced for a long time under the feedback of cloud big data services and machine learning. The sound system 100 can apply the concept of an equalizer to divide the channel audio into a plurality of sub-bands to be processed separately, so that the final synthesized sound quality can be effectively improved.

The following uses FIG. 10 to further illustrate how the control circuit 132 calculates the output compensation value of each channel according to the spatial configuration information of the environmental object 175 .

FIG. 10 is a flowchart of an audio processing method according to an embodiment of the present invention, illustrating an embodiment of calculating an output compensation value according to the positional relationship of environmental objects. The process in FIG. 10 is mainly executed by the control circuit 132 in the host device 130 .

In the process 1002, the control circuit 132 determines the relative positional relationship of the environmental object, the target listening point and the speaker. Multiple speakers and multiple environmental objects 175 in the target space 170 can be arranged and combined with the target listening point to form multiple sets of positional relationships. Each set of positional relationships includes a speaker, an environmental object 175, and a target listening point. The control circuit 132 checks and judges each positional relationship combination in the target space 170 and calculates a corresponding output compensation value. The following takes one set of positional relationships in the sound system 100 as an example to illustrate how the control circuit 132 compensates for the interference caused by an environmental object 175 to a speaker at a target listening point.

In the process 1004, the control circuit 132 determines whether the environmental object 175 is between the target listening point and the speaker. The position of the environmental object 175 in the target space 170 can also be obtained by the identification circuit 134 , or obtained by the man-machine interface circuit 133 through a configuration program. After synthesizing the above information, the control circuit 132 can determine the relative positional relationship between each environmental object 175 , the target listening point, and each speaker, and perform corresponding compensation calculations for each speaker. The situation to be judged in the process 1004 is the situation shown in FIG. 7 . If the condition is met, go to process 1008. If not, go to process 1006.

In the process 1006, the control circuit 132 determines whether the target listening point is located between the environmental object 175 and the speaker. The situation to be judged in the process 1006 is the situation shown in FIG. 8 . If the conditions are met, go to process 1010. If the situation is not met, go to process 1012.

In the process 1008, the control circuit 132 uses the absorption rate of the environmental object 175 to calculate the output compensation value of the channel audio. In a preferred embodiment, the output compensation value of the channel audio of the speaker is divided into a plurality of sub-bands and calculated respectively. For detailed calculations, refer to the target space 700 in FIG. 7 and formula (8). The control circuit 132 can search the absorption rate of the environmental object 175 from the object database, and substitute it into the formula (8) to obtain the output compensation value.

In the process 1010 , the control circuit 132 calculates the output compensation value of the channel audio by using the reflectivity of the environmental object 175 . Referring to the target space 800 and formula (8) in FIG. 8 , the control circuit 132 can search the reflectivity of the environmental object 175 from the object database, and substitute it into the formula (8) to obtain the output compensation value.

It can be understood that the output compensation value calculated according to the absorption rate of the environmental object 175 may amplify the adjusted gain value, sound pressure value, or equivalent volume of the channel audio to compensate for the absorbed energy. In contrast, the output compensation value calculated according to the reflectivity of the environmental object 175 may reduce the adjusted gain value, sound pressure value, or equivalent volume of the channel audio to balance the reflected energy. In other words, the signs of the output compensation values calculated according to the absorptivity and reflectivity are usually opposite to each other.

In the process 1012, if the environmental object 175 does not meet the conditions of the process 1004 and does not meet the conditions of the process 1006, then the control circuit 132 can determine that the environmental object 175 is in a position that will not affect the playback of the speaker to the target listening point. In this case, the control circuit 132 may not calculate the impact of the environmental object 175 on the speaker and the target listening point for the set of positional relationships. However, it should be understood that a target space 170 usually contains multiple speakers. The environmental object 175 will not affect the playback of one of the speakers to the target listening point, but may still affect the playback of other speakers to the target listening point. In other words, the control circuit 132 needs to perform the process shown in FIG. 10 for each set of positional relationships in the target space 170 .

In some specific cases, the existence of the environment object 175 can be ignored directly. For example, if the reflection rate or absorption rate of the environmental object 175 to sound is less than a certain threshold, it means that its existence in the target space 170 can be ignored. On the other hand, if the control circuit 132 determines that the volume of the environmental object 175 is smaller than a certain size, the existence of the environmental object 175 may also be ignored.

In a further derivative embodiment, if more than one user is detected in the target space 170, then the judgment of the target listening point can be based on the location centers of multiple users, or selectively based on one of the users s position. As for the users who are not selected as target listening points, the host device 130 can compare them to environmental objects, and process them according to the embodiments in FIGS. 7 to 8 .

The embodiment of Fig. 10 highlights the following advantages. The embodiment in FIG. 10 continues the processing methods in FIG. 7 and FIG. 8 , and simplifies complex environmental problems into multiple linearly related problems and solves them respectively. The environment object 175 for a specific situation can also be ignored to simplify the calculation complexity.

FIG. 11 is a schematic diagram of a target space 1100 of the present invention, which is used to illustrate an embodiment of optimizing a sound field by operating object floor compensation.

In the target space 1100 , a plurality of speakers are included, such as a first speaker 1110 , a second speaker 1120 , a third speaker 1130 and a fourth speaker 1140 . In the case that the audio system 100 operates based on the object basis compensation operation, the control circuit 132 logically regards the target space 1100 as a spatial coordinate system. The spatial coordinates may be two-dimensional plane coordinates or three-dimensional plane coordinates. For ease of description, FIG. 11 is illustrated in a two-dimensional plane coordinate manner including an X axis and a Y axis.

In the target space 1100, the user 180 is located at the origin P0. The control circuit 132 assigns the user 180 as the target listening point. As described in the embodiment of FIG. 3 , the object-based acoustic system is based on the array operation of a large number of acoustic parameters. Each audio source object has a metadata for describing the type, location, size (length, width, height), divergence, etc. of the audio source object. After the object-based calculation, the sound represented by an audio source object will be assigned to one or more speakers to play together, and each speaker will play a part of the sound of the audio source object. In other words, object-based acoustic systems can utilize multiple speakers to simulate the physical presence of an audio source object. For example, through the object base compensation operation, the user 180 at the target listening point can hear a virtual sound source object 1105 moving from the first position P1 to the new first position P1' along the movement track 1103 .

The object floor compensation operation of this embodiment can optimize the channel audio output from all the speakers for the target listening point. The object base compensation operation utilizes the array calculation module in the existing object base acoustic system to parameterize various distance factors and sound field types, and perform calculations similar to formulas (4) to (7). For the audio system 100, the host device 130 only needs to apply the location information of the user 180 to the object floor compensation operation, so that the channel audio output by all the speakers can be optimized for the target listening point.

In one embodiment, the control circuit 132 can define the target listening point as the origin of the entire spatial coordinate system. When the user 180 moves, the entire spatial coordinate system moves along with the origin. In other words, the position of the virtual sound source object 1105 relative to the origin remains unchanged. When the control circuit 132 plays the effect of the virtual sound source object 1105 through the object base compensation operation, the relative position of the virtual sound source object 1105 felt by the user 180 will not change as the user 180 moves.

In the target space 1100 of this embodiment, there may be environmental objects 175 that will substantially affect the listening effect of the user 180 . The control circuit 132 can obtain the spatial configuration information of the environmental objects in the target space 1100 through the process 902 in FIG. 9 , and know that the environmental object 175 is located at the second position P2. When the user 180 moves, the origin of the entire spatial coordinate system changes with the user 180 . Although the environment object 175 has not moved, its relative position to the origin has changed. Therefore, it can be understood that in the moved spatial coordinate system, the coordinate value of the environmental object 175 is moved in the opposite direction.

In order to counteract the interference caused by the environmental object 175 to the user 180 , the control circuit 132 of this embodiment establishes an object-based compensation sound source object according to the environmental object 175 . The metadata of the compensation sound source object includes: the coordinate position, the size, and the reflection rate and absorption rate of the sound source of the environmental object 175 . The reflection rate and absorption rate of the environmental object 175 to sound can be obtained by the process 906 in FIG. 9 . The compensated sound source object will be regarded as the negative sound source object of the environmental object 175 and applied to the object base compensation operation to become a virtual sound source capable of offsetting the environmental object 175 .

It can be understood that the essence of the compensation sound source object is the negative sound source object corresponding to the environment object 175 , and its position overlaps with the environment object 175 , so it is not marked in FIG. 11 . Additionally, the four-speaker configuration of target space 1100 is just an example. In the actual application of the sound system 100 , the number of speaker configurations can be more, even including the stereo configuration of the upper speaker and the lower speaker. This description does not limit other possible configurations.

The embodiment of Fig. 11 illustrates the advantages of object floor compensation operation. The control circuit 132 converts the information of the target space 1100 into the form of the space coordinate system, which can simplify the complex multi-object interaction operation into the array operation of the relay data. The position of the moving user 180 is set as the origin of the spatial coordinate system, so that the processing of the virtual object is not affected by the movement of the user 180 at all, and the calculation process is simplified. This embodiment also proposes the concept of compensating the sound source object, and directly applies the object base compensation operation to offset the interference of environmental objects, eliminating the need for complex multi-channel interactive calculations.

The following uses FIG. 12 to illustrate the convenience and possible derivative applications of the object base compensation operation.

FIG. 12 is a schematic diagram of a target space 1200 of the present invention, which is used to illustrate an embodiment of optimizing a sound field by performing object floor compensation operation.

The target space 1200 may contain multiple speakers, such as the first speaker 1210 , the second speaker 1220 , the third speaker 1230 , the fourth speaker 1240 , the fifth speaker 1250 and the sixth speaker 1260 , arranged as a strip sound field. Each horn corresponds to an ID. When the user 180 is at the first position P1, the metadata of a virtual sound source object (not shown) is mapped to the IDs of the first speaker 1210 and the second speaker 1220 . After the control circuit 132 performs the object base compensation operation, the first speaker 1210 and the second speaker 1220 will play the first channel output 1212 and the second channel output 1222, so that the user 180 can feel the existence of the virtual audio source object. When the user 180 moves to the second position P2 along the moving track 1203 , the control circuit 132 maps the metadata of the virtual sound source object to the fifth speaker 1250 and the sixth speaker 1260 through recalculation of the target listening point. After the control circuit 132 performs the object base compensation operation, the fifth speaker 1250 and the fifth speaker 1250 will play the fifth channel output 1252 and the sixth channel output 1262, so that the user 180 can feel that the virtual sound source object still exists The left and right sides of the user 180 do not move away with the movement of the user 180 .

This embodiment mainly illustrates the flexible application and convenience of the object base compensation operation. In many special cases, only a small amount of calculation is required to complete the optimization of the sound field. For example, if the user 180 is located in a sphere-shaped sound field, the control circuit 132 only needs to perform calculations on rotating coordinates, so that the user 180 can experience consistent sound field effects when facing various directions.

The following summarizes the basic logic of the control circuit 132 when performing the object base compensation operation with FIG. 13 .

FIG. 13 is a flow chart of object base compensation operation according to an embodiment of the present invention, illustrating the concept of creating a compensated audio source object.

In the process 1304 , the control circuit 132 establishes a corresponding compensation sound source object according to the environmental object 175 . For the user 180 at the target listening point, the presence of the environmental object 175 is a physical sound source. Environmental objects 175 may reflect sound from a speaker to the target listening point. Environmental objects 175 may also block or absorb some sound, attenuating the sound emitted by a speaker to the target listening point. The compensation sound source object is a negative sound source object created for the environment object 175. When the host device 130 substitutes the compensation sound source object into the object base compensation operation to generate channel audio, the presence of the environment object 175 can be eliminated. The specific details of the object-based calculation itself can continue to use the existing object-based acoustic product calculation method, and use the metadata of the sound source object to perform a large number of related array calculations. For example, a metadata of the compensating sound source object includes: the coordinate position, size, and reflection rate and absorption rate of the environmental object 175 .

In the process 1306, the control circuit 132 calculates the sound source effect of the compensated sound source object. In this embodiment, the compensating sound source object is correspondingly established according to the environmental object 175, and its metadata has the same coordinate position, size, and sound reflection rate and absorption rate as the environmental object 175, but the generated sound source effect is Inverse buff value for environment object 175.

The embodiment in FIG. 13 may also refer to calculations similar to those in FIG. 7 and FIG. 8 . Formula (8) can be derived into formula (9), and the gain value passively generated by the environmental object 175 is calculated according to the sound pressure value received by the environmental object 175 from the first speaker 110 :

A _t [m][n]=R[n]*SPL _t [m] (9)

Wherein, m represents the number of the horn, and n represents the number of the sub-band. _At [m][n] represents the gain value of the nth sub-band affected by the mth speaker. R[n] represents the absorption rate of the nth sub-band. SPL _t [m] represents the sound pressure value received by the environmental object 175 from the mth speaker at the tth time point. The time point t may represent the time difference when the sound is transmitted from the speaker to the environmental object 175 . If the time difference is greater than a non-negligible range, it indicates that there is an echo in the target space 170 .

It can be known from formula (9) that the calculation result corresponding to each environmental object includes an array of gain values of multiple speakers and multiple sub-bands at a time point. The sound source effect of the compensation sound source object is the negative value of the gain value array. In other words, the object base compensation operation based on the formula (9) includes an array operation in which parameters of multiple dimensions are alternately arranged and combined. For the sake of illustration, the gain value corresponding to one of the speakers and one of the sub-bands at a time point is used for illustration.

The embodiment of FIG. 13 is similar to the embodiments of FIG. 7 and FIG. 8 in that this embodiment can use an appropriate calculation method to calculate the acoustic impact of the environmental object according to the spatial configuration information of the environmental object 175 . For example, if the target listening point is located between a speaker and the visual line of the environmental object 175 , the control circuit 132 calculates and compensates the sound source effect of the sound source object according to the reflectivity of the environmental object 175 . In contrast, if the environmental object 175 is located between the target listening point and the line of sight of the speaker, the control circuit 132 calculates and compensates the sound source effect of the sound source object according to the absorption rate of the environmental object 175 .

For example, when an environmental object 175 absorbs the sound from a speaker, the volume effect received by the target listening point is reduced. At this time, the control circuit 132 creates a virtual sound source object that can generate a corresponding volume effect at the coordinate position of the environmental object 175 as compensation. Conversely, if an environmental object 175 reflects the sound of a speaker, the target listening point receives too much volume. At this time, the control circuit 132 creates a virtual sound source object with a negative gain value at the coordinate position of the environmental object 175 .

It can be understood that the visible line is defined as a straight line connecting two objects in space. Since the object has a certain volume and area, the volume may be large, and the situation where the visible line is blocked may include partial occlusion and complete occlusion. In this embodiment, based on the formula (9), different weight coefficients or different offset corrections may be added depending on various situations.

In the process 1308, the control circuit 132 mixes the sound source effect of the compensation sound source object into the channel audio, and makes the corresponding speaker play it. When the control circuit 132 performs the object floor compensation operation, it can handle complex object-to-matrix calculations, and mix multiple audio source signals assigned to each speaker into corresponding one-channel audio. After applying the object base compensation operation, the volume effect received at the target listening point will include the sound source effect produced by the compensated sound source object. In this way, the interference caused by the environmental object 175 can be effectively offset by the compensating sound source object.

In the process 1310, the control circuit 132 determines whether the target listening point is moved to a new position. As described in the process 208 , the audio system 100 can continuously track the movement of the user 180 to update the target listening point. If the target listening point has moved, go to process 1312 . Otherwise, continue the playback operation of the process 1308 .

In the process 1312, the control circuit 132 updates the metadata of the compensated audio source object. In this embodiment, the control circuit 132 establishes the object base space with the target listening point as the coordinate origin. If the target listening point moves to a new location, the control circuit 132 assigns the new location as the new coordinate origin of the object base space. The position difference between the new coordinate origin and the original coordinate origin can be expressed as a moving vector. The spatial coordinates of the environmental object 175 relative to the target listening point will also change inversely with the moving vector. The control circuit 132 then updates the metadata of the compensated sound source object corresponding to the environmental object 175 according to the motion vector. In a further embodiment, all the speakers in the object base space can also be regarded as an object, and have corresponding IDs, metadata and coordinates.

In another embodiment, the audio system 100 is not limited to take the target listening point as the coordinate origin. The audio system 100 can also adopt a fixed reference point as the origin of the object base space. When the relative position of the sound source object in the object base space changes, the control circuit 132 correspondingly updates the coordinate value in the metadata of the sound source object.

After the process 1312 is completed, the control circuit 132 repeats the process 1308 .

The embodiment of Fig. 13 illustrates the advantages of object floor compensation operation. The control circuit 132 converts the information of the target space 1100 into the form of the space coordinate system, which can simplify the complex multi-object interaction operation into the array operation of the relay data. The position of the moving user 180 is set as the origin of the spatial coordinate system, so that the processing of the virtual object is not affected by the movement of the user 180 at all, and the calculation process is simplified. This embodiment also proposes the concept of compensating the sound source object, and directly applies the object base compensation operation to offset the interference of environmental objects, eliminating the need for complex multi-channel interactive calculations.

In a further derivative embodiment, if the host device 130 itself does not have the object-based mixing operation capability, the control circuit 132 can provide the function of channel mapping (Channel mapping) by executing software, so that the object-based operation result can be correct Corresponds to each horn.

To sum up, the present application proposes an audio system 100 that can dynamically track the user's position to optimize the sound field, and can also intelligently eliminate the interference caused by environmental objects. The means of tracking the user's location can be the separate use or combination of various methods such as cameras, infrared rays, or wireless detectors. The spatial configuration information of the environmental objects 175 in the target space 170 can be obtained through recognition of images captured by the camera, or can be manually input by the user. The way to optimize the sound field can be channel floor compensation operation or object floor compensation operation. When calculating the impact of the environmental object 175 on the target listening point, the relative positional relationship between the environmental object 175 and the loudspeaker and the target listening point can also be considered, and different calculation methods can be adopted. When using the object base compensation operation, the control circuit 132 establishes a corresponding compensation audio source object for each environmental object 175 , so that the channel audio generated by the final mixing eliminates the interference caused by the environmental object 175 to the target listening point.

Certain words are used to refer to specific elements in the specification and scope of claims, but those skilled in the art may use different terms to refer to the same element. This specification and the scope of the patent application do not use the difference in name as a way to distinguish components, but use the difference in function of components as a basis for differentiation. The "comprising" mentioned in the specification and scope of patent application is an open term and should be interpreted as "including but not limited to". In addition, the term "coupled" herein includes any direct and indirect means of connection. Therefore, if it is described that the first element is coupled to the second element, it means that the first element can be directly connected to the second element through electrical connection or signal connection means such as wireless transmission or optical transmission, or through other elements or connections. The means is indirectly electrically or signally connected to the second element.

The description of "and/or" used in the specification includes any combination of one or more of the listed items. In addition, unless otherwise specified in the specification, any singular term also includes a plural meaning.

The above are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the claims of the present invention shall fall within the scope of the present invention.

100: Audio system (audio system)

110: first speaker

112: first channel audio (first channel signal)

120: second speaker

122: Second channel audio (second channel signal)

130: host device (host device)

131: Storage circuit (storage circuit)

132: Control circuit (control circuit)

133: Human-machine interface circuit (user interface circuit)

134: Identification circuit (recognizer circuit)

135: audio transmission circuit (audio transmission circuit)

136: Communication circuit (communication circuit)

140: sensor circuit (sensor circuit)

150: user equipment (user equipment)

160: remote database

170: target space

171: first position (first position)

172: Second position (second position)

173:Movement trail

175:ambient object

180: user (user)

202～218: Process (operation)

312～316: Process (operation)

410: Process (operation)

600: Target space (target space)

601: first location

602: second location (first location)

610: camera

620: infrared sensor (infrared sensor)

630: wireless detector (radio detector)

700: Target space (target space)

800: Target space (target space)

902～910: Process (operation)

1002～1012: Process (operation)

1100: Target space (target space)

1103: Object movement track (movement trail)

1105: Virtual audio object (virtual audio object)

1110: first speaker (first speaker)

1120: Second speaker (second speaker)

1130: third speaker (third speaker)

1140: fourth speaker

P0: origin (original point)

P1: first position (first position)

P1': new first position

P2: second position (second position)

1200: Target space (target space)

1201: target listening spot

1203: Movement trail (movement trail)

1210: first speaker (first speaker)

1212: first channel output (first channel output)

1220: Second speaker (second speaker)

1222: Second channel output (second channel output)

1230: third speaker (third speaker)

1240: fourth speaker (fourth speaker)

1250: fifth speaker (fifth speaker)

1252: Fifth channel output (fifth channel output)

1260: sixth speaker (sixth speaker)

1262: sixth channel output (sixth channel output)

1304～1312: Process (operation)

FIG. 1 is a functional block diagram of an audio system according to an embodiment of the present invention.

FIG. 2 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

FIG. 3 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

FIG. 4 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

FIG. 5 is a flowchart of a method for optimizing dynamic sound effects according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of a target space according to the present invention, which is used to illustrate an embodiment of calculating an audio adjustment amount based on the position of the best listening point.

FIG. 7 is a schematic diagram of a target space of the present invention, which is used to illustrate an embodiment of calculating an audio adjustment amount based on the absorption rate of an environmental object.

FIG. 8 is a schematic diagram of a target space according to the present invention, which is used to illustrate an embodiment of calculating an audio adjustment amount according to the reflectivity of an environmental object.

FIG. 9 is a flow chart of identifying an object by a host device according to an embodiment of the invention.

FIG. 10 is a flowchart of an audio processing method according to an embodiment of the present invention, illustrating an embodiment of calculating an output compensation value according to the positional relationship of environmental objects.

FIG. 11 is a schematic diagram of a target space of the present invention, which is used to illustrate an embodiment of optimizing a sound field by operating object floor compensation.

FIG. 12 is a schematic diagram of a target space of the present invention, which is used to illustrate an embodiment of optimizing a sound field by operating object floor compensation.

FIG. 13 is a flow chart of an object base compensation operation according to an embodiment of the present invention.

100: Audio system

110: The first horn

112: First channel audio

120: second horn

122:Second channel audio

130: host device

131: storage circuit

132: control circuit

133: Human-machine interface circuit

134: Identification circuit

135: audio transmission circuit

136: Communication circuit

140: Sensor circuit

150: user equipment

160:Remote database

170: target space

171: First position

172: second position

173:Movement track

175: Environmental objects

180: user

Claims (10)

An audio system (100), which can dynamically optimize the playback effect according to the user's position, comprising: A sensor circuit (140), configured to dynamically sense a target space (170) to generate sound field environmental information; A first speaker (110) and a second speaker (120), set to play audio; A host device (130), coupled to the sensor circuit (140), the first speaker (110) and the second speaker (120), comprising: An identification circuit (134), configured to identify a user from the sound field environment information and determine a user position of the user in the target space; a control circuit (132), coupled to the identification circuit (134), configured to dynamically assign the user's position as a target listening point; and An audio transmission circuit (135), coupled to the control circuit (132), the first speaker (110) and the second speaker (120), configured to transmit audio; Wherein, the sensor circuit (140) includes a camera (610), configured to capture a sound field environment image of the target space (170); Wherein, the identification circuit (134) analyzes the sound field environment image to obtain spatial configuration information and acoustic attribute information of an environmental object (175) in the target space (170); Wherein, the control circuit (132) performs a channel floor compensation operation to generate a first channel optimized for the target listening point according to the target listening point, and the spatial configuration information and acoustic property information of the environmental object (175) audio (112) and a second channel audio (122). Wherein, the control circuit (132) respectively outputs the first channel audio (112) and the second channel audio (122) to the corresponding first speaker (110) and the first speaker (110) through the audio transmission circuit (135). Second horn (120). The audio system (100) as claimed in claim 1, wherein the channel floor compensation operation includes: splitting the first channel audio (112) emitted by the first speaker (110) into a plurality of sub-band signals; According to the wavelength of a sub-band signal among the plurality of sub-band signals and the distance between the target listening point and the first speaker (110), it is judged that a sound field type generated by the sub-band signal at the target listening point belongs to a near sound field or a far sound field; When the distance between the target listening point and the first speaker (110) is greater than the wavelength of the sub-band signal or a specific ratio of the size of the first speaker (110), the sound field type is determined to be a far sound field; and When the distance between the target listening point and the first speaker (110) is smaller than the wavelength of the sub-band signal or the specific ratio of the size of the first speaker (110), it is judged that the sound field type is a near sound field. The sound system (100) according to claim 2, wherein the channel floor compensation operation further includes, when the control circuit (132) determines the distance between the position of the target listening point and the first speaker (110) from a first When a distance (R1) is changed to a second distance (R2), and the sound field type belongs to the far sound field, a far sound field formula is used to calculate the sound pressure value of the sub-band signal; wherein, the far sound field formula includes: SPL ' = SPL + 20 log ₁₀ (R2/R1) wherein, SPL is the sound pressure value of the sub-band signal before adjustment, SPL' is the sound pressure value of the sub-band signal after adjustment, R1 is the first distance, R2 is the second distance; and wherein, if the adjusted sound pressure value of the sub-band signal is less than zero, the control circuit (132) makes the adjusted sound pressure value of the sub-band signal zero. The sound system (100) according to claim 2, wherein the channel floor compensation operation further includes, when the control circuit (132) determines the distance between the position of the target listening point and the first speaker (110) from a first When a distance (R1) is changed to a second distance (R2), and the sound field type belongs to the near sound field, a near sound field formula is used to calculate the sound pressure value of the sub-band signal; wherein, the near sound field formula Including: SPL' = SPL + 10 log ₁₀ (R2/R1) Among them, SPL is the sound pressure value of the sub-band signal before adjustment, SPL' is the sound pressure value of the sub-band signal after adjustment, and R1 is the sound pressure value of the sub-band signal A distance, R2 is the second distance; and wherein, if the adjusted sound pressure value of the sub-band signal is less than zero, the control circuit (132) makes the adjusted sound pressure value of the sub-band signal zero. The sound system (100) according to claim 2, wherein the spatial configuration information of the environmental object includes the location, size and appearance characteristics of the environmental object, and the acoustic property information of the environmental object includes the reflection rate and absorption rate of sound ; Wherein, when the control circuit (132) generates the first channel audio, if the target listening point is between the first speaker (110) and the visual line of the environmental object (175), the control circuit (132 ) Calculate the degree to which the playback effect of the first speaker (110) at the target listening point is affected by the environmental object (175) according to the reflectivity of the environmental object, so as to determine the sound pressure of the first channel audio (112) value; and Wherein, when the control circuit (132) generates the first channel audio, if the environmental object (175) is located between the target listening point and the visible line of the first speaker (110), the control circuit (132 ) Calculate the degree to which the playback effect of the first speaker (110) at the target listening point is affected by the environmental object (175) according to the absorption rate of the environmental object, so as to determine the sound pressure of the first channel audio (112) value. The sound system (100) according to claim 2, wherein the recognition circuit (134) dynamically recognizes the user's head position, face direction, or ear position to determine the user's position. The sound system (100) according to claim 2, wherein the sensor circuit (140) further includes an infrared sensor (620), configured to capture a thermal imaging data in the target space; the identification The circuit (134) analyzes the movement track of the thermal imaging data to dynamically determine the user's position. The audio system (100) according to claim 2, wherein the sensor circuit (140) further includes a wireless detector (630), which is arranged in the target space and detects a wireless signal of an electronic device ; Wherein, the identification circuit (134) dynamically locates the position of the electronic device according to the characteristics of the wireless signal detected by the wireless detector (630); and Wherein, the identification circuit (134) dynamically judges the user's position according to the position of the electronic device. The audio system (100) according to claim 2, wherein the host device (130) further includes a storage circuit (131), coupled to the control circuit (132), configured to store one or more object databases, Each of the object databases corresponds to an application scenario category, and includes appearance feature information and acoustic property information of multiple environmental objects; Wherein, when the identification circuit (134) analyzes the sound field environment information, it identifies the sound field environment information to determine an applicable application scenario category; and Wherein, the control circuit (132) preferentially selects and uses an object database related to the application scenario category from the storage circuit (131) according to the application scenario category to identify the environmental object and search for the acoustic properties of the environmental object Information. The audio system (100) according to claim 2, wherein the host device (130) further includes a communication circuit (136), coupled to the control circuit (132), configured to be controlled by the control circuit (132) Controlled and connected to a remote database (160) corresponding to the application scenario category; the remote database (160) is configured to store one or more object databases, wherein each object database corresponds to an application scenario category , including appearance feature information and acoustic attribute information of multiple environmental objects; Wherein, when the identification circuit (134) analyzes the sound field environment information, it identifies the sound field environment information to determine an applicable application scenario category; and Wherein, the control circuit (132) preferentially selects and uses an object database related to the application scenario category from the remote database (160) according to the application scenario category to identify the environment object and search for the environment object. Acoustic property information.

Images