RU2803062C2 - Methods, apparatus and systems for expanding three degrees of freedom (3dof+) of mpeg-h 3d audio - Google Patents

Abstract

FIELD: means for processing information.

SUBSTANCE: invention relates to means for processing information about the position of a sound object. The technical result consists in increasing the efficiency of representing the movement of the listening position relative to the sound object. Listener orientation information is obtained indicating the orientation of the listener's head. Listener offset information is received indicating the offset of the listener's head relative to the nominal listening position. The position of the object is determined from the position information. The position of the object is modified based on the information about the displacement of the listener by applying translational movement to the position of the object. Additionally, the modified position of the object is modified based on the information about the orientation of the listener. When the listener displacement information indicates a displacement of the listener's head from the nominal listening position due to a small position displacement (0.5 meters or less than 0.5 meters), the distance between the position of the sound object and the listening position after the displacement of the listener's head is equal to the distance between the modified position of the sound object and nominal listening position.

EFFECT: increasing the efficiency of representing the movement of the listening position relative to the sound object.

8 cl, 5 dwg

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the following priority applications: US Provisional Application 62/654915 (ref: D18045USP1), filed April 9, 2018; US Provisional Application 62/695446 (ref: D18045USP2), filed July 9, 2018, and US Provisional Application 62/823159 (ref: D18045USP3), filed March 25, 2019, which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to methods and apparatus for processing position information indicating the position of an audio object and information indicating a displacement of the listener's head position.

BACKGROUND OF THE INVENTION

The first edition (October 15, 2015) and revisions of the ISO/IEC 23008-3 MPEG-H 3D Audio standard do not support small translational movements of the user's head in a Three Degrees of Freedom (3DoF) environment.

BRIEF DESCRIPTION OF THE INVENTION

The first edition (October 15, 2015) and revisions 1-4 of the ISO/IEC 23008-3 MPEG-H 3D Audio standard introduces functionality to enable a 3DoF environment where the user (listener) performs head-turning actions. However, such functionality at best only supports the transmission of a scene rotation signal and rendering accordingly. This means that the sound stage can remain stationary in space when the listener's head orientation changes, which corresponds to the 3DoF property. However, there is no way to account for slight forward motion of the user's head within a true MPEG-H 3D Audio ecosystem.

Thus, there is a need for methods and apparatus for processing information about the position of audio objects that can account for slight translational movement of the user's head, potentially in combination with rotational movement of the user's head.

The present invention provides apparatus and systems for processing position information having the features of the corresponding independent and dependent claims.

According to one aspect of the present invention, a method for processing position information indicating the position of an audio object is described, which processing may be compatible with the MPEG-H 3D Audio standard. The object's position can be used to render a sound object. An audio object may be included in object-based audio content along with information about its position. The position information may represent (part of) metadata for the audio object. Audio content (eg, an audio object along with its position information) may be conveyed in an encoded audio bitstream. The method may include receiving audio content (eg, an encoded audio bitstream). The method may include obtaining listener orientation information indicating the orientation of the listener's head. The listener may be referred to as a user, such as a decoder performing the method. The listener's head orientation (listener orientation) may represent the orientation of the listener's head relative to the nominal orientation. The method may further include obtaining listener displacement information indicating displacement of the listener's head. The listener's head displacement may represent a displacement relative to the nominal listening position. The nominal listening position (or nominal listening position) may be a default position (eg, a target position, an expected position for the listener's head, or a sweet spot for speaker placement). Listener orientation information and listener displacement information can be obtained through the input interface of the MPEG-H 3D Audio decoder. Listener orientation information and listener displacement information can be inferred based on information from the sensors. The combination of orientation information and position information may be called position information. The method may further include determining the position of an object from the position information. For example, the position of an object can be extracted from the position information. Determining (eg, retrieving) the position of an object may be further based on information regarding the speaker geometry of one or more speakers in the listening environment. The object position may also be referred to as the audio object channel position. The method may further include modifying the position of an object based on the listener's displacement information by applying a translational movement to the object's position. Object position modification may refer to adjusting the position of the object to move the listener's head from the nominal listening position. In other words, modifying the position of an object may refer to the application of position offset compensation to the position of the object. The method may also further include further modifying the modified object position based on the listener orientation information, such as by applying a rotational transformation to the modified object position (eg, rotation relative to the listener's head or nominal listening position). An additional modification to the modified object position for rendering the audio object may include rotating the sound stage.

The proposed method, adapted as described above, provides a more realistic listening experience, especially for sound objects that are close to the listener's head. In addition to the three (rotational) degrees of freedom typically offered to the listener in a 3DoF environment, the proposed method can also take into account translational movements of the listener's head. This allows the listener to reach close sound objects from different angles and even sides. For example, a listener can listen to the sound object “mosquito”, which is located near the listener's head, from different angles by slightly moving his head, possibly in addition to turning his head. As a result, the proposed method can provide an improved, more realistic, immersive listening experience for the listener.

In some embodiments, the modification of the object position and the further modification of the modified object position may be performed such that the sound object, after rendering on one or more real or virtual speakers in accordance with the additional modified object position, is psychoacoustically perceived by the listener as arising from a fixed position relative to the nominal position listening regardless of the displacement of the listener's head from the nominal listening position and the orientation of the listener's head relative to the nominal orientation. Accordingly, the sound object may be perceived as moving relative to the listener's head when the listener's head makes a displacement from the nominal listening position. Likewise, a sound object may be perceived as rotating relative to the listener's head when the listener's head makes an orientation change from its nominal orientation. The one or more speakers may be part of a headset, for example, or may be part of a speaker arrangement (eg, a 2.1, 5.1, 7.1, etc. speaker arrangement).

In some embodiments, modification of the object's position based on the listener's displacement information can be accomplished by translationally moving the object's position using a vector that is positively correlated with amplitude and negatively correlated with the direction of the listener's head displacement vector from the nominal listening position.

In this way, it is ensured that nearby sound objects are perceived by the listener as moving in accordance with the movement of his head. This contributes to a more realistic listening experience for these sound objects.

In some embodiments, the listener displacement information may indicate a displacement of the listener's head from the nominal listening position due to a slight positional shift. For example, the absolute value of the offset may be no more than 0.5 m. The offset may be expressed in Cartesian coordinates (eg x, y, z) or spherical coordinates (eg azimuth, elevation, radius).

In some embodiments, the listener displacement information may indicate a displacement of the listener's head from the nominal listening position, which may be achieved by movement of the listener's upper body and/or head. In this way, the listener can perform the shift without moving the lower body. For example, displacement of the listener's head can be accomplished while the listener is sitting in a chair.

In some embodiments, the position information may include an indication of the distance from the sound object to the nominal listening position. The distance (radius) may be less than 0.5 m. For example, the distance may be less than 1 cm. Alternatively, the distance from the sound object to the nominal listening position may be set to a default value by the decoder.

In some embodiments, the listener orientation information may include yaw, pitch, and head roll information of the listener. Yaw, pitch, roll may be provided relative to the nominal orientation (eg, home orientation) of the listener's head.

In some embodiments, the listener displacement information may comprise information about the displacement of the listener's head from the nominal listening position, expressed in Cartesian coordinates or spherical coordinates. Thus, the displacement can be expressed in terms of x, y, z coordinates for Cartesian coordinates and in terms of azimuth, elevation, radius coordinates for spherical coordinates.

In some embodiments, the method may further include detecting the listener's head orientation via wearable and/or fixed equipment. Likewise, the method may further include detecting movement of the listener's head from the nominal listening position through the wearable and/or fixed equipment. The wearable equipment may be, correspond to, and/or include, for example, an augmented reality (AR)/virtual reality (VR) headset or headset. The fixed equipment may be, correspond to and/or contain, for example, camera sensors. This provides accurate information about the displacement and/or orientation of the listener's head and thereby enables realistic processing of nearby sound objects according to the orientation and/or displacement.

In some embodiments, the method may further include rendering an audio object on one or more real or virtual speakers in accordance with a further modified position of the object. For example, rendering an audio object can be done for the left and right speakers of a headset. In some embodiments, rendering may be performed taking into account acoustic absorption for short distances from the sound object to the listener's head based on head-related transfer functions (HRTF) for the listener's head. This way, the rendering of nearby sound objects will be perceived as even more realistic by the listener.

In some embodiments, the further modified object position may be adjusted for the input format used by the MPEG-H 3D Audio renderer. In some embodiments, rendering may be performed using an MPEG-H 3D Audio renderer. In some embodiments, processing may be performed using an MPEG-H 3D Audio decoder. In some embodiments, processing may be performed by a scene shifter of an MPEG-H 3D Audio decoder. Accordingly, the proposed method provides the implementation of a limited sense of six degrees of freedom (Six Degrees of Freedom, 6D0F) (i.e. 3DoF+) within the MPEG-H 3D Audio standard.

According to another aspect of the present invention, an additional method for processing position information indicating the position of an object for an audio object is described. The object's position can be used to render a sound object. The method may include obtaining listener displacement information indicating displacement of the listener's head. The method may further include determining the position of an object from the position information. The method may also further include modifying the position of an object based on information about the listener's displacement by applying a translational movement to the position of the object.

The proposed method, adapted as described above, provides a more realistic listening experience, especially for sound objects that are close to the listener's head. Due to the ability to take into account small translational movements of the listener's head, the proposed method allows the listener to reach close sound objects from different angles and even sides. As a result, the proposed method can provide an improved, more realistic, immersive listening experience for the listener.

In some embodiments, modification of an object's position based on listener displacement information may be performed such that an audio object, after rendering on one or more real or virtual speakers in accordance with the modified object position, is psychoacoustically perceived by the listener as originating from a fixed position relative to the nominal listening position regardless of the listener's head displacement from the nominal listening position.

In some embodiments, modification of the object's position based on the listener's displacement information can be accomplished by translationally moving the object's position using a vector that is positively correlated with amplitude and negatively correlated with the direction of the listener's head displacement vector from the nominal listening position.

According to another aspect of the present invention, an additional method for processing position information indicating the position of an object for an audio object is described. The object's position can be used to render a sound object. The method may include obtaining listener orientation information indicating the orientation of the listener's head. The method may further include determining the position of an object from the position information. The method may also further include modifying the position of an object based on the listener's orientation information, such as by applying a rotational transformation to the object's position (eg, rotation relative to the listener's head or nominal listening position).

The proposed method, adapted as described above, can take into account the orientation of the listener's head to provide the listener with a more realistic listening experience.

In some embodiments, modification of an object's position based on listener orientation information may be performed such that an audio object, after rendering on one or more real or virtual speakers in accordance with the modified object position, is psychoacoustically perceived by the listener as originating from a fixed position relative to the nominal listening position regardless of the listener's head orientation relative to the nominal orientation.

According to another aspect of the present invention, an apparatus for processing position information indicating the position of an object for an audio object is described. The object's position can be used to render a sound object. The apparatus may include a processor and a storage device coupled to the processor. The processor may be adapted to receive listener orientation information indicating the orientation of the listener's head. The processor may be further adapted to receive listener displacement information indicative of the displacement of the listener's head. The processor may be further adapted to determine the position of an object from the position information. The processor may be further adapted to modify the position of an object based on the listener's displacement information by applying translational movement to the position of the object. The processor may also be further adapted to further modify the modified object position based on listener orientation information, for example by applying a rotational transformation to the modified object position (eg, rotation relative to the listener's head or nominal listening position).

In some embodiments, the processor may be adapted to modify the position of an object and further modify the modified object position such that an audio object, after rendering on one or more real or virtual speakers in accordance with the additional modified object position, is psychoacoustically perceived by the listener as arising from a fixed position relative to the nominal listening position, regardless of the displacement of the listener's head from the nominal listening position and the orientation of the listener's head relative to the nominal orientation.

In some embodiments, the processor may be adapted to modify the position of an object based on the listener's displacement information by progressively moving the object's position using a vector that is positively correlated with amplitude and negatively correlated with the direction of the vector of displacement of the listener's head from the nominal listening position.

In some embodiments, the listener displacement information may indicate a displacement of the listener's head from the nominal listening position due to a slight positional shift.

In some embodiments, the listener displacement information may indicate a displacement of the listener's head from the nominal listening position, which may be achieved by movement of the listener's upper body and/or head.

In some embodiments, the position information may include an indication of the distance from the sound object to the nominal listening position.

In some embodiments, the listener orientation information may include yaw, pitch, and head roll information of the listener.

In some embodiments, the listener displacement information may comprise information about the displacement of the listener's head from the nominal listening position, expressed in Cartesian coordinates or spherical coordinates.

In some embodiments, the apparatus may further comprise wearable and/or fixed equipment for detecting the listener's head orientation. In some embodiments, the apparatus may further comprise wearable and/or fixed equipment for detecting movement of the listener's head from the nominal listening position.

In some embodiments, the processor may be further adapted to perform rendering of an audio object on one or more real or virtual speakers in accordance with a further modified position of the object.

In some embodiments, the processor may be adapted to perform acoustic absorption rendering for short distances from the audio object to the listener's head based on the HRTF for the listener's head.

In some embodiments, the processor may be adapted to adjust a further modified object position for the input format used by the MPEG-H 3D renderer. In some embodiments, rendering may be performed using an MPEG-H 3D Audio renderer. That is, the processor can implement an MPEG-H 3D Audio renderer. In some embodiments, the processor may be adapted to implement an MPEG-H 3D Audio decoder. In some embodiments, the processor may be adapted to implement a scene offset block of an MPEG-H 3D Audio decoder.

According to another aspect of the present invention, an additional apparatus for processing position information indicating the position of an object for an audio object is described. The object's position can be used to render a sound object. The apparatus may include a processor and a storage device coupled to the processor. The processor may be adapted to receive listener displacement information indicative of the displacement of the listener's head. The processor may be further adapted to determine the position of an object from the position information. The processor may be further adapted to modify the position of an object based on the listener's displacement information by applying translational movement to the position of the object.

In some embodiments, the processor may be adapted to modify the position of an object based on the listener's displacement information such that the sound object, after rendering on one or more real or virtual speakers in accordance with the modified object position, is psychoacoustically perceived by the listener as arising from a fixed position relative to the nominal listening position regardless of the listener's head displacement from the nominal listening position.

In some embodiments, the processor may be adapted to modify the position of an object based on the listener's displacement information by progressively moving the object's position using a vector that is positively correlated with amplitude and negatively correlated with the direction of the vector of displacement of the listener's head from the nominal listening position.

According to another aspect of the present invention, an additional apparatus for processing position information indicating the position of an object for an audio object is described. The object's position can be used to render a sound object. The apparatus may include a processor and a storage device coupled to the processor. The processor may be adapted to receive listener orientation information indicating the orientation of the listener's head. The processor may be further adapted to determine the position of an object from the position information. The processor may also be further adapted to modify the position of an object based on the listener's orientation information, for example by applying a rotational transformation to the modified position of the object (eg, rotation relative to the listener's head or nominal listening position).

In some embodiments, the processor may be adapted to modify the position of an object based on the listener's orientation information such that the sound object, after rendering on one or more real or virtual speakers in accordance with the modified object position, is psychoacoustically perceived by the listener as arising from a fixed position relative to the nominal listening position regardless of the listener's head orientation relative to the nominal orientation.

In yet another aspect, a system is described. The system may comprise an apparatus according to any of the foregoing aspects and wearable and/or fixed equipment capable of detecting the orientation of the listener's head and detecting movement of the listener's head.

It should be understood that the steps of the method and the characteristic features of the apparatus can be interchanged in various ways. In particular, the details of the disclosed method may be implemented in the form of an apparatus adapted to perform some or all of the steps of the method, and vice versa, as will be appreciated by one skilled in the art. In particular, it should be understood that the apparatus of the present invention may refer to apparatus for implementing or executing the methods of the above embodiments and variations thereof, and that the corresponding statements made with respect to the methods similarly apply to the corresponding apparatus. Likewise, it should be understood that the methods of the present invention may relate to methods of operating the apparatus of the above embodiments and variations thereof, and that the corresponding statements made with respect to the apparatus apply similarly to the corresponding methods.

BRIEF DESCRIPTION OF THE FIGURES

The invention is explained below in an illustrative manner with reference to the accompanying drawings, in which:

in fig. 1 schematically shows an example of an MPEG-H 3D Audio system;

in fig. 2 is a schematic diagram of an example of an MPEG-H 3D Audio system according to the present invention;

in fig. 3 is a schematic illustration of an example of an audio rendering system according to the present invention;

in fig. 4 schematically shows an example system of Cartesian coordinate axes and their relationship to spherical coordinates; And

in fig. 5 is a flowchart schematically illustrating an example of a position information processing method for an audio object according to the present invention.

DETAILED DESCRIPTION

In the context of this document, 3DoF is generally a system that can correctly handle the user's head motion, in particular head rotation, characterized by three parameters (e.g., yaw, pitch, roll). Such systems are often available in various gaming systems such as virtual reality (VR) / augmented reality (AR) / mixed reality (MR) systems, or other audio environments of this type.

As used herein, a user (eg, an audio decoder or a playback system including an audio decoder) may also be referred to as a “listener.”

In the context of this document, 3DoF+ shall mean that in addition to the movement of the user's head, which can be properly processed by the 3DoF system, translational movements must also be processed.

In the context of this document, the expression “small” should indicate that movements are limited to a threshold value, which is typically no more than 0.5 meters. This means that movements do not exceed 0.5 meters from the original position of the user's head. For example, the user's movement is limited due to sitting on a chair.

As used herein, the term "MPEG-H 3D Audio" refers to the technical specification specified in the ISO/IEC 23008-3 standard and/or any future revisions, editions or other versions of the ISO/IEC 23008-3 standard.

In the context of the audio standards provided by the MPEG organization, the difference between 3DoF and 3DoF+ can be defined as follows:

• 3DoF: Allows the user to experience yaw, pitch, roll motion (such as the user's head);

• 3DoF+: Allows the user to experience yaw, pitch, roll, and limited translational motion (such as the user's head), such as while sitting in a chair.

Limited (small) forward movements of the head may be movements limited to a specific radius of movement. For example, movement may be limited due to the user's sitting position, for example without use of the lower body. Small forward movements of the head may relate to or correspond to a displacement of the user's head relative to the nominal listening position. The nominal listening position (or nominal listening position) may be a default position (such as, for example, a target position, an expected position for the listener's head, or a sweet spot for speaker placement).

3DoF+ perception can be compared to limited 6DoF perception, in which forward movements can be described as limited or small movements of the head. In one example, audio is also rendered based on the position and orientation of the user's head, including possible acoustic absorption. Rendering can be done taking into account acoustic absorption for short distances from the sound object to the listener's head, for example based on the simulated sound experience functions (HRTF) for the listener's head.

With respect to methods, systems, apparatus and other devices compatible with functionality established by the MPEG-H 3D Audio standard, which may indicate that 3DoF+ is supported for any future version(s) of MPEG standards, such as future versions of the Omnidirectional Multimedia Format ( e.g., standardized in future versions of MPEG-I), and/or in any updates to MPEG-H Audio (e.g., revisions or newer standards based on the MPEG-H 3D Audio standard), or any related or reference standards that may require updates (for example, standards that clarify specific types of SEI metadata and messages).

For example, the functionality of an audio renderer that is normative for the audio standard specified in the MPEG-H 3D Audio specification may be extended to include sound stage rendering to accurately account for user interaction with the sound stage, such as when the user moves their head slightly in sides.

The present invention provides various technical advantages, including the advantage of providing MPEG-H 3D Audio capable of handling 3DoF+ use cases. The present invention extends the MPEG-H 3D Audio standard to support 3DoF+ functionality.

To support 3DoF+ functionality, the audio rendering system must accommodate limited/small displacements of the user/listener's head position. Position offsets should be determined based on the relative deviation from the initial position (ie default position/nominal listening position). In one example, the amplitude of this deviation (for example, radius deviation, which can be determined based on where P ₀ represents the nominal listening position and P ₁ represents the offset position of the listener's head) the maximum is approximately 0.5 m. In another example, the amplitude of the deflection is limited to being the deflection obtained only when the user is sitting on a chair and does not move his lower body (however his head moves relative to his body). This (small) offset distance provides very small (perceived) level and panning differences for distant sound objects. However, for close objects such a small deviation distance can become significant for perception. Of course, the listener's head movement can influence the perception of where the exact location of the sound object is. This perceptual effect can remain important (i.e. be perceptually noticeable to the user/listener) provided that the relationship between (i) the user's head displacement (e.g. )) and the distance to the sound object (e.g., d) trigonometrically provides angles that are within the range of users' psychoacoustic ability to detect the direction of sound. This range may vary depending on the audio renderer settings, audio material, and playback configuration. For example, assuming a localization accuracy range of, for example, +/-3° with listener head freedom of +/-0.25 m from side to side, this would correspond to ~5 m of object distance.

For objects that are located close to the listener (e.g., objects less than 1 m away from the user), proper handling of listener head position displacement is important for 3DoF+ scenarios, since there are significant perceptual effects during both panning and level changes.

One example of near object processing in listeners, for example, is when a sound object (e.g., a mosquito) is located very close to the listener's face. A sound system, such as a sound system that provides VR/AR/MR capabilities, must allow the user to perceive that sound object from all sides and angles, even when the user makes small forward movements of the head. For example, the user must be able to accurately perceive an object (such as a mosquito) even when the user moves their head without moving their lower body.

However, a system compatible with the current version of MPEG-H 3D Audio cannot currently handle this correctly. Instead, using an MPEG-H 3D Audio compatible system results in the mosquito being perceived from an incorrect position relative to the user. In scenarios that involve 3DoF+ execution, small translational movements should result in a significant difference in the perception of the sound object (for example, when moving the head to the left, the sound object "mosquito" should be perceived on the right side of the user's head, etc.).

The MPEG-H 3D Audio standard contains a bitstream syntax that allows distance information to be conveyed through a bitstream syntax, such as a syntax element (starting from 0.5 m).

Syntax element can be input into the bitstream provided by the MPEG-H 3D Audio standard, which can be used to signal that an object is at very close distances from the listener. For example, the syntax may notify that the distance between the user and an object is less than a specific threshold distance (for example, less than 1 cm).

In fig. 1 and fig. 2 shows the present invention based on headphone rendering (i.e., when the speakers move with the listener's head).

In fig. 1 shows an example of the behavior 100 of a system that is compatible with the MPEG-H 3D Audio system. In this example, it is assumed that the listener's head is located at position P ₀ 103 at time t ₀ and moves to position P ₁ 104 at time t ₁ >t ₀ . The dotted circles around the positions P ₀ and P ₁ indicate the permissible area of 3DoF+ movement (for example, with a radius of 0.5 m). The position A 101 indicates the transmitted position of the object (at time t ₀ and time t ₁ , ie, assume that the transmitted position of the object is constant for some time). Position A also indicates the position of the object being rendered by the MPEG-H 3D Audio renderer at time t ₀ . Position B 102 indicates the position of the object being rendered by MPEG-H 3D Audio at time. Vertical lines extending upward from positions P ₀ and P ₁ indicate the corresponding orientations (eg, viewing directions) of the listener's head at times t ₀ and t ₁ . The displacement of the user's head between position P ₀ and position P ₁ can be represented by 106. If the listener is in the default position (nominal listening position) P ₀ 103 at time t ₀ , he will perceive the sound object (for example, a mosquito) at the correct position A 101. If the user moves to position P ₁ 104 at time t ₁ it will perceive the audio object at position B 102 if MPEG-H 3D Audio processing is applied as the current standard, which introduces the error δ _AB 105 shown. That is, despite the listener's head movement, the audio object (eg, a mosquito) will still be perceived as located directly in front of the listener's head (ie, essentially moving with the listener's head). It should be noted that the introduced error δ _AB 105 occurs regardless of the listener's head orientation.

In fig. 2 shows an example of system behavior relative to an MPEG-H 3D Audio system 200 according to the present invention. In fig. 2, the listener's head is in position P ₀ 203 at the moment t ₀ and moves to position P ₁ 204 at the moment t ₁ >t ₀ . Again, the dotted circles around the P ₀ and P ₁ positions indicate the permissible area of 3DoF+ movement (eg with a radius of 0.5 m). In paragraph 201, it is shown that the position A=B, which means that the transmitted position of the object (at time t0 and time t ₁ , ie, assume that the transmitted position of the object) is constant for some time. The position A=B 201 also indicates the position of the object being rendered by MPEG-H 3D Audio at time t ₀ and time t ₁ . Vertical arrows extending upward from positions P ₀ 203 and P ₁ 204 indicate the corresponding orientations (eg, viewing directions) of the listener's head at moments t ₀ and t ₁ . If the listener is in the home/default position (nominal listening position) P ₀ 203 at time t ₀ , he will perceive the sound object (eg, a mosquito) at the correct position A 201. If the user moves to position P ₁ 203 at time t _1, it will still perceive the sound object at position B 201, which is similar to (eg, substantially the same as) position A 201 in accordance with the present invention. Thus, the present invention allows the user to change position over time (for example, from position P ₀ 203 to position P ₁ 204), while at the same time allowing the perception of sound from the same (spatially fixed) location (for example, position A=B 201 , etc.). In other words, the sound object (eg, a mosquito) moves relative to the listener's head in accordance with the movement of (eg, negatively correlated with) the listener's head. This allows the user to move around a sound object (such as a mosquito) and perceive the sound object from different angles or even from different sides. The displacement of the user's head between position P ₀ and position P ₁ can be represented as 206.

In fig. 3 shows an example of an audio rendering system 300 according to the present invention. The audio rendering system 300 may correspond to or include a decoder, such as, for example, an MPEG-H 3D audio decoder. The audio rendering system 300 may include a stage shifter 310 with a corresponding stage shift processing interface (eg, an interface for scene shift data according to the MPEG-H 3D Audio standard). The sound stage shifter 310 may output object positions 321 for rendering corresponding sound objects. For example, a scene displacement block can output object position metadata to render corresponding audio objects.

The audio rendering system 300 may further include an audio object rendering module 320. For example, a rendering engine may consist of hardware, software, and/or any partial or complete processing performed through cloud computing, including various services such as software development platforms, servers, storage, and software over the Internet, often referred to as " cloud" that are compatible with the description established by the MPEG-H 3D Audio standard. Audio object rendering module 320 may render audio objects for one or more (real or virtual) speakers in accordance with corresponding object positions (these object positions may be modified or further modified object positions described below). Audio object rendering module 320 may render audio objects for headphones and/or speakers. That is, the audio object rendering module 320 can generate waveforms of the object according to a predetermined reproduction format. To this end, audio object renderer 320 may use compressed object metadata. Each object can be rendered to specific output channels according to its object position (eg, modified object position, or further modified object position). Thus, object positions can also be called channel positions of their audio objects. Audio object positions 321 may be included in the output of object position metadata or scene offset metadata via scene offset block 310.

The processing according to the present invention may be compatible with the MPEG-H 3D Audio standard. Thus, it may be performed by an MPEG-H 3D Audio decoder, or more specifically by an MPEG-H scene shifter and/or an MPEG-H 3D Audio renderer. Accordingly, the audio rendering system 300 of FIG. 3 may correspond to or include an MPEG-H 3D Audio decoder (ie, a decoder compatible with the specification established by the MPEG-H 3D Audio standard). In one example, the audio rendering system 300 may be an apparatus comprising a processor and a storage device coupled to the processor, wherein the processor is adapted to implement an MPEG-H 3D Audio decoder. In particular, the processor may be adapted to implement an MPEG-H scene offset unit and/or an MPEG-H 3D Audio renderer. Thus, the processor may be adapted to perform the processing steps described in the present invention (eg, steps S510-S560 of the method 500 described with reference to FIG. 5). In another example, the audio processing or rendering system 300 may be implemented in the cloud.

The audio rendering system 300 may receive (eg, receive) listening location data 301 . The audio rendering system 300 may receive listening location data 301 via an MPEG-H 3D Audio decoder input interface.

The listening location data 301 may indicate the orientation and/or position (eg, displacement) of the listener's head. Thus, the listening location data 301 (which may also be referred to as position information) may include listener orientation information and/or listener displacement information.

The listener displacement information may indicate displacement of the listener's head (eg, from a nominal listening position). The listener displacement information may correspond to or include an indication of the amplitude of displacement of the listener's head from the nominal listening position, 206, as shown in FIG. 2. In the context of the present invention, the listener displacement information indicates a slight displacement of the listener's head position from the nominal listening position. For example, the absolute value of the displacement may be no more than 0.5 m. Typically, this represents a displacement of the listener's head from the nominal listening position, which can be achieved by movement of the listener's upper body and/or head. That is, the listener can perform the shift without moving the lower body. For example, shifting the listener's head can be accomplished while the listener is sitting on a chair, as described above. The displacement can be expressed in terms of various coordinate systems, such as, for example, Cartesian coordinates (eg, in terms of x, y, z) or spherical coordinates (eg, in terms of azimuth, elevation, radius). Alternative coordinate systems for expressing the displacement of the listener's head are also possible and are understood to be covered by the present invention.

The listener orientation information may indicate the orientation of the listener's head (eg, the orientation of the listener's head relative to the nominal orientation/reference orientation of the listener's head). For example, the listener orientation information may include information about the listener's yaw, pitch, and head roll. As used herein, yaw, pitch and roll may be specified relative to the nominal orientation.

Listening location data 301 may be continuously collected from the receiver, which may provide information regarding the user's translational movements. For example, the listening location data 301 that is used in a particular time event may have been recently collected from the receiver. Listening location data can be obtained/collected/generated based on information from sensors. For example, listening location data 301 may be received/collected/generated through wearable and/or fixed equipment having suitable sensors. That is, the listener's head orientation can be detected by wearable and/or fixed equipment. Likewise, movement of the listener's head (eg, from the nominal listening position) can be detected through wearable and/or fixed equipment. The wearable equipment may be, correspond to, and/or include, for example, a headset (e.g., an AR/VR headset). The fixed equipment may be, correspond to and/or contain, for example, camera sensors. Fixed equipment can be built into, for example, a television or set-top box. In some embodiments, the listening location data 301 may be received from an audio encoder (eg, an MPEG-H 3D Audio-compatible encoder) that may have received (eg, received) sensor information.

In one example, wearable and/or fixed-mount listening location data detection equipment 301 may be referred to as tracking devices that support head position estimation/detection and/or head orientation estimation/detection. There are various solutions that allow precise tracking of user head movements using computer or smartphone cameras (for example, based on facial recognition and tracking “FaceTrackNoIR”, “opentrack”). Also, several virtual reality systems for head-mounted display (HMD) (for example, NTS VIVE, Oculus Rift) have built-in technology for tracking the user’s head position. Any of these solutions can be used in the context of the present invention.

It is also important to note that the real-world head offset distance should not correspond to the one-to-one offset indicated by the listening location data 301. To achieve a hyper-realistic effect (for example, a super-enhanced parallax effect of the user's movement), specific applications may use different sensor calibration settings or set different mappings between movement in real and virtual spaces. Therefore, a small physical movement can be expected to result in a larger displacement in VR in some use cases. In any case, it can be said that the displacement amplitudes in real conditions and in virtual reality (i.e., the displacement indicated by the 301 listening location data) are positively correlated. Similarly, the displacement directions in real and virtual reality conditions are positively correlated.

The audio rendering system 300 may further receive (object) position information 302 (eg, object position data) and audio data 322. Audio data 322 may include one or more audio objects. Position information 302 may be part of the metadata for audio data 322. Position information 302 may indicate corresponding object positions for one or more audio objects. For example, position information 302 may include an indication of the distance to corresponding audio objects relative to the nominal listening position of the user/listener. The distance (radius) may be less than 0.5 m. For example, the distance may be less than 1 cm. If the position information 302 does not include an indication of the distance of a given sound object to the nominal listening position, the audio rendering system may set the distance from that sound object to nominal listening position to the default value (eg 1 m). The position information 302 may further include indications of the elevation angle and/or azimuth of the corresponding sound objects.

Each object position can be used to render a corresponding audio object. Accordingly, position information 302 and audio data 322 may be contained in or constitute object-based audio content. Audio content (eg, audio objects/audio data 322 along with their position information 302) may be transmitted in an encoded audio bitstream. For example, the audio content may be in a bitstream format received via transmission over a network. In this case, as defined, the audio rendering system may receive audio content (eg, from an encoded audio bitstream).

In one example of the present invention, metadata parameters can be used to correct the processing of use cases with backwards compatible enhancement for 3DoF and 3DoF+. The metadata may contain information about the listener's offset in addition to information about the listener's orientation. Such metadata parameters may be used by the systems shown in FIGS. 2 and 3, as well as any other embodiments of the present invention.

A backwards-compatible enhancement may provide adjustments to the processing of use cases (eg, implementations of the present invention) based on the normative MPEG-H 3D Audio scene offset interface. This means that a legacy MPEG-H 3D Audio decoder/renderer will still provide output, even if incorrect. However, the improved MPEG-H 3D Audio decoder/renderer according to the present invention will correctly apply extension data (eg, extension metadata) and processing, and therefore can control the script of objects located near the listener in a correct manner.

In one example, the present invention relates to providing data for small translational movements of a user's head in formats other than those noted below, and the formulas can be adapted accordingly. For example, the data may be given in a format such as x, y, z coordinates (in Cartesian coordinates) instead of azimuth, elevation, and radius (in spherical coordinates). An example of these coordinate systems relative to each other is shown in FIG. 4.

In one example, the present invention relates to providing metadata (eg, listener displacement information is included in listening location data 301 shown in FIG. 3) to input translational movement of the listener's head. The metadata can be used, for example, to interface to scene offset data. Metadata (eg, listener bias information) can be obtained through the use of a tracker that supports 3DoF+ or 6DoF tracking.

In one example, metadata (e.g., listener displacement information, particularly listener head displacement, or equivalently stage displacement) may be represented by the following three parameters: sd_azimuth, sdelevation, and sd_radius, relating to azimuth, elevation, and radius (spherical coordinates ) displacement of the listener's head (or stage displacement).

The syntax for these parameters is shown in the following table.

sd_azimuth

this field denotes the scene offset azimuth position.

This field can have values from -180 to 180

sd_elevation

This field denotes the stage offset elevation angle position.

This field can have values from -90 to 90

sd_radius

this field indicates the radius of the scene offset.

This field can have values from 0.015626 to 0.25

In another example (for example, information about the listener's displacement) can be represented by three parameters sd_x, sd_y, sd_z in Deckard coordinates, thereby reducing the processing of data from spherical coordinates to Deckard coordinates. Metadata can be based on the following syntax

As described above, the above syntax or equivalents to this syntax may signal information related to rotations about the x, y, z axis.

In one example of the present invention, the processing of scene offset angles for channels and objects can be improved by extending the equations to account for changes in the user's head position. That is, processing of object positions may take into account (eg, be based at least in part on) information about the listener's displacement.

An example of a method 500 for processing position information indicating the position of an object for an audio object is shown in a flowchart in FIG. 5. This method can be performed by a decoder such as an MPEG-H 3D audio decoder. The audio rendering system 300 of FIG. 3 can act as an example of such a decoder.

In a first step (not shown in FIG. 5), audio content including an audio object and corresponding position information, for example from an encoded audio bitstream, is received. The method may then further include decoding the encoded audio content to obtain audio object and position information.

At step S510, information about the orientation of the listener is obtained (eg, received). The listener orientation information may indicate the orientation of the listener's head.

At step S520, listener offset information is acquired (eg, received). The listener displacement information may indicate the displacement of the listener's head.

In step S530, the position of the object is determined from the position information. For example, the position of an object (eg, in terms of azimuth, elevation, radius, or x, y, z, or equivalents) may be extracted from the position information. Determining the position of an object may also be based, at least in part, on information regarding the speaker geometry of one or more (real or virtual) speakers in the listening environment. If the radius is not included in the position information for this audio object, the decoder may set the radius to a default value (eg, 1 m). In some embodiments, the default value may depend on the geometry of the speakers.

It should be noted that steps S510, S520 and S520 may be performed in any order.

In step S540, the object position determined in step S530 is modified based on the listener displacement information. This can be accomplished by applying a translational movement to the position of the object in accordance with the displacement information (eg, in accordance with the displacement of the listener's head). Thus, as defined, object position modification may refer to an adjustment of the object position to move the listener's head (eg, a shift from the nominal listening position). In particular, modification of the object's position based on the listener's displacement information can be accomplished by translationally moving the object's position using a vector that is positively correlated with the amplitude and negatively correlated with the direction of the listener's head displacement vector from the nominal listening position. An example of such translational movement is illustrated schematically in FIG. 2.

In step S550, the modified object position obtained in step S540 is further modified based on the listener orientation information. For example, this may be accomplished by applying a rotational transformation to a modified object position in accordance with the listener's orientation information. This rotation may be, for example, a rotation relative to the listener's head or nominal listening position. Rotational transformation can be performed using a scene shifting algorithm.

As noted above, user displacement compensation (ie, modification of the object's position based on listener displacement information) is taken into account when applying a rotational transform. For example, applying a rotation transform might include:

• calculating a rotation transformation matrix (based on user orientation, such as listener orientation information);

• converting the object's position from spherical to Cartesian coordinates;

• applying a rotational transformation to audio objects that compensate for the deviation of the user's position (ie, to a modified object position); And

• converting the object's position after a rotational transformation back from Cartesian to spherical coordinates.

In a next step S560 (not shown in FIG. 5), method 500 may include rendering an audio object on one or more real or virtual speakers in accordance with a further modified position of the object. To this end, the further modified object position may be adjusted for the input format used by the MPEG-H 3D Audio renderer (eg, audio object renderer 320 described above). The above one or more (real or virtual) speakers may be, for example, part of a headset, or may be part of a speaker arrangement (eg, a 2.1 speaker arrangement, a 5.1 speaker arrangement, a 7.1 speaker arrangement, etc.). In some embodiments, the audio object may be rendered to, for example, the left and right speakers of a headset.

The purpose of steps S540 and S550 described above is as follows. Namely, the modification of the object position and the further modification of the modified object position are performed such that the sound object, after rendering on one or more (real or virtual) speakers in accordance with the further modified object position, is psychoacoustically perceived by the listener as arising from a fixed position relative to the nominal listening position . This fixed position of the sound object must be psychoacoustically perceived regardless of the displacement of the listener's head from the nominal listening position and regardless of the orientation of the listener's head relative to the nominal orientation. In other words, the sound object may be perceived as moving (translationally) relative to the listener's head when the listener's head performs a displacement from the nominal listening position. Likewise, an audio object may be perceived as moving (rotating) relative to the listener's head when the listener's head makes a change in orientation from its nominal orientation. Thus, the listener can perceive a close sound object from different angles and distances through the movement of his head.

The object position modification and further modification of the modified object position in steps S540 and S550, respectively, can be performed in the context of (rotational/translational) sound stage shifting, for example, by the sound stage shifting unit 310 described above.

It should be noted that specific steps may be skipped depending on the specific use case. For example, if the listening location data 301 contains only listener displacement information (but does not contain listener orientation information, or only listener orientation information indicating that there is no deviation of the listener's head orientation from the nominal orientation), step S550 may be skipped. Then, rendering in step S560 will be performed in accordance with the modified object position determined in step S540. Likewise, if the listening location data 301 contains only listener orientation information (but does not contain listener offset information, or only listener offset information indicating that there is no deviation of the listener's head position from the nominal listening position), step S540 may be skipped. Next, step S550 will relate to modifying the position of the object determined in step S530 based on the listener's orientation information. The rendering in step S560 will be performed in accordance with the modified position determined in step S550.

In general, the present invention proposes position updating for object positions obtained as part of object-based audio content (eg, position information 302 along with audio data 322) based on listening location data 301 for the listener.

First, determine the position of the object (or position of the channel) This may be performed in the context of (e.g., as part of) step 530 of method 500.

For channel-based signals, the radius r can be determined as follows:

- if the intended loudspeaker (of a channel for a channel-based input signal) exists in the playback speaker layout and the distance to the playback layout is known, the radius r is set to the distance to the loudspeaker (eg in cm);

- if the intended loudspeaker does not exist in the playback loudspeaker arrangement, but the distance to the playback loudspeakers (for example, from the nominal listening position) is known, the radius r is set to the maximum distance to the playback loudspeaker;

- if the intended loudspeaker does not exist in the playback speaker layout and the distance to the playback speaker is not known, the radius r is set to the default value (for example, 1023 cm).

For object-based signals, the radius r is determined as follows:

- if the distance to the object is known (for example, from production tools and from production formats and passed to prodMetadataConfig(), the radius r is set to the known distance to the object (for example, passed through goa_bsObjectDistance[] (in cm) according to the AMD5.7 table of the MPEG standard -H 3D Audio);

- if the distance to the object is known from the position information (for example, from the object's metadata and passed to object_metadata()), the radius r is set to the distance to the object passed in the position information (for example, to the radius [] (in cm) passed with object metadata). The radius r can be passed according to the sections: “object metadata scaling” and “object metadata constraint” presented below.

Scaling object metadata

As an optional step in the context of determining the position of an object, the position of the object determined from position information can be scaled. This may involve applying a scaling factor to descale the input data encoder for each component. This can be done for each object. The actual scaling of the object's position can be implemented along with the pseudo code below:

Limiting Object Metadata

As an optional step in the context of determining the position of an object (possibly scaled), the object position p=(az,el,r) determined from the position information may be constrained. This may include applying a per-component decoded value constraint to keep the values within an acceptable range. This can be done for each object. The actual constraint on the object's position can be implemented according to the functionality of the pseudocode presented below:

After this, the defined (and optionally scaled and/or constrained) position of the object can be converted to a given coordinate system, such as, for example, a coordinate system according to the "usual convention" in which 0° azimuth is at the right ear (positive values are counterclockwise) and 0° elevation is at the top of the head (positive values) values go downwards). Thus, the position of an object p can be converted into a position p' according to the "usual" convention. This provides the object's position p's

with constant radius r.

At the same time, the listener's head offset indicated by the listener offset information can be converted to a given coordinate system. Using the "usual convention" this amounts to

with a constant radius r _offset .

It should be noted that conversion to a given coordinate system for both the object position and the listener's head displacement may be performed in the context of step S530 or step S540.

The actual position update may be performed in the context of (eg, as part of) step S540 of method 500. The position update may include the following steps.

As a first step, position p or, if a transfer to a given coordinate system has been carried out, position p', is transferred to Cartesian coordinates (x, y, z). Further, without any intended limitation, the process will be described for the position p' in a given coordinate system. Also without intended limitation, the following orientation/direction of the coordinate axes can be assumed to be: the x-axis points to the right (shown from the listener's head in the nominal orientation), the y-axis points straight ahead, and the z-axis points straight up. At the same time, the listener's head offset indicated by the listener offset information converted to Cartesian coordinates.

As a second step, the position of the object in Cartesian coordinates is shifted (progressively moved) in accordance with the displacement of the listener's head (scene displacement) in the manner described above. This can be done through

The above translational movement is an example of modifying the object's position based on the listener's displacement information in step S540 of the method 500.

The shifted position of the object in Cartesian coordinates is converted to spherical coordinates and may be called p''. The shifted position of an object can be expressed in a given coordinate system according to the usual convention as

When there are listener head displacements that provide a small change in the radius parameter (i.e. r'≈r), the modified object position p'' can be redefined as

In another example, when there are large displacements of the listener's head that can provide a significant change in the radius parameter (i.e. r''»r), the modified position p'' of the object can also be defined as instead of with a modified radius parameter r'.

The corresponding value of the modified radius parameter r' can be obtained from the distance of the listener's head displacement (i.e. ) and the initial radius parameter (i.e. ), (see, for example, Fig. 1 and 2). For example, the modified radius parameter r' can be determined based on the following trigonometric relationship:

Converting this modified radius parameter r' into an object/channel gain and applying it to subsequent audio rendering can greatly improve the effects of perceiving level changes due to user movement. By providing such a modification of the radius parameter r', an “adaptive sweet spot” is provided. This will mean that the MPEG rendering system dynamically adjusts the position of the sweet spot according to the listener's current location. In general, rendering an audio object according to a modified (or further modified) object position may be based on a modified radius parameter r'. In particular, object/channel gains for rendering an audio object may be based on (eg, modified based on) a modified radius parameter r'.

In another example, during setup and rendering of the playback speaker (eg, step S560 above), scene shifting may be disabled. However, the optional inclusion of scene shift is available. This ensures that the 3DoF+ renderer creates a dynamically adjustable sweet spot based on the listener's current location and orientation.

It should be noted that the step of converting the object position and listener's head displacement into Cartesian coordinates is optional, and the translational movement/shift (modification) according to the listener's head displacement (scene displacement) can be performed in any suitable coordinate system. In other words, the choice of Cartesian coordinates in the above description should be understood as a non-limiting example.

In some embodiments, scene offset processing (including object position modification and/or further modification of the modified object position) may be enabled or disabled via a flag (field, element, set bit) in the bitstream (eg, a useTrackingMode element). The subclauses “17.3 Interface for local loudspeaker setup and rendering) and “17.4 Interface for binaural room impulse responses (BRIRs)” in ISO/IEC 23008-3 contain descriptions of the useTrackingMode element that enables scene offset processing. In the context of the present invention, the useTrackingMode element shall determine (clause 17.3) whether processing of scene offset values sent through the mpegh3daSceneDisplacementData() and mpegh3daPositionalSceneDisplacementData() interfaces should or should not occur. Alternatively or additionally (subclause 17.4), the useTrackingMode field shall specify whether a tracking device is connected and whether the binaural rendering should be processed in a specific user head tracking mode, which means that the processing of scene offset values sent through the mpegh3daSceneDisplacementData() and mpegh3daPositionalSceneDisplacementData() interfaces , must be implemented.

The methods and systems described herein may be implemented as software, firmware, and/or hardware. Some components may be implemented, for example, as software running on a digital signal processor or microprocessor. Other components may be implemented, for example, as hardware or as special purpose integrated circuits. The signals encountered in the described methods and systems may be stored on media such as random access memory or optical storage media. They can be transmitted over networks such as radio networks, satellite networks, wireless networks or wired networks such as the Internet. Typical devices using the methods and systems described herein are portable electronic devices or other consumer equipment that are used to store and/or generate audio signals.

Although references are made herein to MPEG and, in particular, MPEG-H 3D Audio, the present invention should not be construed as limited to these standards. On the contrary, as will be appreciated by those skilled in the art, the present invention may also find advantageous application in other audio coding standards.

In addition, although reference is often made herein to small shifts in the position of a listener's head (eg, from a nominal listening position), the present invention is not limited to small shifts in position and can generally be applied to arbitrary shifts in the position of a listener's head.

It should be noted that the description and graphic materials illustrate only the principles of the proposed methods, systems and apparatus. Those skilled in the art will be able to implement various circuits that, although not expressly described or shown herein, implement the principles of the present invention and are included within the spirit and scope thereof. Moreover, all examples and embodiment set forth herein are primarily and expressly intended for explanatory purposes to assist the reader in understanding the principles of the proposed method. In addition, all statements herein presenting the principles, aspects and embodiments of the present invention, as well as specific examples thereof, are intended to cover their equivalents.

In addition to the above, various exemplary implementations and exemplary embodiments of the present invention will become clear from the numbered example embodiments (NEXs) listed below, which are not claims.

The first APR relates to a method for decoding an encoded bitstream of an audio signal, the method comprising: receiving, by an audio decoding apparatus 300, an encoded bitstream (302, 322) of an audio signal, wherein the encoded bitstream of the audio signal contains encoded audio data (322) and metadata, corresponding to at least one audio signal (302) of the object; decoding by the audio decoding apparatus (300) the encoded bit stream (302, 322) of the audio signal to obtain a representation of multiple audio sources; reception by the device (300) of audio decoding of listening location data (301); generating by the audio decoding apparatus (300) sound object position data (321), wherein the sound object position data (321) describes multiple sound sources relative to the listening location based on the listening location data (301).

The second TPPR refers to the method according to the first TPPR, wherein the listening location data (301) is based on the first set of first translational position data and the second set of second translational position and orientation data.

The third TDPR refers to the method according to the second TDPR, wherein either the first translational position data or the second translational position data is based on at least one of a set of spherical coordinates or a set of Cartesian coordinates.

The fourth TPPR refers to the method according to the first TPPR, wherein the listening location data (301) is obtained through the input interface of the MPEG-H 3D Audio decoder.

The fifth VPOP relates to the method according to the first VPOP, wherein the encoded audio bitstream comprises MPEG-H 3D Audio bitstream syntax elements, and wherein the MPEG-H 3D Audio bitstream syntax elements comprise encoded audio data (322) and metadata corresponding to at least one sound signal (302) of the object.

The sixth VPRO refers to the method according to the first VPRO, which further includes rendering by the apparatus 300 of audio decoding on multiple speakers of multiple audio sources, the rendering process being compatible with at least the MPEG-H 3D Audio standard.

The seventh OPER refers to the method according to the first OPER, which further includes converting the audio decoding apparatus (300) based on the broadcast of listening location data (301) of a position p corresponding to at least one audio signal (302) of the object into a second position p'', corresponding to the positions (321) of the sound object.

The eighth OPER refers to the method according to the seventh OPER, wherein the position p' for the positions of the sound object in a given coordinate system (for example, in accordance with the usual convention) is determined based on:

where az corresponds to the first azimuth parameter, corresponds to the first elevation angle parameter and r corresponds to the first radius parameter, herein az' corresponds to the second azimuth parameter, corresponds to the second parameter of the elevation angle and r' corresponds to the second parameter of the radius, with az _0ffset corresponding to the third parameter of the azimuth, corresponds to the third parameter of the elevation angle, and at the same time az' _0ffset corresponds to the fourth parameter of the azimuth, corresponds to the fourth parameter of the elevation angle.

The ninth OPER refers to the method according to the eighth OPER, wherein the shifted position p'' (321) of the sound object for the position (302) of the sound object is determined in Cartesian coordinates (x, y, z) based on:

wherein the position in Cartesian coordinates (x, y, z) contains the parameters x, y and z, and wherein x _0ffset refers to the first x-axis deviation parameter, y _0ffset refers to the first y-axis deviation parameter, and z _0ffset refers to the first deviation parameter z axis

The tenth OPVO refers to the method according to the ninth OPPO, where the parameters x_0ffset,y_0ffsetAnd z_0ffsetbased on

The eleventh TFR refers to the method according to the seventh TFR, wherein the azimuth parameter az _0ffset refers to the scene offset azimuth position and is based on:

wherein sd_azimuth is a parameter indicating the azimuth offset of the MPEG-H 3DA scene, wherein the elevation angle parameter el _offset refers to the elevation angle position of the scene offset and is based on:

wherein sd_elevation is an elevation angle metadata parameter indicating the elevation angle offset of the MPEG-H 3DA scene, wherein the radius parameter r _0ffset refers to the scene offset radius and is based on:

wherein sd_radius is a radius metadata parameter indicating the offset of the MPEG-H 3DA scene radius, and wherein the X and Y parameters are scalar variables.

The twelfth TDPR refers to the method according to the tenth TDPR, wherein the parameter x _0jjset refers to the scene offset deviation position sd_x in the x-axis direction; the y _offset parameter refers to the scene offset deviation position sd_y in the y-axis direction; and the parameter z _0ffset refers to the stage offset deviation position sd_z in the z-axis direction.

The thirteenth TARP refers to the method according to the first TARP, which further includes interpolating, by the audio decoding apparatus, the first position data related to the listening location data (301) and the object audio signal (102) at an update rate.

The fourteenth OPER refers to the method according to the first OPER, which further includes determining by the audio decoding apparatus 300 an effective entropy encoding of the listening location data (301).

The fifteenth OPER refers to the method according to the first OPER, wherein position data related to the listening location (301) is obtained based on information from the sensors.

Claims (25)

1. A method (500) for processing position information indicating the position of an object for an audio object, wherein the processing is performed using an MPEG-H 3D Audio decoder, wherein the position of the object can be used to render the audio object, wherein the method includes: obtaining (S510) listener orientation information indicating the listener's head orientation; obtaining (S520) listener offset information indicating an offset of the listener's head relative to the nominal listening position through an input interface of the MPEG-H 3D Audio decoder; determining (S530) the position of the object from the position information; modifying (S540) the object's position based on the listener's displacement information by applying translational movement to the object's position; And further modifying (S550) the modified position of the object based on the listener orientation information, wherein when the listener displacement information indicates the displacement of the listener's head from the nominal listening position due to a small position displacement, and the small position displacement has an absolute value of 0.5 meters or less than 0.5 meters, the distance between the position of the sound object and the listening position after the head displacement listener is equal to the distance between the modified sound object position and the nominal listening position. 2. Method (500) according to claim 1, characterized in that: modification (S540) of the object position and further modification (S550) of the modified object position are performed such that the sound object, after rendering on one or more real or virtual speakers in accordance with the additional modified object position, is psychoacoustically perceived by the listener as arising from a fixed position relative to the nominal position listening regardless of the displacement of the listener's head from the nominal listening position and the orientation of the listener's head relative to the nominal orientation. 3. Method (500) according to claim 1 or 2, characterized in that: modification (S540) of the object position based on the listener displacement information is performed by progressively moving the object position equal to the displacement of the listener's head from the nominal listening position, but in the opposite direction. 4. Method (500) according to any one of paragraphs. 1-3, characterized in that: listener displacement information indicates displacement of the listener's head from the nominal listening position, which can be achieved by movement of the listener's upper body and/or head. 5. Method (500) according to any one of paragraphs. 1-4, characterized in that it additionally includes: detecting the listener's head orientation using wearable and/or stationary equipment. 6. Method (500) according to any one of paragraphs. 1-5, characterized in that it additionally includes: detection of the listener's head displacement from the nominal listening position using wearable and/or fixed equipment. 7. Method (500) according to any one of paragraphs. 1-6, characterized in that the distance between the modified position of the sound object and the listening position after the offset is converted into gain to modify the level of the sound signal. 8. An MPEG-H 3D Audio decoder (300) for processing position information indicating an object position (321) for an audio object, wherein the object position can be used to render the audio object, the decoder comprising a processor and a storage device coupled to the processor , while the processor is adapted for: obtaining listener orientation information indicating the orientation of the listener's head; obtaining listener displacement information indicating a displacement of the listener's head relative to the nominal listening position through an input interface of the MPEG-H 3D Audio decoder; determining the position of an object from the position information; modifying the position of an object based on information about the listener's displacement by applying translational movement to the position of the object; And additional modification of the modified position of the object based on information about the orientation of the listener, and when the listener displacement information indicates a displacement of the listener's head from the nominal listening position due to a small position shift, wherein the small position shift has an absolute value of 0.5 meters or less than 0.5 meters, the processor is configured to maintain a distance between the position of the sound object and listening position after displacement of the listener's head equal to the distance between the modified position of the sound object and the nominal listening position.