CN117476014A - Audio processing methods, devices, storage media and equipment - Google Patents

Audio processing methods, devices, storage media and equipment Download PDF

Info

Publication number
CN117476014A
CN117476014A CN202210869408.7A CN202210869408A CN117476014A CN 117476014 A CN117476014 A CN 117476014A CN 202210869408 A CN202210869408 A CN 202210869408A CN 117476014 A CN117476014 A CN 117476014A
Authority
CN
China
Prior art keywords
audio
virtual space
field
target
target virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210869408.7A
Other languages
Chinese (zh)
Inventor
黄祺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Priority to CN202210869408.7A priority Critical patent/CN117476014A/en
Publication of CN117476014A publication Critical patent/CN117476014A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Stereophonic System (AREA)

Abstract

本申请公开一种音频处理方法、装置、存储介质及设备,该方法包括:获取发声者的近场音频;获取目标虚拟空间的音频配置参数,音频配置参数用于模拟目标虚拟空间的空间音效;根据目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到第一目标音频,第一目标音频具有目标虚拟空间的空间音效。本申请实施例通过预先构建的目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到具有目标虚拟空间的空间音效的音频,提升了音频效果,提升了用户体验。

This application discloses an audio processing method, device, storage medium and equipment. The method includes: obtaining the near-field audio of the speaker; obtaining the audio configuration parameters of the target virtual space, and the audio configuration parameters are used to simulate the spatial sound effects of the target virtual space; According to the audio configuration parameters of the target virtual space, the near-field audio is mixed to obtain the first target audio, and the first target audio has the spatial sound effect of the target virtual space. The embodiment of the present application mixes the near-field audio through the pre-constructed audio configuration parameters of the target virtual space to obtain audio with spatial sound effects of the target virtual space, thereby improving the audio effect and user experience.

Description

音频处理方法、装置、存储介质及设备Audio processing methods, devices, storage media and equipment

技术领域Technical field

本申请涉及虚拟现实技术领域,具体涉及一种音频处理方法、装置、存储介质及设备。This application relates to the field of virtual reality technology, and specifically to an audio processing method, device, storage medium and equipment.

背景技术Background technique

动捕音频的音频收集地点为动捕棚,但是,动捕棚的环境比较单一,不可能因为本次需要录一个音乐会,那就需要专门寻找设置在一个音乐会里面的动捕棚。动捕棚一般都是设置在一个固定的场所,动捕棚的音频收集方式一般采用靠近发声者的嘴边的近场麦克风进行收集的,近场麦克风收集到的音频也可以称为近场音频。近场音频比较单调,没有回声,音频效果不理想。The audio collection location for motion capture audio is the motion capture studio. However, the environment of the motion capture studio is relatively simple. It is impossible to record a concert this time, so you need to specifically look for a motion capture studio set up in a concert. The motion capture studio is generally set up in a fixed place. The audio collection method in the motion capture studio generally uses a near-field microphone close to the speaker's mouth. The audio collected by the near-field microphone can also be called near-field audio. . Near-field audio is relatively monotonous, has no echo, and the audio effect is not ideal.

发明内容Contents of the invention

本申请实施例提供一种音频处理方法、装置、存储介质、设备及程序产品,可以基于预先构建的目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到具有目标虚拟空间的空间音效的音频,提升了音频效果,提升了用户体验。Embodiments of the present application provide an audio processing method, device, storage medium, equipment and program product, which can perform mixing processing on near-field audio based on the pre-constructed audio configuration parameters of the target virtual space to obtain a space with the target virtual space. The audio of the sound effects improves the audio effect and enhances the user experience.

一方面,本申请实施例提供一种音频处理方法,所述方法包括:On the one hand, embodiments of the present application provide an audio processing method, which includes:

获取发声者的近场音频;Get the near-field audio of the speaker;

获取目标虚拟空间的音频配置参数,所述音频配置参数用于模拟所述目标虚拟空间的空间音效;Obtain the audio configuration parameters of the target virtual space, the audio configuration parameters being used to simulate the spatial sound effects of the target virtual space;

根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,所述第一目标音频具有所述目标虚拟空间的空间音。According to the audio configuration parameters of the target virtual space, the near-field audio is mixed to obtain the first target audio, and the first target audio has the spatial sound of the target virtual space.

在一些实施例中,所述目标虚拟空间的音频配置参数包括所述目标虚拟空间的音频反射参数,音频传播的衰减参数,以及所述目标虚拟空间的背景音频。In some embodiments, the audio configuration parameters of the target virtual space include audio reflection parameters of the target virtual space, attenuation parameters of audio propagation, and background audio of the target virtual space.

在一些实施例中,所述根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,包括:In some embodiments, mixing the near-field audio according to the audio configuration parameters of the target virtual space to obtain the first target audio includes:

根据所述音频反射参数与所述衰减参数,对所述近场音频与所述背景音频进行第一音频调节处理;Perform a first audio adjustment process on the near-field audio and the background audio according to the audio reflection parameter and the attenuation parameter;

对进行第一音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第一目标音频。The near-field audio and the background audio that have been subjected to the first audio adjustment processing are mixed to obtain the first target audio.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

获取所述发声者对应的虚拟发声对象在所述目标虚拟空间中的第一位置,以及收听者对应的虚拟收听对象在所述目标虚拟空间中的第二位置;Obtain the first position of the virtual speaking object corresponding to the speaker in the target virtual space, and the second position of the virtual listening object corresponding to the listener in the target virtual space;

所述根据所述音频反射参数与所述衰减参数,对所述近场音频与所述背景音频进行第一音频调节处理,包括:Performing a first audio adjustment process on the near-field audio and the background audio according to the audio reflection parameter and the attenuation parameter includes:

根据所述音频反射参数,所述衰减参数,所述第一位置与所述第二位置,对所述近场音频与所述背景音频进行第一音频调节处理。According to the audio reflection parameter, the attenuation parameter, the first position and the second position, a first audio adjustment process is performed on the near-field audio and the background audio.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理;When it is detected that the relative positional relationship between the first position and the second position changes, the near-field audio and the relative positional relationship between the first position and the second position are compared. The background audio is subjected to a second audio adjustment process;

对进行第二音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第二目标音频,所述第二目标音频具有所述目标虚拟空间的空间音效、以及表征所述第一位置与所述第二位置之间的相对位置关系发生变化时产生的音效变化效果。The near-field audio and the background audio after the second audio adjustment processing are mixed to obtain a second target audio. The second target audio has the spatial sound effect of the target virtual space and represents the target virtual space. The sound effect changes when the relative positional relationship between the first position and the second position changes.

在一些实施例中,所述相对位置关系包括距离关系,所述当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理,包括:In some embodiments, the relative position relationship includes a distance relationship, and when a change in the relative position relationship between the first position and the second position is detected, based on the first position and the The relative positional relationship between the second positions is used to perform a second audio adjustment process on the near-field audio and the background audio, including:

当检测到所述第一位置与所述第二位置之间的距离关系发生变化时,根据所述第一位置与所述第二位置之间的距离对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the distance relationship between the first position and the second position changes, the near-field audio and the background audio are compared according to the distance between the first position and the second position. Perform second audio adjustment processing.

在一些实施例中,所述相对位置关系包括方位关系,所述当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理,包括:In some embodiments, the relative position relationship includes an orientation relationship, and when a change in the relative position relationship between the first position and the second position is detected, the relative position relationship between the first position and the second position is determined based on the first position and the second position. The relative positional relationship between the second positions is used to perform a second audio adjustment process on the near-field audio and the background audio, including:

当检测到所述第一位置与所述第二位置之间的方位关系发生变化时,根据所述第一位置与所述第二位置之间的相对方位对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the orientation relationship between the first position and the second position changes, the near-field audio and the background are compared according to the relative orientation between the first position and the second position. The audio undergoes a second audio adjustment process.

在一些实施例中,所述相对位置关系包括距离关系与方位关系,所述当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理,包括:In some embodiments, the relative position relationship includes a distance relationship and an orientation relationship, and when a change in the relative position relationship between the first position and the second position is detected, based on the first position Performing a second audio adjustment process on the near-field audio and the background audio based on the relative positional relationship with the second position, including:

当检测到所述第一位置与所述第二位置之间的距离关系与方位关系发生变化时,根据所述第一位置与所述第二位置之间的距离与相对方位,对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the distance relationship and the azimuth relationship between the first position and the second position change, based on the distance and relative azimuth between the first position and the second position, the near The field audio and the background audio undergo a second audio adjustment process.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

当检测到所述收听者的头部摆动时,获取头部摆动过程中所述收听者的左耳朝向方位和右耳朝向方位;When the head swing of the listener is detected, the left ear orientation and the right ear orientation of the listener during the head swing are obtained;

根据所述收听者的左耳朝向方位和右耳朝向方位对所述近场音频与所述背景音频进行第三音频调节处理;Perform a third audio adjustment process on the near-field audio and the background audio according to the listener's left ear orientation and right ear orientation;

对进行第三音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第三目标音频,所述第三目标音频具有所述目标虚拟空间的空间音效、以及表征所述收听者的头部摆动时产生的音效变化效果。The near-field audio and the background audio after the third audio adjustment processing are mixed to obtain a third target audio. The third target audio has the spatial sound effect of the target virtual space and represents the target virtual space. Describes the sound effect changes produced when the listener's head moves.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

根据不同的虚拟场景,预先设定不同类型的虚拟空间,其中,每一类型的虚拟空间具有对应的音频配置参数;Different types of virtual spaces are preset according to different virtual scenes, where each type of virtual space has corresponding audio configuration parameters;

所述获取目标虚拟空间的音频配置参数,包括:The obtaining the audio configuration parameters of the target virtual space includes:

响应于针对虚拟空间的选择指令,从所述不同类型的虚拟空间中确定出所述目标虚拟空间,并获取所述目标虚拟空间的音频配置参数。In response to a selection instruction for a virtual space, the target virtual space is determined from the different types of virtual spaces, and audio configuration parameters of the target virtual space are obtained.

另一方面,本申请实施例提供一种音频处理装置,所述装置包括:On the other hand, embodiments of the present application provide an audio processing device, which includes:

第一获取单元,用于获取发声者的近场音频;The first acquisition unit is used to acquire the near-field audio of the speaker;

第二获取单元,用于获取目标虚拟空间的音频配置参数,所述音频配置参数用于模拟所述目标虚拟空间的空间音效;The second acquisition unit is used to acquire the audio configuration parameters of the target virtual space, where the audio configuration parameters are used to simulate the spatial sound effects of the target virtual space;

处理单元,用于根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,所述第一目标音频具有所述目标虚拟空间的空间音效。A processing unit configured to perform mixing processing on the near-field audio according to the audio configuration parameters of the target virtual space to obtain a first target audio, where the first target audio has a spatial sound effect of the target virtual space.

另一方面,本申请实施例提供一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,所述计算机程序适于处理器进行加载,以执行如上任一实施例所述的音频处理方法。On the other hand, embodiments of the present application provide a computer-readable storage medium that stores a computer program, and the computer program is suitable for loading by a processor to execute the steps described in any of the above embodiments. Audio processing methods.

另一方面,本申请实施例提供一种虚拟现实设备,所述虚拟现实设备包括处理器和存储器,所述存储器中存储有计算机程序,所述处理器通过调用所述存储器中存储的所述计算机程序,用于执行如上任一实施例所述的音频处理方法。On the other hand, embodiments of the present application provide a virtual reality device. The virtual reality device includes a processor and a memory. A computer program is stored in the memory. The processor invokes the computer program stored in the memory. Program, used to execute the audio processing method described in any of the above embodiments.

另一方面,本申请实施例提供一种服务器,所述服务器包括处理器和存储器,所述存储器中存储有计算机程序,所述处理器通过调用所述存储器中存储的所述计算机程序,用于执行如上任一实施例所述的音频处理方法。On the other hand, embodiments of the present application provide a server. The server includes a processor and a memory. A computer program is stored in the memory. The processor calls the computer program stored in the memory. Execute the audio processing method as described in any of the above embodiments.

另一方面,本申请实施例提供一种计算机程序产品,包括计算机程序,所述计算机程序被处理器执行时实现如上任一实施例所述的音频处理方法。On the other hand, embodiments of the present application provide a computer program product, which includes a computer program. When the computer program is executed by a processor, the audio processing method as described in any of the above embodiments is implemented.

本申请实施例通过获取发声者的近场音频;获取目标虚拟空间的音频配置参数,音频配置参数用于模拟目标虚拟空间的空间音效;根据目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到第一目标音频,第一目标音频具有目标虚拟空间的空间音效。本申请实施例通过预先构建的目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到具有目标虚拟空间的空间音效的音频,提升了音频效果,提升了用户体验。The embodiment of the present application obtains the near-field audio of the speaker; obtains the audio configuration parameters of the target virtual space, and the audio configuration parameters are used to simulate the spatial sound effects of the target virtual space; and mixes the near-field audio according to the audio configuration parameters of the target virtual space. Sound processing is performed to obtain the first target audio, which has the spatial sound effect of the target virtual space. The embodiment of the present application mixes the near-field audio through the pre-constructed audio configuration parameters of the target virtual space to obtain audio with spatial sound effects of the target virtual space, thereby improving the audio effect and user experience.

附图说明Description of the drawings

为了更清楚地说明本申请实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

图1为本申请实施例提供的音频处理方法的流程示意图。Figure 1 is a schematic flowchart of an audio processing method provided by an embodiment of the present application.

图2为本申请实施例提供的音频处理方法的应用场景示意图。Figure 2 is a schematic diagram of an application scenario of the audio processing method provided by the embodiment of the present application.

图3为本申请实施例提供的音频处理装置的结构示意图。Figure 3 is a schematic structural diagram of an audio processing device provided by an embodiment of the present application.

图4为本申请实施例提供的虚拟现实设备的第一结构示意图。Figure 4 is a first structural schematic diagram of a virtual reality device provided by an embodiment of the present application.

图5为本申请实施例提供的虚拟现实设备的第二结构示意图。Figure 5 is a second structural schematic diagram of a virtual reality device provided by an embodiment of the present application.

具体实施方式Detailed ways

下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of protection of this application.

本申请实施例提供一种音频处理方法、装置、计算机可读存储介质、虚拟现实设备、服务器及计算机程序产品。具体地,本申请实施例的音频处理方法可以由虚拟现实设备或者由服务器执行。Embodiments of the present application provide an audio processing method, device, computer-readable storage medium, virtual reality device, server and computer program product. Specifically, the audio processing method in the embodiment of the present application can be executed by a virtual reality device or a server.

本申请实施例可以应用于扩展现实(eXtend ed Rea lity,XR)、虚拟现实(Virtual Reality,VR)、增强现实(Augmented Reality,AR)、混合现实(Mixed Reality,MR)等各种应用场景。The embodiments of the present application can be applied to various application scenarios such as extended reality (eXtended Reality, XR), virtual reality (Virtual Reality, VR), augmented reality (Augmented Reality, AR), and mixed reality (Mixed Reality, MR).

首先,在对本申请实施例进行描述的过程中出现的部分名词或者术语作如下解释:First, some nouns or terms that appear in the description of the embodiments of this application are explained as follows:

扩展现实(eXtend ed Rea lity,XR),是包括虚拟现实(Virtual Reality,VR)、增强现实(Augumented Reality,AR)及混合现实(Mixed Reality,MR)的概念,表示制成虚拟世界与现实世界相连接的环境,用户能够与该环境实时交互的技术。Extended reality (eXtend ed Reality, Connected environments, technologies that enable users to interact with that environment in real time.

虚拟现实(Virtual Reality,VR),创建和体验虚拟世界的技术,计算生成一种虚拟环境,是一种多源信息(本文中提到的虚拟现实至少包括视觉感知,此外还可以包括听觉感知、触觉感知、运动感知,甚至还包括味觉感知、嗅觉感知等),实现虚拟环境的融合的、交互式的三维动态视景和实体行为的仿真,使用户沉浸到模拟的虚拟现实环境中,实现在诸如地图、游戏、视频、教育、医疗、模拟、协同训练、销售、协助制造、维护和修复等多种虚拟环境的应用。Virtual Reality (VR), a technology for creating and experiencing virtual worlds, calculates and generates a virtual environment, which is a kind of multi-source information (the virtual reality mentioned in this article at least includes visual perception, and can also include auditory perception, Tactile perception, motion perception, and even taste perception, smell perception, etc.), realizing the integration of virtual environment, interactive three-dimensional dynamic vision and simulation of entity behavior, allowing users to immerse themselves in the simulated virtual reality environment and realize Applications in various virtual environments such as maps, games, videos, education, medical care, simulation, collaborative training, sales, assisted manufacturing, maintenance and repair, etc.

增强现实(Augmented Reality,AR),一种在相机采集图像的过程中,实时地计算相机在现实世界(或称三维世界、真实世界)中的相机姿态参数,根据该相机姿态参数在相机采集的图像上添加虚拟元素的技术。虚拟元素包括但不限于:图像、视频和三维模型。AR技术的目标是在屏幕上把虚拟世界套接在现实世界上进行互动。Augmented Reality (AR) is a method that calculates the camera's posture parameters in the real world (or three-dimensional world, real world) in real time during the process of camera collecting images. A technique for adding virtual elements to images. Virtual elements include, but are not limited to: images, videos, and three-dimensional models. The goal of AR technology is to connect the virtual world to the real world on the screen for interaction.

混合现实(Mixed Reality,MR),将计算机创建的感官输入(例如,虚拟对象)与来自物理布景的感官输入或其表示集成的模拟布景,一些MR布景中,计算机创建的感官输入可以适应于来自物理布景的感官输入的变化。另外,用于呈现MR布景的一些电子系统可以监测相对于物理布景的取向和/或位置,以使虚拟对象能够与真实对象(即来自物理布景的物理元素或其表示)交互。例如,系统可监测运动,使得虚拟植物相对于物理建筑物看起来是静止的。Mixed Reality (MR), a simulated set that integrates computer-created sensory input (e.g., virtual objects) with sensory input from the physical set or its representation. In some MR sets, the computer-created sensory input can be adapted to the sensory input from the physical set. Changes in sensory input to the physical set. Additionally, some electronic systems used to render the MR scenery may monitor orientation and/or position relative to the physical scenery to enable virtual objects to interact with real objects (i.e., physical elements from the physical scenery or representations thereof). For example, the system can monitor motion so that virtual plants appear stationary relative to physical buildings.

增强虚拟(Augmented Virtuality,AV):AV布景是指计算机创建布景或虚拟布景并入来自物理布景的至少一个感官输入的模拟布景。来自物理布景的一个或多个感官输入可为物理布景的至少一个特征的表示。例如,虚拟对象可呈现由一个或多个成像传感器捕获的物理元素的颜色。又如,虚拟对象可呈现出与物理布景中的实际天气条件相一致的特征,如经由天气相关的成像传感器和/或在线天气数据所识别的。在另一个示例中,增强现实森林可具有虚拟树木和结构,但动物可具有从对物理动物拍摄的图像精确再现的特征。Augmented Virtuality (AV): AV scenery refers to a computer-created scenery or a simulated scenery that incorporates at least one sensory input from a physical scenery. The one or more sensory inputs from the physical setting may be a representation of at least one feature of the physical setting. For example, a virtual object may take on the color of a physical element captured by one or more imaging sensors. As another example, virtual objects may exhibit characteristics consistent with actual weather conditions in the physical scene, as identified via weather-related imaging sensors and/or online weather data. In another example, an augmented reality forest can have virtual trees and structures, but the animals can have features accurately recreated from images taken of physical animals.

虚拟视场,用户在虚拟现实设备中通过透镜所能够感知到的虚拟环境中的区域,使用虚拟视场的视场角(Field Of View,FOV)来表示所感知到区域。Virtual field of view refers to the area in the virtual environment that the user can perceive through the lens in the virtual reality device. The field of view (Field Of View, FOV) of the virtual field of view is used to represent the perceived area.

虚拟现实设备,实现虚拟现实效果的终端,通常可以提供为眼镜、头盔式显示器(Head Mount Display,HMD)、隐形眼镜的形态,以用于实现视觉感知和其他形式的感知,当然虚拟现实设备实现的形态不限于此,根据需要可以进一步小型化或大型化。Virtual reality equipment, a terminal that realizes virtual reality effects, can usually be provided in the form of glasses, helmet-mounted displays (HMD), and contact lenses to achieve visual perception and other forms of perception. Of course, virtual reality equipment realizes The form is not limited to this, and can be further miniaturized or enlarged as needed.

本申请实施例记载的虚拟现实设备可以包括但不限于如下几个类型:The virtual reality devices described in the embodiments of this application may include but are not limited to the following types:

电脑端虚拟现实(PCVR)设备,利用PC端进行虚拟现实功能的相关计算以及数据输出,外接的电脑端虚拟现实设备利用PC端输出的数据实现虚拟现实的效果。Computer-side virtual reality (PCVR) equipment uses the PC side to perform calculations and data output related to virtual reality functions. The external computer-side virtual reality equipment uses the data output from the PC side to achieve virtual reality effects.

移动虚拟现实设备,支持以各种方式(如设置有专门的卡槽的头戴式显示器)设置移动终端(如智能手机),通过与移动终端有线或无线方式的连接,由移动终端进行虚拟现实功能的相关计算,并输出数据至移动虚拟现实设备,例如通过移动终端的APP观看虚拟现实视频。Mobile virtual reality equipment supports setting up a mobile terminal (such as a smartphone) in various ways (such as a head-mounted display with a special card slot), and through a wired or wireless connection with the mobile terminal, the mobile terminal performs virtual reality Function-related calculations and output data to mobile virtual reality devices, such as viewing virtual reality videos through mobile terminal APPs.

一体机虚拟现实设备,具备用于进行虚拟功能的相关计算的处理器,因而具备独立的虚拟现实输入和输出的功能,不需要与PC端或移动终端连接,使用自由度高。The all-in-one virtual reality device has a processor for performing calculations related to virtual functions, so it has independent virtual reality input and output functions. It does not need to be connected to a PC or mobile terminal, and has a high degree of freedom in use.

以下分别进行详细说明。需说明的是,以下实施例的描述顺序不作为对实施例优先顺序的限定。Each is explained in detail below. It should be noted that the description order of the following embodiments is not used to limit the priority order of the embodiments.

本申请各实施例提供了一种音频处理方法,该方法可以由终端或服务器执行,也可以由终端和服务器共同执行;本申请实施例以音频处理方法由终端(虚拟现实设备)执行为例来进行说明。Each embodiment of the present application provides an audio processing method, which can be executed by a terminal or a server, or jointly by a terminal and a server; the embodiments of this application take the audio processing method being executed by a terminal (virtual reality device) as an example. Be explained.

请参阅图1至图2,图1为本申请实施例提供的音频处理方法的流程示意图,图2为本申请实施例提供的相关应用场景示意图,其中,图2中的空白背景可以为虚拟现实空间层。该方法包括:Please refer to Figures 1 to 2. Figure 1 is a schematic flow chart of an audio processing method provided by an embodiment of the present application. Figure 2 is a schematic diagram of a related application scenario provided by an embodiment of the present application. The blank background in Figure 2 can be a virtual reality space layer. The method includes:

步骤110,获取发声者的近场音频。Step 110: Obtain the near-field audio of the speaker.

例如,可以预先通过动捕棚中的近场麦克风收集发声者的近场音频。该发声者可以为明星偶像、表演者、演讲者、主持人、教师等人员。在收集到发声者的近场音频后,可以将该近场音频上传至计算机设备,该计算机设备可以为虚拟现实设备或者服务器。For example, the near-field audio of the speaker can be collected in advance through a near-field microphone in the motion capture studio. The speaker can be a celebrity idol, performer, speaker, host, teacher, etc. After collecting the near-field audio of the speaker, the near-field audio can be uploaded to a computer device, which can be a virtual reality device or a server.

步骤120,获取目标虚拟空间的音频配置参数,所述音频配置参数用于模拟所述目标虚拟空间的空间音效。Step 120: Obtain the audio configuration parameters of the target virtual space. The audio configuration parameters are used to simulate the spatial sound effects of the target virtual space.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

根据不同的虚拟场景,预先设定不同类型的虚拟空间,其中,每一类型的虚拟空间具有对应的音频配置参数;Different types of virtual spaces are preset according to different virtual scenes, where each type of virtual space has corresponding audio configuration parameters;

所述获取目标虚拟空间的音频配置参数,包括:The obtaining the audio configuration parameters of the target virtual space includes:

响应于针对虚拟空间的选择指令,从所述不同类型的虚拟空间中确定出所述目标虚拟空间,并获取所述目标虚拟空间的音频配置参数。In response to a selection instruction for a virtual space, the target virtual space is determined from the different types of virtual spaces, and audio configuration parameters of the target virtual space are obtained.

其中,虚拟环境中可以包括虚拟场景,以及在虚拟场景中活动的至少一个虚拟对象,该虚拟对象可以包括由用户(或玩家)控制的玩家角色,也可以由系统控制的非玩家角色(NPC,Non-Player Character)。例如,该虚拟对象还可以包括一个或多个角色属性,诸如技能属性、角色状态属性等,以向玩家提供帮助、提供虚拟服务、增加与玩家表现相关的分值等。例如,虚拟环境以游戏为例,游戏的虚拟场景中还可以包括一个或多个虚拟障碍物,诸如栏杆、沟壑、墙壁等,以限制虚拟对象的移动,例如将一个或多个虚拟对象的移动限制到虚拟现实场景内的特定区域。此外,虚拟场景中还可以呈现一个或多个指示器,以向玩家提供指示信息。例如,虚拟环境以视听体验类游戏为例,该视听体验类游戏的虚拟场景,可能是一个虚拟音乐厅,或者该虚拟场景可能是一个虚拟演播室,该虚拟场景可能是一个虚拟沙滩音乐会,或者该虚拟场景可能是一个虚拟运动场等,在虚拟场景对应的虚拟空间中还可以包括一个或多个虚拟物品,诸如舞台、演奏道具、观众席、桌椅等。The virtual environment may include a virtual scene and at least one virtual object active in the virtual scene. The virtual object may include a player character controlled by the user (or player) or a non-player character (NPC) controlled by the system. Non-Player Character). For example, the virtual object may also include one or more character attributes, such as skill attributes, character status attributes, etc., to provide help to the player, provide virtual services, increase points related to the player's performance, etc. For example, the virtual environment takes a game as an example. The virtual scene of the game may also include one or more virtual obstacles, such as railings, ravines, walls, etc., to limit the movement of virtual objects. Restricted to specific areas within the VR scene. In addition, one or more indicators may be presented in the virtual scene to provide instructions to the player. For example, the virtual environment is an audio-visual experience game. The virtual scene of the audio-visual experience game may be a virtual concert hall, or the virtual scene may be a virtual studio, and the virtual scene may be a virtual beach concert. Or the virtual scene may be a virtual sports field, etc. The virtual space corresponding to the virtual scene may also include one or more virtual items, such as a stage, performance props, auditoriums, tables and chairs, etc.

例如,对象(比如用户或现实玩家)可以通过在虚拟现实设备中使用当前账户登录并接入虚拟环境中的虚拟场景,当前账户对应的虚拟对象为用户通过虚拟现实设备控制的玩家角色。For example, an object (such as a user or a real player) can log in and access the virtual scene in the virtual environment by using the current account in the virtual reality device. The virtual object corresponding to the current account is the player character controlled by the user through the virtual reality device.

例如,可以预先在计算机设备中创建虚拟场景,该虚拟场景可能是一个虚拟音乐厅,或者该虚拟场景可能是一个虚拟演播室,该虚拟场景可能是一个虚拟沙滩音乐会,或者该虚拟场景可能是一个虚拟运动场等。可以根据不同的虚拟场景,预先设定不同类型的虚拟空间,并设置每一类型的虚拟空间对应的音频配置参数。For example, a virtual scene may be created in the computer device in advance, the virtual scene may be a virtual concert hall, or the virtual scene may be a virtual studio, the virtual scene may be a virtual beach concert, or the virtual scene may be a virtual sports field and more. Different types of virtual spaces can be preset according to different virtual scenes, and the audio configuration parameters corresponding to each type of virtual space can be set.

在一些实施例中,所述目标虚拟空间的音频配置参数包括所述目标虚拟空间的音频反射参数,音频传播的衰减参数,以及所述目标虚拟空间的背景音频。In some embodiments, the audio configuration parameters of the target virtual space include audio reflection parameters of the target virtual space, attenuation parameters of audio propagation, and background audio of the target virtual space.

例如,以虚拟音乐厅对应的虚拟空间为例,在构建虚拟空间时,虚拟音乐厅的四周墙壁的角度按照固定的角度进行设置。然后,设置虚拟空间的音频配置参数。例如,设置虚拟空间的音频反射参数,该音频反射参数可以包括虚拟空间的反射量、虚拟空间能反射的音频频段。例如,设置虚拟音乐厅墙壁的音频反射参数,比如:设置虚拟音乐厅墙壁的反射量是多少?设置虚拟音乐厅墙壁能反射哪些频段的这个音频。例如,还设置音频传播的衰减参数;比如,在构建虚拟空间时,设置虚拟音乐厅墙壁当前的空气密度,根据已设置的空气密度来设置音频传播的衰减参数,比如,当前设置的空气密度对应的音频传播的衰减参数是多少。例如,还需设置虚拟空间的背景音频。不同的虚拟场景对应的虚拟空间,具有不同的背景音频。For example, taking the virtual space corresponding to the virtual concert hall as an example, when constructing the virtual space, the angles of the surrounding walls of the virtual concert hall are set at fixed angles. Then, set the audio configuration parameters of the virtual space. For example, set the audio reflection parameters of the virtual space. The audio reflection parameters may include the reflection amount of the virtual space and the audio frequency band that the virtual space can reflect. For example, set the audio reflection parameters of the walls of the virtual concert hall, such as: What is the reflection amount of the walls of the virtual concert hall? Set which frequency bands of this audio can be reflected by the walls of the virtual concert hall. For example, the attenuation parameters of audio propagation are also set; for example, when building a virtual space, the current air density of the walls of the virtual concert hall is set, and the attenuation parameters of audio propagation are set according to the set air density. For example, the currently set air density corresponds to What is the attenuation parameter of audio propagation. For example, you also need to set the background audio of the virtual space. The virtual spaces corresponding to different virtual scenes have different background audio.

步骤130,根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,所述第一目标音频具有所述目标虚拟空间的空间音效。Step 130: Mix the near-field audio according to the audio configuration parameters of the target virtual space to obtain a first target audio, where the first target audio has a spatial sound effect of the target virtual space.

在本申请实施例中,可以在计算机设备中去模拟目标虚拟空间的空间音效。In the embodiment of the present application, the spatial sound effect of the target virtual space can be simulated in the computer device.

例如,发声者以明星偶像为例,发声者对应的虚拟发声对象也可以称为虚拟偶像,虚拟偶像在演出的过程中是在一个虚拟场景中的进行的。例如,若虚拟场景是一个沙滩,为一个开放式的虚拟场景,则开放式的虚拟场景对应的目标虚拟空间可以不涉及音频反射参数,可以将音频反射参数设置为空。收听者在进行视听体验时,需要感觉到这个虚拟偶像好像真的是在一个沙滩音乐场景中进行演唱,则需要模拟出沙滩音乐场景的演出音频。具体的,会在计算机设备中去创建一个沙滩音乐场景对应的三维虚拟空间,然后把收集到的近场音频放到沙滩音乐场景对应的目标虚拟空间中进行混音处理,在进行混音处理过程中,结合目标虚拟空间的音频配置参数进行相应的音频调整,最终得到具有目标虚拟空间的空间音效的第一目标音频。For example, the speaker is a celebrity idol, and the virtual vocal object corresponding to the speaker can also be called a virtual idol. The virtual idol performs in a virtual scene during the performance. For example, if the virtual scene is a beach and is an open virtual scene, the target virtual space corresponding to the open virtual scene may not involve audio reflection parameters, and the audio reflection parameters may be set to empty. When the listener has an audio-visual experience, they need to feel that this virtual idol is really singing in a beach music scene, and they need to simulate the performance audio of the beach music scene. Specifically, a three-dimensional virtual space corresponding to the beach music scene will be created in the computer device, and then the collected near-field audio will be placed in the target virtual space corresponding to the beach music scene for mixing processing. During the mixing process , make corresponding audio adjustments based on the audio configuration parameters of the target virtual space, and finally obtain the first target audio with spatial sound effects of the target virtual space.

例如,若虚拟偶像在一个虚拟音乐会或虚拟音乐厅中进行演出,虚拟场景对应的目标虚拟空间可以涉及音频反射参数,可以根据构建的目标虚拟空间来设置可以对不同位置产生混响效果的音频配置参数。收听者在进行视听体验时,需要感觉到这个虚拟偶像好像真的是在一个音乐厅场景中进行演唱,则需要模拟出音乐厅场景的演出音频。具体的,会在计算机设备中去创建一个音乐厅场景对应的三维虚拟空间,然后把收集到的近场音频放到音乐厅场景对应的目标虚拟空间中进行混音处理,在进行混音处理过程中,结合目标虚拟空间的音频配置参数进行相应的音频调整,最终得到具有目标虚拟空间的空间音效的第一目标音频。For example, if a virtual idol performs in a virtual concert or virtual concert hall, the target virtual space corresponding to the virtual scene can involve audio reflection parameters, and the audio that can produce a reverberation effect for different locations can be set according to the constructed target virtual space. Configuration parameters. When the listener has an audio-visual experience, they need to feel that the virtual idol is really singing in a concert hall scene, and they need to simulate the performance audio of the concert hall scene. Specifically, a three-dimensional virtual space corresponding to the concert hall scene will be created in the computer device, and then the collected near-field audio will be placed in the target virtual space corresponding to the concert hall scene for mixing processing. During the mixing process , make corresponding audio adjustments based on the audio configuration parameters of the target virtual space, and finally obtain the first target audio with spatial sound effects of the target virtual space.

在一些实施例中,所述根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,包括:In some embodiments, mixing the near-field audio according to the audio configuration parameters of the target virtual space to obtain the first target audio includes:

根据所述音频反射参数与所述衰减参数,对所述近场音频与所述背景音频进行第一音频调节处理;Perform a first audio adjustment process on the near-field audio and the background audio according to the audio reflection parameter and the attenuation parameter;

对进行第一音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第一目标音频。The near-field audio and the background audio that have been subjected to the first audio adjustment processing are mixed to obtain the first target audio.

例如,首先,根据音频反射参数与衰减参数,对近场音频与背景音频进行第一音频调节处理,比如第一音频调节处理可以为:调节近场音频与背景音频的音量大小,根据确定的近场音频中能被反射的音频频段(过滤掉近场音频中不能被反射的音频频段)调节近场音频,根据确定的近场音频的衰减量调节近场音频等;然后,对进行第一音频调节处理后的近场音频与背景音频进行进行混音处理,得到第一目标音频。For example, first, perform a first audio adjustment process on the near-field audio and background audio according to the audio reflection parameter and attenuation parameter. For example, the first audio adjustment process may be: adjusting the volume of the near-field audio and the background audio, and adjusting the volume of the near-field audio and the background audio according to the determined near-field audio and background audio. The audio frequency band that can be reflected in the field audio (filter out the audio frequency band that cannot be reflected in the near-field audio) adjusts the near-field audio, adjusts the near-field audio according to the determined attenuation of the near-field audio, etc.; then, perform the first audio The adjusted near-field audio and background audio are mixed to obtain the first target audio.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

获取所述发声者对应的虚拟发声对象在所述目标虚拟空间中的第一位置,以及收听者对应的虚拟收听对象在所述目标虚拟空间中的第二位置;Obtain the first position of the virtual speaking object corresponding to the speaker in the target virtual space, and the second position of the virtual listening object corresponding to the listener in the target virtual space;

所述根据所述音频反射参数与所述衰减参数,对所述近场音频与所述背景音频进行第一音频调节处理,包括:Performing a first audio adjustment process on the near-field audio and the background audio according to the audio reflection parameter and the attenuation parameter includes:

根据所述音频反射参数,所述衰减参数,所述第一位置与所述第二位置,对所述近场音频与所述背景音频进行第一音频调节处理。According to the audio reflection parameter, the attenuation parameter, the first position and the second position, a first audio adjustment process is performed on the near-field audio and the background audio.

例如,为了更进一步提升空间音效的音效效果,在进行音频调节处理的过程中,除了目标虚拟空间的音频配置参数,还需结合发声者对应的虚拟发声对象在目标虚拟空间中的第一位置,以及收听者对应的虚拟收听对象在目标虚拟空间中的第二位置,进行第二音频调节处理。For example, in order to further improve the sound effects of spatial sound effects, during the audio adjustment process, in addition to the audio configuration parameters of the target virtual space, it is also necessary to combine the first position of the virtual vocal object corresponding to the speaker in the target virtual space. and perform the second audio adjustment process at the second position of the virtual listening object corresponding to the listener in the target virtual space.

例如,计算第一位置与第二位置之间的距离,根据距离确定近场音频与背景音频的音量大小,根据距离与音频反射参数之间的对应关系确定近场音频中的能被反射的音频频段,以及根据距离与衰减参数之间的对应关系确定近场音频在空气中的衰减量,在进行第一音频调节处理时,调节近场音频与背景音频的音量大小,并过滤掉近场音频中不能被反射的音频频段,以及根据近场音频在空气中的衰减量调节近场音频。然后,对进行第一音频调节处理后的近场音频与背景音频进行进行混音处理,得到第一目标音频。For example, calculate the distance between the first position and the second position, determine the volume of the near-field audio and background audio based on the distance, and determine the audio that can be reflected in the near-field audio based on the correspondence between the distance and the audio reflection parameter. frequency band, and determine the attenuation amount of near-field audio in the air based on the correspondence between distance and attenuation parameters. When performing the first audio adjustment process, adjust the volume of the near-field audio and background audio, and filter out the near-field audio. The audio frequency band that cannot be reflected in the audio frequency band, and adjust the near-field audio according to the attenuation of the near-field audio in the air. Then, the near-field audio and the background audio that have been subjected to the first audio adjustment processing are mixed to obtain the first target audio.

比如,在目标虚拟空间中,当虚拟发声对象所在的第一位置与虚拟收听对象所在的第二位置之间的距离为1米时,收听者能听清楚发声者所发出的所有频段的所有声音。在目标虚拟空间中,当虚拟发声对象所在的第一位置与虚拟收听对象所在的第二位置之间的距离为100米时,发声者的高频的声音和低频的声音就会被当前目标虚拟空间中的空气所过滤掉,那么收听者只能听到发声者比较偏中频的声音。在本申请中可以在计算机中去模拟这样一个有环境且有空间的混响音频。For example, in the target virtual space, when the distance between the first position of the virtual speaking object and the second position of the virtual listening object is 1 meter, the listener can clearly hear all sounds in all frequency bands emitted by the speaker. . In the target virtual space, when the distance between the first position of the virtual speaking object and the second position of the virtual listening object is 100 meters, the high-frequency sound and low-frequency sound of the speaker will be virtualized by the current target. If the air in the space is filtered out, the listener can only hear the mid-frequency sound of the speaker. In this application, such an ambient and spatial reverberation audio can be simulated in a computer.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理;When it is detected that the relative positional relationship between the first position and the second position changes, the near-field audio and the relative positional relationship between the first position and the second position are compared. The background audio is subjected to a second audio adjustment process;

对进行第二音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第二目标音频,所述第二目标音频具有所述目标虚拟空间的空间音效、以及表征所述第一位置与所述第二位置之间的相对位置关系发生变化时产生的音效变化效果。The near-field audio and the background audio after the second audio adjustment processing are mixed to obtain a second target audio. The second target audio has the spatial sound effect of the target virtual space and represents the target virtual space. The sound effect changes when the relative positional relationship between the first position and the second position changes.

例如,发声者以明星偶像为例,发声者对应的虚拟发声对象也可以称为虚拟偶像。如图2所示,音乐厅场景下的目标虚拟空间10,设有四个互动场,具体设有一个台上互动场101,以及三个台下互动场102(包括第一台下互动场102A、第二台下互动场102B、台下互动场102C)。例如,台上互动场101可以为舞台,台下互动场102可以为观众区,发声者对应的虚拟发声对象11所在的第一位置位于台上互动场101中,收听者对应的虚拟收听对象12所在的第二位置位于第一台下互动场102A中。For example, the speaker is a celebrity idol, and the virtual vocal object corresponding to the speaker can also be called a virtual idol. As shown in Figure 2, the target virtual space 10 in the concert hall scene has four interactive fields, specifically one on-stage interactive field 101 and three off-stage interactive fields 102 (including the first off-stage interactive field 102A , the second off-stage interactive field 102B, the second off-stage interactive field 102C). For example, the on-stage interactive field 101 can be a stage, and the off-stage interactive field 102 can be an audience area. The first position of the virtual speaking object 11 corresponding to the speaker is located in the on-stage interactive field 101, and the virtual listening object 12 corresponding to the listener is located in the on-stage interactive field 101. The second location is located in the first off-stage interactive field 102A.

例如,若虚拟发声对象11(虚拟偶像)在演出过程中从互动场101的左边的位置A走到了互动场101的右边的位置B,此时,实际上从收听者的听感来说,希望这个演唱的音乐的发音点从左边移动到右边。因此,当检测到虚拟发声对象所在的第一位置与虚拟收听对象所在的第二位置之间的相对位置关系发生变化时,根据第一位置与第二位置之间的相对位置关系对近场音频与背景音频进行第二音频调节处理,比如,调节近场音频与背景音频的音量大小,根据确定的近场音频中能被反射的音频频段以及确定的近场音频的衰减量调节近场音频等;然后,对进行第二音频调节处理后的近场音频与背景音频进行进行混音处理,得到第二目标音频,第二目标音频具有目标虚拟空间的空间音效、以及表征第一位置与第二位置之间的相对位置关系发生变化时产生的音效变化效果。音频调节处理以及混音处理是可以实时进行的,因此得到第二目标音频,具有目标虚拟空间的空间音效,且能表征第一位置与第二位置之间的相对位置关系发生变化时产生的音效变化效果。For example, if the virtual vocal object 11 (virtual idol) walks from the position A on the left side of the interactive field 101 to the position B on the right side of the interactive field 101 during the performance, at this time, actually from the perspective of the listener's sense of hearing, it is expected that The point of articulation of this sung music moves from left to right. Therefore, when it is detected that the relative positional relationship between the first position where the virtual sound-emitting object is located and the second position where the virtual listening object is located changes, the near-field audio is processed according to the relative positional relationship between the first position and the second position. Perform a second audio adjustment process with the background audio, for example, adjust the volume of the near-field audio and the background audio, adjust the near-field audio according to the determined audio frequency band that can be reflected in the near-field audio and the determined attenuation of the near-field audio, etc. ; Then, mix the near-field audio and background audio after the second audio adjustment processing to obtain the second target audio. The second target audio has the spatial sound effect of the target virtual space and represents the first position and the second The sound effect changes when the relative positional relationship between positions changes. The audio adjustment processing and mixing processing can be performed in real time, so the second target audio is obtained, which has the spatial sound effect of the target virtual space, and can represent the sound effect produced when the relative positional relationship between the first position and the second position changes. Change effect.

在一些实施例中,所述相对位置关系包括距离关系,所述当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理,包括:In some embodiments, the relative position relationship includes a distance relationship, and when a change in the relative position relationship between the first position and the second position is detected, based on the first position and the The relative positional relationship between the second positions is used to perform a second audio adjustment process on the near-field audio and the background audio, including:

当检测到所述第一位置与所述第二位置之间的距离关系发生变化时,根据所述第一位置与所述第二位置之间的距离对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the distance relationship between the first position and the second position changes, the near-field audio and the background audio are compared according to the distance between the first position and the second position. Perform second audio adjustment processing.

例如,当相对位置关系主要为距离关系时,当检测到第一位置与第二位置之间的距离关系发生变化时,根据第一位置与第二位置之间的距离对近场音频与背景音频进行第二音频调节处理,比如,在距离关系发生变化之前已根据目标虚拟空间的音频配置参数对近场音频和背景音频进行第一音频调节处理;在第一音频调节处理的基础上,进行第二音频调节处理时,主要调节近场音频和背景音频的音量大小;例如,背景音频的音量大小可以不变,若第一位置与第二位置之间的距离变小时,将近场音频的音量调大;若第一位置与第二位置之间的距离变大时,将近场音频的音量调小。若第一位置与第二位置之间的距离变化较大时,也可以进一步根据距离与音频反射参数之间的对应关系确定近场音频中的能被反射的音频频段,以及根据距离与衰减参数之间的对应关系确定近场音频在空气中的衰减量,在进行第二音频调节处理时,调节近场音频与背景音频的音量大小,并过滤掉近场音频中不能被反射的音频频段,以及根据近场音频在空气中的衰减量调节近场音频。For example, when the relative position relationship is mainly a distance relationship, when a change in the distance relationship between the first position and the second position is detected, the near-field audio and the background audio are compared according to the distance between the first position and the second position. Perform a second audio adjustment process. For example, before the distance relationship changes, the first audio adjustment process has been performed on the near-field audio and background audio according to the audio configuration parameters of the target virtual space; based on the first audio adjustment process, perform a second audio adjustment process. During the second audio adjustment process, the volume of the near-field audio and background audio is mainly adjusted; for example, the volume of the background audio can remain unchanged. If the distance between the first position and the second position becomes smaller, the volume of the near-field audio is adjusted. Large; if the distance between the first position and the second position becomes larger, reduce the volume of the near-field audio. If the distance between the first position and the second position changes greatly, the audio frequency band that can be reflected in the near-field audio can also be determined based on the correspondence between the distance and the audio reflection parameters, and based on the distance and attenuation parameters. The corresponding relationship determines the attenuation of near-field audio in the air. When performing the second audio adjustment process, adjust the volume of the near-field audio and background audio, and filter out the audio frequency bands in the near-field audio that cannot be reflected. And adjust the near-field audio according to the attenuation of the near-field audio in the air.

在一些实施例中,所述相对位置关系包括方位关系,所述当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理,包括:In some embodiments, the relative position relationship includes an orientation relationship, and when a change in the relative position relationship between the first position and the second position is detected, the relative position relationship between the first position and the second position is determined based on the first position and the second position. The relative positional relationship between the second positions is used to perform a second audio adjustment process on the near-field audio and the background audio, including:

当检测到所述第一位置与所述第二位置之间的方位关系发生变化时,根据所述第一位置与所述第二位置之间的相对方位对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the orientation relationship between the first position and the second position changes, the near-field audio and the background are compared according to the relative orientation between the first position and the second position. The audio undergoes a second audio adjustment process.

例如,当相对位置关系主要为方位关系时,当检测到第一位置与第二位置之间的方位关系发生变化时,根据第一位置与第二位置之间的相对方位对近场音频与背景音频进行第二音频调节处理,比如,在方位关系发生变化之前已根据目标虚拟空间的音频配置参数对近场音频和背景音频进行第一音频调节处理;在第一音频调节处理的基础上,进行第二音频调节处理时,主要调节近场音频和背景音频的声道音量大小;例如,背景音频的声道音量大小可以不变,若相对方位为虚拟发声对象11位于虚拟收听对象12的左侧时,将近场音频的左声道音量调大,将近场音频的右声道音量调小;若相对方位为虚拟发声对象11位于虚拟收听对象12的前方时,将近场音频的左声道音量与有声道音量可以调整为一样大;若相对方位为虚拟发声对象11位于虚拟收听对象12的右侧时,将近场音频的左声道音量调小,将近场音频的右声道音量调大。For example, when the relative position relationship is mainly an orientation relationship, when a change in the orientation relationship between the first position and the second position is detected, the near-field audio and background are compared based on the relative orientation between the first position and the second position. The audio undergoes a second audio adjustment process. For example, before the orientation relationship changes, the first audio adjustment process is performed on the near-field audio and background audio according to the audio configuration parameters of the target virtual space; on the basis of the first audio adjustment process, During the second audio adjustment process, the channel volume of the near-field audio and background audio is mainly adjusted; for example, the channel volume of the background audio can remain unchanged, if the relative orientation is that the virtual speaking object 11 is located on the left side of the virtual listening object 12 When , the volume of the left channel of the near-field audio is turned up, and the volume of the right channel of the near-field audio is turned down; if the relative orientation is that the virtual speaking object 11 is in front of the virtual listening object 12, the volume of the left channel of the near-field audio is equal to the volume of the left channel of the near-field audio. The volume of the audio channels can be adjusted to the same volume; if the relative orientation is that the virtual speaking object 11 is located on the right side of the virtual listening object 12, the volume of the left channel of the near-field audio is turned down, and the volume of the right channel of the near-field audio is turned up.

在一些实施例中,所述相对位置关系包括距离关系与方位关系,所述当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理,包括:In some embodiments, the relative position relationship includes a distance relationship and an orientation relationship, and when a change in the relative position relationship between the first position and the second position is detected, based on the first position Performing a second audio adjustment process on the near-field audio and the background audio based on the relative positional relationship with the second position, including:

当检测到所述第一位置与所述第二位置之间的距离关系与方位关系发生变化时,根据所述第一位置与所述第二位置之间的距离与相对方位,对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the distance relationship and the azimuth relationship between the first position and the second position change, based on the distance and relative azimuth between the first position and the second position, the near The field audio and the background audio undergo a second audio adjustment process.

例如,当相对位置关系包括距离关系与方位关系,当检测到第一位置与第二位置之间的距离关系与方位关系发生变化时,根据第一位置与第二位置之间的距离与相对方位,对近场音频与背景音频进行第二音频调节处理。For example, when the relative position relationship includes a distance relationship and an orientation relationship, and when it is detected that the distance relationship and the orientation relationship between the first position and the second position change, based on the distance and relative orientation between the first position and the second position, , perform second audio adjustment processing on near-field audio and background audio.

比如,在距离关系发和方位关系生变化之前已根据目标虚拟空间的音频配置参数对近场音频和背景音频进行第一音频调节处理;在第一音频调节处理的基础上,进行第二音频调节处理时,先根据距离调节近场音频和背景音频的音量大小;例如,背景音频的音量大小可以不变,若第一位置与第二位置之间的距离变小时,将近场音频的音量调大;若第一位置与第二位置之间的距离变大时,将近场音频的音量调小。若第一位置与第二位置之间的距离变化较大时,也可以进一步根据距离与音频反射参数之间的对应关系确定近场音频中的能被反射的音频频段,以及根据距离与衰减参数之间的对应关系确定近场音频在空气中的衰减量,在进行第二音频调节处理时,调节近场音频与背景音频的音量大小,并过滤掉近场音频中不能被反射的音频频段,以及根据近场音频在空气中的衰减量调节近场音频。然后,在根据距离调节完成后,再根据相对方位调节近场音频和背景音频的声道音量大小;例如,背景音频的声道音量大小可以不变,若相对方位为虚拟发声对象11位于虚拟收听对象12的左侧时,将近场音频的左声道音量调大,将近场音频的右声道音量调小;若相对方位为虚拟发声对象11位于虚拟收听对象12的前方时,将近场音频的左声道音量与有声道音量可以调整为一样大;若相对方位为虚拟发声对象11位于虚拟收听对象12的右侧时,将近场音频的左声道音量调小,将近场音频的右声道音量调大。For example, before the distance relationship and the orientation relationship change, a first audio adjustment process has been performed on the near-field audio and background audio according to the audio configuration parameters of the target virtual space; on the basis of the first audio adjustment process, a second audio adjustment process is performed During processing, first adjust the volume of the near-field audio and background audio according to the distance; for example, the volume of the background audio can remain unchanged. If the distance between the first position and the second position becomes smaller, the volume of the near-field audio can be increased. ; If the distance between the first position and the second position becomes larger, reduce the volume of the near-field audio. If the distance between the first position and the second position changes greatly, the audio frequency band that can be reflected in the near-field audio can also be determined based on the correspondence between the distance and the audio reflection parameters, and based on the distance and attenuation parameters. The corresponding relationship determines the attenuation of near-field audio in the air. When performing the second audio adjustment process, adjust the volume of the near-field audio and background audio, and filter out the audio frequency bands in the near-field audio that cannot be reflected. And adjust the near-field audio according to the attenuation of the near-field audio in the air. Then, after the adjustment according to the distance is completed, the channel volume of the near-field audio and the background audio is adjusted according to the relative orientation; for example, the channel volume of the background audio can remain unchanged, if the relative orientation is that the virtual sounding object 11 is located in the virtual listening position When the object 12 is on the left side, the volume of the left channel of the near-field audio is turned up, and the volume of the right channel of the near-field audio is turned down; if the relative orientation is that the virtual speaking object 11 is in front of the virtual listening object 12, the volume of the near-field audio is turned down. The volume of the left channel and the volume of the sound channel can be adjusted to be the same; if the relative orientation is that the virtual sounding object 11 is located on the right side of the virtual listening object 12, the volume of the left channel of the near-field audio is reduced, and the volume of the right channel of the near-field audio is reduced. Turn up the channel volume.

在一些实施例中,所述方法还包括:In some embodiments, the method further includes:

当检测到所述收听者的头部摆动时,获取头部摆动过程中所述收听者的左耳朝向方位和右耳朝向方位;When the head swing of the listener is detected, the left ear orientation and the right ear orientation of the listener during the head swing are obtained;

根据所述收听者的左耳朝向方位和右耳朝向方位对所述近场音频与所述背景音频进行第三音频调节处理;Perform a third audio adjustment process on the near-field audio and the background audio according to the listener's left ear orientation and right ear orientation;

对进行第三音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第三目标音频,所述第三目标音频具有所述目标虚拟空间的空间音效、以及表征所述收听者的头部摆动时产生的音效变化效果。The near-field audio and the background audio after the third audio adjustment processing are mixed to obtain a third target audio. The third target audio has the spatial sound effect of the target virtual space and represents the target virtual space. Describes the sound effect changes produced when the listener's head moves.

例如,为了更进一步提升用户的沉浸式体验,可以结合收听者的头部摆动过程中收听者的左耳朝向方位和右耳朝向方位来模拟相应的音效。For example, in order to further enhance the user's immersive experience, the corresponding sound effects can be simulated by combining the direction of the listener's left ear and the direction of the right ear during the movement of the listener's head.

收听者在摆头时,因为收听者的两只耳朵的位置在变化,若收听者看向右侧,则收听者的左耳朝向前方,那么收听者的左耳就应该听到发声者的声音,收听者的右耳听到的发声者的声音会比较小,或者收听者的右耳听不到发声者的声音。When the listener moves his head, because the positions of the listener's two ears are changing, if the listener looks to the right, then the listener's left ear faces forward, then the listener's left ear should hear the speaker's voice. , the speaker's voice heard by the listener's right ear will be smaller, or the speaker's voice cannot be heard by the listener's right ear.

其中,可以基于对收听者佩戴的头戴式的虚拟现实设备进行6DoF追踪以获取收听者的6DoF数据,并根据6DoF数据确定收听者的左耳朝向方位和右耳朝向方位。Among them, the listener's 6DoF data can be obtained based on 6DoF tracking of the head-mounted virtual reality device worn by the listener, and the listener's left ear orientation and right ear orientation are determined based on the 6DoF data.

6DoF追踪:六自由度追踪,Six degrees of freedom tracking。物体可以在三维空间中移动的六个自由度(6DOF)。六个角度是(1)向前/向后,(2)向上/向下,(3)左/右,(4)偏航,(5)俯仰和(6)滚动。使用允许6DOF的VR系统,可以在有限的空间内自由移动,这使用户可以充分利用所有6个自由度:偏航,俯仰,侧倾,前/后,上/下和左/右。这使得视野更加逼真和真实。6DoF tracking: Six degrees of freedom tracking. The six degrees of freedom (6DOF) with which an object can move in three-dimensional space. The six angles are (1) forward/backward, (2) up/down, (3) left/right, (4) yaw, (5) pitch and (6) roll. Free movement within a limited space is possible using a VR system that allows for 6DOF, which allows the user to take full advantage of all 6 degrees of freedom: yaw, pitch, roll, forward/backward, up/down and left/right. This makes the view more realistic and realistic.

例如,在第一音频调节处理或者第二音频调节处理的基础上,进行第三音频调节处理时,主要调节近场音频和背景音频的声道音量大小;例如,背景音频的声道音量大小可以不变,若左耳朝向方位为前方,右耳朝向方位为后方,则将近场音频的左声道音量调大,将近场音频的右声道音量调小;若左耳朝向方位为左侧,右耳朝向方位为右侧,近场音频的左声道音量与有声道音量可以调整为一样大;若左耳朝向方位为后方,右耳朝向方位为前方,将近场音频的左声道音量调小,将近场音频的右声道音量调大。For example, based on the first audio adjustment process or the second audio adjustment process, when performing the third audio adjustment process, the channel volume of the near-field audio and the background audio is mainly adjusted; for example, the channel volume of the background audio can be unchanged, if the left ear is facing forward and the right ear is facing backward, then the volume of the left channel of the near-field audio is turned up and the volume of the right channel of the near-field audio is turned down; if the left ear is turned to the left, If the right ear is facing to the right, the volume of the left channel of the near-field audio can be adjusted to be the same as the volume of the active channel; if the left ear is facing to the rear, and the right ear is facing to the front, the volume of the left channel of the near-field audio can be adjusted to the same level. Turn down and turn up the right channel volume of near-field audio.

例如,若目标虚拟空间是在收听者使用的虚拟现实设备创建的,那通过虚拟现实设备就可以直接获取收听者位置变化,即获取收听者的左耳朝向方位和右耳朝向方位。并根据收听者的左耳朝向方位和右耳朝向方位对近场音频与背景音频进行第三音频调节处理;对进行第三音频调节处理后的近场音频与背景音频进行进行混音处理,得到第三目标音频,第三目标音频具有目标虚拟空间的空间音效、以及表征收听者的头部摆动时产生的音效变化效果。For example, if the target virtual space is created on a virtual reality device used by the listener, the change in the listener's position can be directly obtained through the virtual reality device, that is, the direction of the listener's left ear and right ear can be obtained. And perform a third audio adjustment process on the near-field audio and background audio according to the listener's left ear orientation and right ear orientation; perform a mixing process on the near-field audio and background audio after the third audio adjustment process, and obtain The third target audio has a spatial sound effect of the target virtual space and a sound effect change effect that represents the listener's head swing.

例如,若目标虚拟空间是在云端创建的,比如该目标虚拟空间可能是在一个视频云端的后台创建的,接收从虚拟现实设备发送的收听者的左耳朝向方位和右耳朝向方位,并根据收听者的左耳朝向方位和右耳朝向方位对近场音频与背景音频进行第三音频调节处理;对进行第三音频调节处理后的近场音频与背景音频进行进行混音处理,得到第三目标音频,第三目标音频具有目标虚拟空间的空间音效、以及表征收听者的头部摆动时产生的音效变化效果。云端输出的第三目标音频可以为一个四路的杜比音效的音频。用户平时收听到的音频一般是左右立体声双声道,该双声道实际上只能表示两个耳朵,但杜比音效就能够把两个音源变成四个音源,这四个音源传输到收听者的头戴显示设备中时,可以以一个独特的算法把四个音源变成一个立体的音源,让收听者整体的感觉是,第三目标音频会有一个前后顺序或者是一个左右顺序的音效。For example, if the target virtual space is created in the cloud, such as the target virtual space may be created in the background of a video cloud, the listener's left ear orientation and right ear orientation are received from the virtual reality device, and based on The listener's left ear orientation and right ear orientation perform third audio adjustment processing on the near field audio and background audio; the near field audio and background audio after the third audio adjustment processing are mixed and processed to obtain the third The target audio and the third target audio have the spatial sound effect of the target virtual space and the sound effect change effect produced when the listener's head swings. The third target audio output from the cloud can be a four-way Dolby sound effect audio. The audio that users usually listen to is generally left and right stereo dual channels. This dual channel can actually only represent two ears, but Dolby sound effects can turn two sound sources into four sound sources. These four sound sources are transmitted to the listening device. When placed in the listener's head-mounted display device, a unique algorithm can be used to turn four sound sources into a three-dimensional sound source, giving the listener the overall feeling that the third target audio will have a front-to-back sequence or a left-to-right sequence of sound effects. .

上述所有的技术方案,可以采用任意结合形成本申请的可选实施例,在此不再一一赘述。All the above technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described again one by one.

本申请实施例通过获取发声者的近场音频;获取目标虚拟空间的音频配置参数,音频配置参数用于模拟目标虚拟空间的空间音效;根据目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到第一目标音频,第一目标音频具有目标虚拟空间的空间音效。本申请实施例通过预先构建的目标虚拟空间的音频配置参数,对近场音频进行混音处理,得到具有目标虚拟空间的空间音效的音频,提升了音频效果,提升了用户体验。The embodiment of the present application obtains the near-field audio of the speaker; obtains the audio configuration parameters of the target virtual space, and the audio configuration parameters are used to simulate the spatial sound effects of the target virtual space; and mixes the near-field audio according to the audio configuration parameters of the target virtual space. Sound processing is performed to obtain the first target audio, which has the spatial sound effect of the target virtual space. The embodiment of the present application mixes the near-field audio through the pre-constructed audio configuration parameters of the target virtual space to obtain audio with spatial sound effects of the target virtual space, thereby improving the audio effect and user experience.

为便于更好的实施本申请实施例的音频处理方法,本申请实施例还提供一种音频处理装置。请参阅图3,图3为本申请实施例提供的音频处理装置的结构示意图。其中,该音频处理装置200可以包括:In order to facilitate better implementation of the audio processing method in the embodiment of the present application, the embodiment of the present application also provides an audio processing device. Please refer to FIG. 3 , which is a schematic structural diagram of an audio processing device provided by an embodiment of the present application. Wherein, the audio processing device 200 may include:

第一获取单元210,用于获取发声者的近场音频;The first acquisition unit 210 is used to acquire the near-field audio of the speaker;

第二获取单元220,用于获取目标虚拟空间的音频配置参数,所述音频配置参数用于模拟所述目标虚拟空间的空间音效;The second acquisition unit 220 is used to acquire the audio configuration parameters of the target virtual space, where the audio configuration parameters are used to simulate the spatial sound effects of the target virtual space;

处理单元230,用于根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,所述第一目标音频具有所述目标虚拟空间的空间音效。The processing unit 230 is configured to perform mixing processing on the near-field audio according to the audio configuration parameters of the target virtual space to obtain a first target audio, where the first target audio has a spatial sound effect of the target virtual space.

在一些实施例中,所述目标虚拟空间的音频配置参数包括所述目标虚拟空间的音频反射参数,音频传播的衰减参数,以及所述目标虚拟空间的背景音频。In some embodiments, the audio configuration parameters of the target virtual space include audio reflection parameters of the target virtual space, attenuation parameters of audio propagation, and background audio of the target virtual space.

在一些实施例中,所述处理单元230,用于:In some embodiments, the processing unit 230 is used to:

根据所述音频反射参数与所述衰减参数,对所述近场音频与所述背景音频进行第一音频调节处理;Perform a first audio adjustment process on the near-field audio and the background audio according to the audio reflection parameter and the attenuation parameter;

对进行第一音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第一目标音频。The near-field audio and the background audio that have been subjected to the first audio adjustment processing are mixed to obtain the first target audio.

在一些实施例中,所述处理单元230,还用于:In some embodiments, the processing unit 230 is also used to:

获取所述发声者对应的虚拟发声对象在所述目标虚拟空间中的第一位置,以及收听者对应的虚拟收听对象在所述目标虚拟空间中的第二位置;Obtain the first position of the virtual speaking object corresponding to the speaker in the target virtual space, and the second position of the virtual listening object corresponding to the listener in the target virtual space;

根据所述音频反射参数,所述衰减参数,所述第一位置与所述第二位置,对所述近场音频与所述背景音频进行第一音频调节处理。According to the audio reflection parameter, the attenuation parameter, the first position and the second position, a first audio adjustment process is performed on the near-field audio and the background audio.

在一些实施例中,所述处理单元230,还用于:In some embodiments, the processing unit 230 is also used to:

当检测到所述第一位置与所述第二位置之间的相对位置关系发生变化时,根据所述第一位置与所述第二位置之间的相对位置关系对所述近场音频与所述背景音频进行第二音频调节处理;When it is detected that the relative positional relationship between the first position and the second position changes, the near-field audio and the relative positional relationship between the first position and the second position are compared. The background audio is subjected to a second audio adjustment process;

对进行第二音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第二目标音频,所述第二目标音频具有所述目标虚拟空间的空间音效、以及表征所述第一位置与所述第二位置之间的相对位置关系发生变化时产生的音效变化效果。The near-field audio and the background audio after the second audio adjustment processing are mixed to obtain a second target audio. The second target audio has the spatial sound effect of the target virtual space and represents the target virtual space. The sound effect changes when the relative positional relationship between the first position and the second position changes.

在一些实施例中,所述相对位置关系包括距离关系,所述处理单元230,还用于:In some embodiments, the relative position relationship includes a distance relationship, and the processing unit 230 is also used to:

当检测到所述第一位置与所述第二位置之间的距离关系发生变化时,根据所述第一位置与所述第二位置之间的距离对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the distance relationship between the first position and the second position changes, the near-field audio and the background audio are compared according to the distance between the first position and the second position. Perform second audio adjustment processing.

在一些实施例中,所述相对位置关系包括方位关系,所述处理单元230,还用于:In some embodiments, the relative position relationship includes an orientation relationship, and the processing unit 230 is also used to:

当检测到所述第一位置与所述第二位置之间的方位关系发生变化时,根据所述第一位置与所述第二位置之间的相对方位对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the orientation relationship between the first position and the second position changes, the near-field audio and the background are compared according to the relative orientation between the first position and the second position. The audio undergoes a second audio adjustment process.

在一些实施例中,所述相对位置关系包括距离关系与方位关系,所述处理单元230,还用于:In some embodiments, the relative position relationship includes distance relationship and orientation relationship, and the processing unit 230 is also used to:

当检测到所述第一位置与所述第二位置之间的距离关系与方位关系发生变化时,根据所述第一位置与所述第二位置之间的距离与相对方位,对所述近场音频与所述背景音频进行第二音频调节处理。When it is detected that the distance relationship and the azimuth relationship between the first position and the second position change, based on the distance and relative azimuth between the first position and the second position, the near The field audio and the background audio undergo a second audio adjustment process.

在一些实施例中,所述处理单元230,还用于:In some embodiments, the processing unit 230 is also used to:

当检测到所述收听者的头部摆动时,获取头部摆动过程中所述收听者的左耳朝向方位和右耳朝向方位;When the head swing of the listener is detected, the left ear orientation and the right ear orientation of the listener during the head swing are obtained;

根据所述收听者的左耳朝向方位和右耳朝向方位对所述近场音频与所述背景音频进行第三音频调节处理;Perform a third audio adjustment process on the near-field audio and the background audio according to the listener's left ear orientation and right ear orientation;

对进行第三音频调节处理后的所述近场音频与所述背景音频进行进行混音处理,得到第三目标音频,所述第三目标音频具有所述目标虚拟空间的空间音效、以及表征所述收听者的头部摆动时产生的音效变化效果。The near-field audio and the background audio after the third audio adjustment processing are mixed to obtain a third target audio. The third target audio has the spatial sound effect of the target virtual space and represents the target virtual space. Describes the sound effect changes produced when the listener's head moves.

在一些实施例中,所述音频处理装置200还包括:In some embodiments, the audio processing device 200 further includes:

设定单元,用于根据不同的虚拟场景,预先设定不同类型的虚拟空间,其中,每一类型的虚拟空间具有对应的音频配置参数;The setting unit is used to pre-set different types of virtual spaces according to different virtual scenes, where each type of virtual space has corresponding audio configuration parameters;

所述第二获取单元220,用于响应于针对虚拟空间的选择指令,从所述不同类型的虚拟空间中确定出所述目标虚拟空间,并获取所述目标虚拟空间的音频配置参数。The second acquisition unit 220 is configured to respond to a selection instruction for a virtual space, determine the target virtual space from the different types of virtual spaces, and acquire the audio configuration parameters of the target virtual space.

上述音频处理装置200中的各个单元可全部或部分通过软件、硬件及其组合来实现。上述各个单元可以以硬件形式内嵌于或独立于虚拟现实设备中的处理器中,也可以以软件形式存储于虚拟现实设备中的存储器中,以便于处理器调用执行上述各个单元对应的操作。Each unit in the above-mentioned audio processing device 200 can be implemented in whole or in part by software, hardware, and combinations thereof. Each of the above-mentioned units may be embedded in or independent of the processor in the virtual reality device in the form of hardware, or may be stored in the memory of the virtual reality device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned units.

音频处理装置200,可以集成在具备储存器并安装有处理器而具有运算能力的终端或服务器中,或者该音频处理装置200为该终端或服务器。The audio processing device 200 may be integrated into a terminal or server that has a storage and a processor installed and has computing capabilities, or the audio processing device 200 is the terminal or server.

在一些实施例中,本申请还提供了一种虚拟现实设备,包括存储器和处理器,存储器中存储有计算机程序,该处理器执行计算机程序时实现上述各方法实施例中的步骤。In some embodiments, the present application also provides a virtual reality device, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the steps in the above method embodiments.

如图4所示,图4为本申请实施例提供的虚拟现实设备的结构示意图,该虚拟现实设备300通常可以提供为眼镜、头盔式显示器(Head Mount Display,HMD)、隐形眼镜的形态,以用于实现视觉感知和其他形式的感知,当然虚拟现实设备实现的形态不限于此,根据需要可以进一步小型化或大型化。该虚拟现实设备300可以包括但不限于以下的构成:As shown in Figure 4, Figure 4 is a schematic structural diagram of a virtual reality device provided by an embodiment of the present application. The virtual reality device 300 can usually be provided in the form of glasses, a helmet-mounted display (HMD), or a contact lens. It is used to realize visual perception and other forms of perception. Of course, the form of virtual reality equipment is not limited to this, and can be further miniaturized or enlarged as needed. The virtual reality device 300 may include but is not limited to the following components:

检测模块301:使用各种传感器检测用户的操作命令,并作用于虚拟环境,如跟随用户的视线而不断更新在显示屏上显示的影像,实现用户与虚拟和场景的交互,例如基于检测到的用户头部的转动方向来不断更新现实内容。Detection module 301: Use various sensors to detect the user's operation commands and act on the virtual environment, such as following the user's line of sight to continuously update the image displayed on the display screen to realize the interaction between the user and the virtual scene, for example, based on the detected The rotation direction of the user's head continuously updates the real-life content.

反馈模块302:接收来自传感器的数据,为用户提供实时反馈;其中,该反馈模块302可以为用于显示图形用户界面,比如在该图形用户界面上显示虚拟环境。例如,该反馈模块302可以包括显示屏幕等。Feedback module 302: receives data from sensors and provides real-time feedback to the user; wherein, the feedback module 302 may be used to display a graphical user interface, such as displaying a virtual environment on the graphical user interface. For example, the feedback module 302 may include a display screen or the like.

传感器303:一方面接受来自用户的操作命令,并将其作用于虚拟环境;另一方面将操作后产生的结果以各种反馈的形式提供给用户。Sensor 303: On the one hand, it accepts operation commands from the user and applies them to the virtual environment; on the other hand, it provides the results of the operation to the user in the form of various feedbacks.

控制模块304:对传感器和各种输入/输出装置进行控制,包括获得用户的数据(如动作、语音)和输出感知数据,如图像、振动、温度和声音等,对用户、虚拟环境和现实世界产生作用。Control module 304: Controls sensors and various input/output devices, including obtaining user data (such as actions, voice) and output sensing data, such as images, vibrations, temperature and sound, etc., to the user, virtual environment and real world Have an effect.

建模模块305:构造虚拟环境的三维模型,还可以包括三维模型中的声音、触感等各种反馈机制。Modeling module 305: Constructs a three-dimensional model of the virtual environment, which can also include various feedback mechanisms such as sound and touch in the three-dimensional model.

在本申请实施例中,可以通过所述建模模块305构建目标虚拟空间,并通过所述控制模块304获取发声者的近场音频,获取目标虚拟空间的音频配置参数,所述音频配置参数用于模拟所述目标虚拟空间的空间音效,并根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,所述第一目标音频具有所述目标虚拟空间的空间音。In the embodiment of the present application, the target virtual space can be constructed through the modeling module 305, and the near-field audio of the speaker can be obtained through the control module 304, and the audio configuration parameters of the target virtual space can be obtained. The audio configuration parameters are To simulate the spatial sound effect of the target virtual space, and perform mixing processing on the near-field audio according to the audio configuration parameters of the target virtual space to obtain the first target audio, the first target audio having the target Spatial sound in virtual space.

在一些实施例中,如图5所示,图5为本申请实施例提供的虚拟现实设备的另一结构示意图,虚拟现实设备300还包括有一个或者一个以上处理核心的处理器310、有一个或一个以上计算机可读存储介质的存储器320及存储在存储器320上并可在处理器上运行的计算机程序。其中,处理器310与存储器320电性连接。本领域技术人员可以理解,图中示出的虚拟现实设备结构并不构成对虚拟现实设备的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。In some embodiments, as shown in Figure 5, which is another structural schematic diagram of a virtual reality device provided by an embodiment of the present application, the virtual reality device 300 also includes a processor 310 with one or more processing cores. Or the memory 320 of more than one computer-readable storage medium and the computer program stored on the memory 320 and executable on the processor. Among them, the processor 310 is electrically connected to the memory 320. Those skilled in the art can understand that the structure of the virtual reality device shown in the figures does not constitute a limitation on the virtual reality device, and may include more or less components than shown in the figures, or combine certain components, or arrange different components. .

处理器310是虚拟现实设备300的控制中心,利用各种接口和线路连接整个虚拟现实设备300的各个部分,通过运行或加载存储在存储器320内的软件程序和/或模块,以及调用存储在存储器320内的数据,执行虚拟现实设备300的各种功能和处理数据,从而对虚拟现实设备300进行整体监控。The processor 310 is the control center of the virtual reality device 300. It uses various interfaces and lines to connect various parts of the entire virtual reality device 300, by running or loading software programs and/or modules stored in the memory 320, and by calling the software programs and/or modules stored in the memory 320. The data in 320 executes various functions of the virtual reality device 300 and processes data, thereby monitoring the virtual reality device 300 as a whole.

在本申请实施例中,虚拟现实设备300中的处理器310会按照如下的步骤,将一个或一个以上的应用程序的进程对应的指令加载到存储器320中,并由处理器310来运行存储在存储器320中的应用程序,从而实现各种功能:In this embodiment of the present application, the processor 310 in the virtual reality device 300 will follow the following steps to load instructions corresponding to the processes of one or more application programs into the memory 320, and the processor 310 will run the instructions stored in the application in memory 320 to implement various functions:

获取发声者的近场音频;获取目标虚拟空间的音频配置参数,所述音频配置参数用于模拟所述目标虚拟空间的空间音效;根据所述目标虚拟空间的音频配置参数,对所述近场音频进行混音处理,得到第一目标音频,所述第一目标音频具有所述目标虚拟空间的空间音。Obtain the near-field audio of the speaker; obtain the audio configuration parameters of the target virtual space, the audio configuration parameters are used to simulate the spatial sound effect of the target virtual space; according to the audio configuration parameters of the target virtual space, the near-field The audio is mixed and processed to obtain the first target audio, which has the spatial sound of the target virtual space.

以上各个操作的具体实施可参见前面的实施例,在此不再赘述。For the specific implementation of each of the above operations, please refer to the previous embodiments and will not be described again here.

在一些实施例中,处理器310可以包括检测模块301、控制模块304和建模模块305。In some embodiments, processor 310 may include detection module 301, control module 304, and modeling module 305.

在一些实施例中,如图5所示,虚拟现实设备300还包括:射频电路306、音频电路307以及电源308。其中,处理器310分别与存储器320、反馈模块302、传感器303、射频电路306、音频电路307以及电源308电性连接。本领域技术人员可以理解,图4或图5中示出的虚拟现实设备结构并不构成对虚拟现实设备的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。In some embodiments, as shown in FIG. 5 , the virtual reality device 300 further includes: a radio frequency circuit 306 , an audio circuit 307 and a power supply 308 . Among them, the processor 310 is electrically connected to the memory 320, the feedback module 302, the sensor 303, the radio frequency circuit 306, the audio circuit 307 and the power supply 308 respectively. Those skilled in the art can understand that the structure of the virtual reality device shown in Figure 4 or Figure 5 does not constitute a limitation on the virtual reality device, and may include more or less components than shown, or combine certain components, or Different component arrangements.

射频电路306可用于收发射频信号,以通过无线通信与网络设备或其他虚拟现实设备建立无线通讯,与网络设备或其他虚拟现实设备之间收发信号。The radio frequency circuit 306 can be used to send and receive radio frequency signals to establish wireless communication with network equipment or other virtual reality equipment through wireless communication, and to send and receive signals with network equipment or other virtual reality equipment.

音频电路307可以用于通过扬声器、传声器提供用户与虚拟现实设备之间的音频接口。音频电路307可将接收到的音频数据转换后的电信号,传输到扬声器,由扬声器转换为声音信号输出;另一方面,传声器将收集的声音信号转换为电信号,由音频电路307接收后转换为音频数据,再将音频数据输出处理器301处理后,经射频电路306以发送给比如另一虚拟现实设备,或者将音频数据输出至存储器以便进一步处理。音频电路307还可能包括耳塞插孔,以提供外设耳机与虚拟现实设备的通信。The audio circuit 307 can be used to provide an audio interface between the user and the virtual reality device through speakers and microphones. The audio circuit 307 can transmit the electrical signal converted from the received audio data to the speaker, which converts it into a sound signal and outputs it; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received and converted by the audio circuit 307 The audio data is processed by the audio data output processor 301 and then sent to, for example, another virtual reality device via the radio frequency circuit 306, or the audio data is output to the memory for further processing. Audio circuitry 307 may also include an earbud jack to provide communication between peripheral headphones and the virtual reality device.

电源308用于给虚拟现实设备300的各个部件供电。The power supply 308 is used to power various components of the virtual reality device 300 .

尽管图4或图5中未示出,虚拟现实设备300还可以包括摄像头、无线保真模块、蓝牙模块、输入模块等,在此不再赘述。Although not shown in FIG. 4 or FIG. 5 , the virtual reality device 300 may also include a camera, a wireless fidelity module, a Bluetooth module, an input module, etc., which will not be described again here.

在一些实施例中,本申请还提供了一种服务器,包括存储器和处理器,存储器中存储有计算机程序,该处理器执行计算机程序时实现上述各方法实施例中的步骤。In some embodiments, this application also provides a server, which includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the steps in the above method embodiments.

在一些实施例中,本申请还提供了一种计算机可读存储介质,用于存储计算机程序。该计算机可读存储介质可应用于虚拟现实设备或服务器,并且该计算机程序使得虚拟现实设备或服务器执行本申请实施例中的音频处理方法中的相应流程,为了简洁,在此不再赘述。In some embodiments, the present application also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to a virtual reality device or server, and the computer program causes the virtual reality device or server to execute the corresponding process in the audio processing method in the embodiment of the present application. For the sake of brevity, details will not be repeated here.

在一些实施例中,本申请还提供了一种计算机程序产品,该计算机程序产品包括计算机程序,该计算机程序存储在计算机可读存储介质中。虚拟现实设备的处理器从计算机可读存储介质读取该计算机程序,处理器执行该计算机程序,使得虚拟现实设备执行本申请实施例中的音频处理方法中的相应流程,为了简洁,在此不再赘述。In some embodiments, the present application also provides a computer program product, the computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the virtual reality device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the virtual reality device to execute the corresponding process in the audio processing method in the embodiment of the present application. For the sake of simplicity, this is not mentioned here. Again.

本申请还提供了一种计算机程序,该计算机程序包括计算机程序,计算机程序存储在计算机可读存储介质中。虚拟现实设备的处理器从计算机可读存储介质读取该计算机程序,处理器执行该计算机程序,使得虚拟现实设备执行本申请实施例中的音频处理方法中的相应流程,为了简洁,在此不再赘述。The application also provides a computer program, the computer program includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the virtual reality device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the virtual reality device to execute the corresponding process in the audio processing method in the embodiment of the present application. For the sake of simplicity, this is not mentioned here. Again.

应理解,本申请实施例的处理器可能是一种集成电路芯片,具有信号的处理能力。在实现过程中,上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器可以是通用处理器、数字信号处理器(Digital SignalProcessor,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现成可编程门阵列(Field Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器,处理器读取存储器中的信息,结合其硬件完成上述方法的步骤。It should be understood that the processor in the embodiment of the present application may be an integrated circuit chip and has signal processing capabilities. During the implementation process, each step of the above method embodiment can be completed through an integrated logic circuit of hardware in the processor or instructions in the form of software. The above-mentioned processor can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field Programmable Gate Array, FPGA) or other programmable processors. Logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logical block diagram disclosed in the embodiment of this application can be implemented or executed. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other mature storage media in this field. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

可以理解,本申请实施例中的存储器可以是易失性存储器或非易失性存储器,或可包括易失性和非易失性存储器两者。其中,非易失性存储器可以是只读存储器(Read-Only Memory,ROM)、可编程只读存储器(Programmable ROM,PROM)、可擦除可编程只读存储器(Erasable PROM,EPROM)、电可擦除可编程只读存储器(Electrically EPROM,EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory,RAM),其用作外部高速缓存。通过示例性但不是限制性说明,许多形式的RAM可用,例如静态随机存取存储器(Static RAM,SRAM)、动态随机存取存储器(Dynamic RAM,DRAM)、同步动态随机存取存储器(Synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data RateSDRAM,DDR SDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM,SLDRAM)和直接内存总线随机存取存储器(DirectRambus RAM,DR RAM)。应注意,本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。It can be understood that the memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically removable memory. Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of illustration, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) and direct memory bus random access memory (DirectRambus RAM, DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

应理解,上述存储器为示例性但不是限制性说明,例如,本申请实施例中的存储器还可以是静态随机存取存储器(static RAM,SRAM)、动态随机存取存储器(dynamic RAM,DRAM)、同步动态随机存取存储器(synchronous DRAM,SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM,DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM,ESDRAM)、同步连接动态随机存取存储器(synch link DRAM,SLDRAM)以及直接内存总线随机存取存储器(Direct Rambus RAM,DR RAM)等等。也就是说,本申请实施例中的存储器旨在包括但不限于这些和任意其它适合类型的存储器。It should be understood that the above memory is illustrative but not restrictive. For example, the memory in the embodiment of the present application can also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), Synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous connection Dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM) and so on. That is, memories in embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

在本申请所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

另外,在本申请实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in the embodiment of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

所述功能若以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台虚拟现实设备(可以是个人计算机,服务器)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a virtual reality device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化或替换,都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应所述以权利要求的保护范围为准。The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application. should be covered by the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims (15)

1. A method of audio processing, the method comprising:
acquiring near-field audio of a sounder;
acquiring audio configuration parameters of a target virtual space, wherein the audio configuration parameters are used for simulating the space sound effect of the target virtual space;
and mixing the near-field audio according to the audio configuration parameters of the target virtual space to obtain a first target audio, wherein the first target audio has the space sound effect of the target virtual space.
2. The audio processing method of claim 1, wherein the audio configuration parameters of the target virtual space include an audio reflection parameter of the target virtual space, an attenuation parameter of audio propagation, and a background audio of the target virtual space.
3. The audio processing method according to claim 2, wherein the performing audio mixing processing on the near-field audio according to the audio configuration parameters of the target virtual space to obtain a first target audio includes:
according to the audio reflection parameter and the attenuation parameter, performing first audio adjustment processing on the near-field audio and the background audio;
and mixing the near-field audio after the first audio adjustment processing with the background audio to obtain a first target audio.
4. The audio processing method of claim 3, wherein the method further comprises:
acquiring a first position of a virtual sounding object corresponding to the sounder in the target virtual space and a second position of a virtual listening object corresponding to a listener in the target virtual space;
and performing a first audio adjustment process on the near-field audio and the background audio according to the audio reflection parameter and the attenuation parameter, including:
and according to the audio reflection parameters and the attenuation parameters, the first position and the second position perform first audio adjustment processing on the near-field audio and the background audio.
5. The audio processing method of claim 4, wherein the method further comprises:
when the relative position relation between the first position and the second position is detected to be changed, performing second audio adjustment processing on the near-field audio and the background audio according to the relative position relation between the first position and the second position;
and mixing the near-field audio subjected to the second audio adjustment processing with the background audio to obtain second target audio, wherein the second target audio has a spatial sound effect of the target virtual space and a sound effect change effect generated when the relative position relation between the first position and the second position is represented.
6. The audio processing method according to claim 5, wherein the relative positional relationship includes a distance relationship, and the performing a second audio adjustment process on the near-field audio and the background audio according to the relative positional relationship between the first position and the second position when the relative positional relationship between the first position and the second position is detected to change, includes:
and when the distance relation between the first position and the second position is detected to change, performing second audio adjustment processing on the near-field audio and the background audio according to the distance between the first position and the second position.
7. The audio processing method according to claim 5, wherein the relative positional relationship includes an azimuth relationship, and the performing, when detecting that the relative positional relationship between the first position and the second position changes, second audio adjustment processing on the near-field audio and the background audio according to the relative positional relationship between the first position and the second position includes:
and when detecting that the azimuth relation between the first position and the second position changes, performing second audio adjustment processing on the near-field audio and the background audio according to the relative azimuth between the first position and the second position.
8. The audio processing method according to claim 5, wherein the relative positional relationship includes a distance relationship and an azimuth relationship, and wherein when a change in the relative positional relationship between the first position and the second position is detected, performing a second audio adjustment process on the near-field audio and the background audio according to the relative positional relationship between the first position and the second position includes:
when the change of the distance relation and the azimuth relation between the first position and the second position is detected, performing second audio adjustment processing on the near-field audio and the background audio according to the distance and the relative azimuth between the first position and the second position.
9. The audio processing method of claim 4, wherein the method further comprises:
when the head swing of the listener is detected, acquiring a left ear orientation direction and a right ear orientation direction of the listener in the head swing process;
performing third audio adjustment processing on the near-field audio and the background audio according to the left ear orientation direction and the right ear orientation direction of the listener;
and mixing the near-field audio subjected to the third audio adjustment processing with the background audio to obtain third target audio, wherein the third target audio has a spatial sound effect of the target virtual space and a sound effect change effect generated when the head of the listener swings.
10. The audio processing method according to any one of claims 1 to 9, characterized in that the method further comprises:
according to different virtual scenes, presetting different types of virtual spaces, wherein each type of virtual space has corresponding audio configuration parameters;
the obtaining the audio configuration parameters of the target virtual space includes:
and responding to a selection instruction aiming at the virtual space, determining the target virtual space from the different types of virtual spaces, and acquiring audio configuration parameters of the target virtual space.
11. An audio processing apparatus, the apparatus comprising:
the first acquisition unit is used for acquiring near-field audio of a sounder;
the second acquisition unit is used for acquiring audio configuration parameters of the target virtual space, wherein the audio configuration parameters are used for simulating the space sound effect of the target virtual space;
and the processing unit is used for carrying out audio mixing processing on the near-field audio according to the audio configuration parameters of the target virtual space to obtain first target audio, wherein the first target audio has the space sound effect of the target virtual space.
12. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which is adapted to be loaded by a processor for performing the audio processing method according to any of claims 1-10.
13. A virtual reality device comprising a processor and a memory, the memory having stored therein a computer program for executing the audio processing method of any of claims 1-10 by invoking the computer program stored in the memory.
14. A server comprising a processor and a memory, the memory having stored therein a computer program for executing the audio processing method according to any one of claims 1-10 by calling the computer program stored in the memory.
15. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the audio processing method of any of claims 1-10.
CN202210869408.7A 2022-07-22 2022-07-22 Audio processing methods, devices, storage media and equipment Pending CN117476014A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210869408.7A CN117476014A (en) 2022-07-22 2022-07-22 Audio processing methods, devices, storage media and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210869408.7A CN117476014A (en) 2022-07-22 2022-07-22 Audio processing methods, devices, storage media and equipment

Publications (1)

Publication Number Publication Date
CN117476014A true CN117476014A (en) 2024-01-30

Family

ID=89633544

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210869408.7A Pending CN117476014A (en) 2022-07-22 2022-07-22 Audio processing methods, devices, storage media and equipment

Country Status (1)

Country Link
CN (1) CN117476014A (en)

Similar Documents

Publication Publication Date Title
US10979842B2 (en) Methods and systems for providing a composite audio stream for an extended reality world
KR101576294B1 (en) Apparatus and method to perform processing a sound in a virtual reality system
US11128977B2 (en) Spatial audio downmixing
JP2025023214A (en) Spatial Audio for Two-Way Audio Environments
US11109177B2 (en) Methods and systems for simulating acoustics of an extended reality world
JP2022502886A (en) Audio equipment and audio processing methods
US11221821B2 (en) Audio scene processing
JP7478100B2 (en) Reverberation Gain Normalization
JP2023546839A (en) Audiovisual rendering device and method of operation thereof
JP2023168544A (en) Low-frequency interchannel coherence control
KR20210056414A (en) System for controlling audio-enabled connected devices in mixed reality environments
WO2024060959A1 (en) Method and apparatus for adjusting viewing picture in virtual environment, and storage medium and device
US20240420718A1 (en) Voice processing for mixed reality
CN117476014A (en) Audio processing methods, devices, storage media and equipment
JP7053074B1 (en) Appreciation system, appreciation device and program
CN116764195A (en) Audio control methods, devices, electronic equipment and media based on virtual reality VR
JP7700185B2 (en) Spatial Audio for Two-Way Audio Environments
CN117115237A (en) Virtual reality position switching method, device, storage medium and equipment
WO2024034396A1 (en) Information processing device, information processing method, and program
CN119421100A (en) Audio processing method, device, electronic device and readable medium
CN116391171A (en) Dynamically changing audio characteristics
CN118161857A (en) Task display method, device, storage medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination