CN112604285A - Method and system for sound source virtualization - Google Patents
Method and system for sound source virtualization Download PDFInfo
- Publication number
- CN112604285A CN112604285A CN202011505534.1A CN202011505534A CN112604285A CN 112604285 A CN112604285 A CN 112604285A CN 202011505534 A CN202011505534 A CN 202011505534A CN 112604285 A CN112604285 A CN 112604285A
- Authority
- CN
- China
- Prior art keywords
- sound
- external
- person
- virtual environment
- personnel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 16
- 239000011521 glass Substances 0.000 claims abstract description 35
- 230000002452 interceptive effect Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 3
- 238000007654 immersion Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/54—Controlling the output signals based on the game progress involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
- A63F2300/8082—Virtual reality
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a method and a system for virtualizing a sound source, which comprise the following steps: an external voice obtaining step: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel; virtual character orientation obtaining step: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment; a sound source virtualization step: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position. The invention can virtualize the sound source. For example, in the game, user a is in front of user B, and the real utterance statement of user a is transmitted from the left of user B (for example, two people are actually sitting side by side), then VR glasses of user B receive the real voice of user a first, and then virtualize the voice of user a as being transmitted from the front of user B according to the position relationship in the game.
Description
Technical Field
The invention relates to the field of VR glasses, in particular to a method and a system for virtualizing sound sources.
Background
Patent document CN109598997A discloses a construction safety training education system based on combination of BIM and VR, which includes a Revit modeling module, a 3Dmax optimization processing module and a UE4 programming module; the method comprises the steps of establishing a BIM (building information modeling) model of a construction project by using a Revit modeling module, outputting a FBX file in the Revit modeling module, importing the FBX file into a 3Dmax optimization processing module, performing processing optimization on the BIM model of the construction project by using 3Dmax in the 3Dmax optimization processing module, continuously outputting the FBX file by using 3Dmax in the 3Dmax optimization processing module, importing the FBX file into a UE4 programming module and performing VR (virtual reality) program programming, rapidly updating the BIM model by importing the BIM model into a UE4 programming module, and making a safety experience point of the BIM model according to the actual danger source customization programming on site. Safety experience personnel wear VR glasses in the VR scene, can see virtual character each other in the VR scene, operate by a main control personnel, take all the other personalities to experience the VR effect together.
Although patent document CN109598997A can see virtual characters in a VR scene, if a person once speaks to communicate, the orientation of the virtual character is different from the orientation of the actual person, which causes the screen to be not in accordance with the sound and the sense of immersion to be lost.
Disclosure of Invention
In view of the deficiencies in the prior art, it is an object of the present invention to provide a method and system for sound source virtualization.
The invention provides a method for virtualizing a sound source, which comprises the following steps:
an external voice obtaining step: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel;
virtual character orientation obtaining step: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment;
a sound source virtualization step: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position.
Preferably, the external person wears VR glasses, the virtual environment is a multi-person interactive scene, and the external person observes the virtual environment through the worn VR glasses at the viewing angle of the virtual person.
Preferably, in the sound source virtualization step, the sound of the external person is played in a virtual environment after being sound-processed.
Preferably, the sound processing is to adjust the tone of the sound of the external person to the tone of a virtual character corresponding to the external person in the virtual environment.
Preferably, the sound processing is to increase the play rate of the sound of the external person.
The invention provides a system for virtualizing a sound source, which comprises:
external human voice acquisition module: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel;
virtual character orientation acquisition module: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment;
a sound source virtualization module: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position.
Preferably, the external person wears VR glasses, the virtual environment is a multi-person interactive scene, and the external person observes the virtual environment through the worn VR glasses at the viewing angle of the virtual person.
Preferably, in the sound source virtualization module, the sound of the external person is played in the virtual environment after being subjected to sound processing.
Preferably, the sound processing is to adjust the tone of the sound of the external person to the tone of a virtual character corresponding to the external person in the virtual environment.
Preferably, the sound processing is to increase the play rate of the sound of the external person.
Compared with the prior art, the invention has the following beneficial effects:
the invention can virtualize the sound source. For example, in the game, user a is in front of user B, and the real utterance statement of user a is coming from the left of user B (for example, two people are actually sitting side by side), then VR glasses of user B receive the real voice of user a first, and then virtualize the voice of user a in the virtual environment as being emitted from the front of user B according to the position relationship in the virtual environment, for example, the game.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flow chart of the steps of the method of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
The invention provides a method for virtualizing a sound source, which comprises the following steps:
an external voice obtaining step: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel; specifically, the VR glasses have a microphone capable of collecting external sounds. For example, VR glasses of user a capture the voice of user B speaking.
Virtual character orientation obtaining step: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment; specifically, a person skilled in the art can realize a virtual character at least by referring to patent document CN109598997A, and details thereof are not described herein, since it can be recognized which external person uttered the sound and the external person has a corresponding virtual character in the virtual environment.
A sound source virtualization step: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position. The VR glasses provide earmuffs to isolate the VR glasses wearer from outside sound transmission. Those skilled in the art can refer to the earphone in the prior art to realize the isolation of the VR glasses wearer from the external sound, which is not described herein.
The external personnel wear VR glasses, the virtual environment is a multi-person interactive scene, and the external personnel observe the virtual environment through the worn VR glasses at the visual angle of the virtual person. In the sound source virtualization step, sound processing is performed on the sound of the external person, and then the sound is played in a virtual environment. For example, in the game, user a is in front of user B, and the real utterance statement of user a is coming from the left of user B (for example, two people are actually sitting side by side), then VR glasses of user B receive the real voice of user a first, and then virtualize the voice of user a in the virtual environment as being emitted from the front of user B according to the position relationship in the virtual environment, for example, the game. The sound processing is to adjust the tone of the sound of the external person to the tone of a virtual character corresponding to the external person in the virtual environment, thereby further improving the sense of immersion. The sound processing is to increase the playing speed of the sound of the external person so as to reduce the influence of sound delay.
The invention provides a system for virtualizing a sound source, which comprises:
external human voice acquisition module: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel; specifically, the VR glasses have a microphone capable of collecting external sounds. For example, VR glasses of user a capture the voice of user B speaking.
Virtual character orientation acquisition module: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment; specifically, a person skilled in the art can realize a virtual character at least by referring to patent document CN109598997A, and details thereof are not described herein, since it can be recognized which external person uttered the sound and the external person has a corresponding virtual character in the virtual environment.
A sound source virtualization module: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position. The VR glasses provide earmuffs to isolate the VR glasses wearer from outside sound transmission. Those skilled in the art can refer to the earphone in the prior art to realize the isolation of the VR glasses wearer from the external sound, which is not described herein.
The external personnel wear VR glasses, the virtual environment is a multi-person interactive scene, and the external personnel observe the virtual environment through the worn VR glasses at the visual angle of the virtual person. And in the sound source virtualization module, the sound of the external personnel is played in a virtual environment after being subjected to sound processing. For example, in the game, user a is in front of user B, and the real utterance statement of user a is coming from the left of user B (for example, two people are actually sitting side by side), then VR glasses of user B receive the real voice of user a first, and then virtualize the voice of user a in the virtual environment as being emitted from the front of user B according to the position relationship in the virtual environment, for example, the game. The sound processing is to adjust the tone of the sound of the external person to the tone of a virtual character corresponding to the external person in the virtual environment, thereby further improving the sense of immersion. The sound processing is to increase the playing speed of the sound of the external person so as to reduce the influence of sound delay.
One skilled in the art will appreciate that, in addition to implementing the system, apparatus, and various modules thereof provided by the present invention in pure computer readable program code, the system, apparatus, and various modules thereof provided by the present invention can be implemented by entirely logically programming the method modules such that the same programs are implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Therefore, the system, the device and the modules thereof provided by the present invention can be considered as a hardware component, and the modules included in the system, the device and the modules thereof for implementing various programs can also be considered as structures in the hardware component; modules for performing various functions may also be considered to be both software programs for performing the methods and structures within hardware components.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (10)
1. A method for sound source virtualization, comprising:
an external voice obtaining step: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel;
virtual character orientation obtaining step: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment;
a sound source virtualization step: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position.
2. The method of sound source virtualization according to claim 1, wherein the external person wears VR glasses, the virtual environment is a multi-person interactive scene, and the external person observes the virtual environment from the perspective of the virtual person through the worn VR glasses.
3. The method of claim 1, wherein in the sound source virtualization step, the sound of the external person is processed and played in the virtual environment.
4. The method of claim 3, wherein the sound processing is adjusting the sound color of the external person to the sound color of the avatar corresponding to the external person in the virtual environment.
5. The method of claim 3, wherein the sound processing is to increase the play rate of the sound of the external person.
6. A system for virtualization of a sound source, comprising:
external human voice acquisition module: enabling the VR glasses to acquire the sound of the outside personnel and recording the sound of the outside personnel;
virtual character orientation acquisition module: obtaining the external personnel according to the voice recognition, and confirming the orientation of a virtual character corresponding to the external personnel in the virtual environment;
a sound source virtualization module: and playing the sound of the external personnel in the virtual environment by taking the orientation of the virtual character as the sound source position.
7. The system of sound source virtualization of claim 6, wherein the external person wears VR glasses, the virtual environment is a multi-person interactive scene, and the external person views the virtual environment from the perspective of the virtual person through the worn VR glasses.
8. The system of claim 6, wherein the sound source virtualization module performs sound processing on the sound of the external person and plays the processed sound in the virtual environment.
9. The system of claim 8, wherein the sound processing is adjusting the sound color of the external person to the sound color of the avatar corresponding to the external person in the virtual environment.
10. The system of claim 8, wherein the sound processing is to increase a playback rate of the external person's sound.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011505534.1A CN112604285A (en) | 2020-12-18 | 2020-12-18 | Method and system for sound source virtualization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011505534.1A CN112604285A (en) | 2020-12-18 | 2020-12-18 | Method and system for sound source virtualization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112604285A true CN112604285A (en) | 2021-04-06 |
Family
ID=75241168
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011505534.1A Pending CN112604285A (en) | 2020-12-18 | 2020-12-18 | Method and system for sound source virtualization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112604285A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102015003949A1 (en) * | 2015-03-26 | 2016-09-29 | Audi Ag | Method for operating a virtual reality system and virtual reality system |
| CN109453526A (en) * | 2018-10-26 | 2019-03-12 | 努比亚技术有限公司 | A kind of sound processing method, terminal and computer readable storage medium |
| CN110692033A (en) * | 2017-05-29 | 2020-01-14 | 奥迪股份公司 | Method for operating a virtual reality system and virtual reality system |
| CN111158459A (en) * | 2018-11-07 | 2020-05-15 | 辉达公司 | Application of geometric acoustics in immersive Virtual Reality (VR) |
| CN111966222A (en) * | 2020-08-12 | 2020-11-20 | 徐雪峰 | High-safety VR virtual reality device, system and method |
-
2020
- 2020-12-18 CN CN202011505534.1A patent/CN112604285A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102015003949A1 (en) * | 2015-03-26 | 2016-09-29 | Audi Ag | Method for operating a virtual reality system and virtual reality system |
| CN110692033A (en) * | 2017-05-29 | 2020-01-14 | 奥迪股份公司 | Method for operating a virtual reality system and virtual reality system |
| CN109453526A (en) * | 2018-10-26 | 2019-03-12 | 努比亚技术有限公司 | A kind of sound processing method, terminal and computer readable storage medium |
| CN111158459A (en) * | 2018-11-07 | 2020-05-15 | 辉达公司 | Application of geometric acoustics in immersive Virtual Reality (VR) |
| CN111966222A (en) * | 2020-08-12 | 2020-11-20 | 徐雪峰 | High-safety VR virtual reality device, system and method |
Non-Patent Citations (1)
| Title |
|---|
| 洪炳镕: "虚拟现实及其应用", 国防工业出版社, pages: 112 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11977670B2 (en) | Mixed reality system for context-aware virtual object rendering | |
| CN107683166B (en) | Filtering and parental control methods for limiting visual activity on a head-mounted display | |
| CN108701369B (en) | Device and method for producing and packaging entertainment data aiming at virtual reality | |
| CN106663219A (en) | Methods and systems of handling a dialog with a robot | |
| US11113884B2 (en) | Techniques for immersive virtual reality experiences | |
| CN107240319B (en) | A kind of interaction Scene Teaching system for the K12 stage | |
| CN106683501B (en) | A kind of AR children scene plays the part of projection teaching's method and system | |
| US20120223952A1 (en) | Information Processing Device Capable of Displaying A Character Representing A User, and Information Processing Method Thereof. | |
| JP2020039029A (en) | Video distribution system, video distribution method, and video distribution program | |
| CN105764581A (en) | Variable audio parameter setting | |
| US20200344549A1 (en) | Synchronized multiuser audio | |
| CN107708819A (en) | Response formula animation for virtual reality | |
| CN111654715A (en) | Live video processing method and device, electronic equipment and storage medium | |
| CN204406327U (en) | Based on the limb rehabilitating analog simulation training system of said three-dimensional body sense video camera | |
| US20220383849A1 (en) | Simulating crowd noise for live events through emotional analysis of distributed inputs | |
| JP2016045814A (en) | Virtual reality service providing system and virtual reality service providing method | |
| US12120493B2 (en) | Method and device for sound processing for a synthesized reality setting | |
| US20240046914A1 (en) | Assisted speech | |
| CN204233735U (en) | A kind of 5D body sense interactive game device | |
| CN112604285A (en) | Method and system for sound source virtualization | |
| KR20210055235A (en) | Hand sign language image generation system based on Generative Adversarial Networks | |
| Al Moubayed et al. | Lip-reading: Furhat audio visual intelligibility of a back projected animated face | |
| Huang et al. | Influence of Sound Direction and Distance on Immersive Experience in VR Gaming | |
| US20240013488A1 (en) | Groups and Social In Artificial Reality | |
| CN110444213A (en) | A kind of audio recognition method and system based on AR equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |