KR100792006B1 - An apparatus and method for reproducing acoustic signal - Google Patents

Abstract

The present invention relates to an apparatus and a method for providing an effective auditory effect when a virtual reality is implemented, and generates a sound signal required for virtual reality of a realistic model such as a communication device, a computer, and other sound reproducing devices and software related to sound reproducing. It relates to an apparatus and a method. The present invention constructs a sound file in a different manner from the conventional method in order to provide a more efficient auditory effect, and creates and plays a sound file by considering and defining sound elements related to the state of reality in the realistic model.

Virtual reality, sound, machine operation sound, realistic model

Description

An apparatus and method for reproducing acoustic signal in virtual reality

1 illustrates an example of a structure of a conventional sound file format.

FIG. 2 illustrates an example of a structure of a new sound file format in which the sound file format of FIG. 1 used in one embodiment of the acoustic signal generating apparatus of the present invention is modified.

3 illustrates an embodiment of a process of converting a sound signal in the sound signal generating apparatus of the present invention.

FIG. 4 illustrates an embodiment of a process of converting an acoustic signal by a sound element of “operation intensity” in the example of FIG. 3.

FIG. 5 illustrates an embodiment of a process of converting an acoustic signal by a sound element of “contact area” in the example of FIG. 3.

6 illustrates a process in which an embodiment of a sound reproducing method in a virtual reality implementing a sensory model is performed.

{Description of main parts of the drawing}

601: Process performed in the virtual reality

602: process performed inside the sound reproducing apparatus

603: process performed inside the sound signal generating apparatus

The present invention relates to a device and a method for providing an effective auditory effect in the implementation of virtual reality, and to a mechanical operation required for virtual reality of a realistic model such as a communication device, a computer, and other sound reproducing devices and software related to sound reproduction. A device and method for generating the same.

Figure 1 shows an example of the structure of the WAVE sound file format of Microsoft Corporation, one of the most representative of the existing sound file format. The WAVE sound file format is defined as configuration data stored in the RIFF method among data storage methods.

In the example of FIG. 1, the RIFF scheme is a file format for multimedia and includes chunks 101 to 103 and sub-chunks 104 to 114.

In the above example, the chunks 101 to 103 include fields such as a chunk identifier (ChunkID) 101, a chunk size (ChunkSize, 102), a file format (Format, 103), and the like. The specified file is in RIFF file format. The chunk identifier 101 is for identifying each chunk included in the file, and a string of “RIFF” is stored in a size of 4 bytes. The chunk size 102 represents the size of data to be continued, and a size of 4 bytes is stored by subtracting 8 bytes from the size of the WAVE sound file. The file format 103 indicates the type of file, and specifies that the character string "WAVE" is stored in a size of 4 bytes and thus is a WAVE sound file format.

In the above example, a sub-chunk 104-111 of "fmt" is a 4-byte sub-chunk identifier (SubChunk1ID, 104) in which a string of "fmt" is stored to identify the sub-chunk. 4 bytes of SubChunk1Size (105) to specify the size of the data following the subchunk, 2 bytes of Audio format (AudioFormat, 106) to specify the format of the sound data such as PCM, mono or stereo, etc. Number of channels (NumChannels, 107) having a size of 2 bytes that designates the number of channels of sound data, Samples having a size of 4 bytes (SampleRate, 108) representing the number of stored samples per second of the sound data, and storage of the sound data per second. 4 bytes of sample bytes per second (ByteRate, 109) representing the sample size of the sound, 2 bytes of samples per block (BlockAlign, 110) representing the size of each sample, and each of the samples The number of bits per sample in a two-byte size indicating teusu (BitsPerSample, 111) comprises a field and the like.

In the above example, sub-chunks 112 to 114 of "data" in which sound data are stored are four-byte sub-chunk identifiers (SubChunk2ID, 112) storing a "data" string that specifies the name of the sub-chunk. SubChunk 2 Size (SubChunk2Size, 113) having a 4-byte size that designates the size of the data following the subchunk, and sound data is stored, the size of which is variable according to the size of the sound data (data, 114), etc. Field of.

In the conventional sound reproducing system as described above, no special operation can be performed on the sound source other than the size adjustment. In a virtual reality situation in which a variety of situations can occur, it is difficult for an existing sound reproduction system to provide an effective auditory element to an experienced person, and the cost of generating a sound source is high due to the need to store a large number of sound sources.

Disclosure of the Invention An object of the present invention, which is devised to solve the above problems, is to provide an apparatus for generating an acoustic signal that effectively provides an auditory effect in the implementation of virtual reality by including sound elements related to the state of reality of the sensory model. have.

Another object of the present invention is to provide a method for generating and reproducing a sound element related to the state of reality in the sensory model.

The sound signal generating apparatus of the present invention devised to solve the above problems, in generating the acoustic signal for the auditory effect in the virtual reality implemented through the sensory model, the basic sound source signal and at least one specific And a module for receiving a control signal representing a sound element and outputting a sound signal obtained by converting the basic sound source signal by the control signal.

In the present invention, the specific sound element is a reality environment using the sensory model, the material of the sensory model, the surface state of the sensory model, the hardness of the sensory model, the size of the sensory model, between the sensory model and the object outside the sensory model At least one of the contact area, the intensity of manipulating the sensory model, and the position of sound generated when manipulating the sensory model in virtual reality is preferable.

In the present invention, the module includes at least one unit for receiving a control signal representing the at least one or more sound elements, respectively, wherein the at least one or more units are each sequentially receiving the basic sound source signal by the received control signal It is desirable to convert.

In the present invention, it is preferable that the module receives the basic sound source signal in the form of a sound file, and receives the control signal as data of an integer value stored in a chunk added to the sound file of the basic sound source signal.

In the present invention, the sound signal generating apparatus further comprises a storage unit for storing a basic sound source, the module preferably receives one selected from the basic sound source stored in the storage unit.

The sound reproducing apparatus of the present invention is characterized by including the above-described sound signal generating apparatus.

The present invention also includes the step of requesting a sound having at least one specific sound element to the sound signal generating apparatus in the virtual reality implementing the sensory model; Searching whether any of the sounds stored in the sound signal generating device have the specific sound element; In the case where there is no sound corresponding to the search result, receiving a basic sound source from the sound signal generating apparatus; Converting the input basic sound source into a sound having the specific sound element; And outputting the sound converted by the basic sound source to the virtual reality in the sound signal generating apparatus.

In the present invention, the specific sound element is a reality environment using the sensory model, the material of the sensory model, the surface state of the sensory model, the hardness of the sensory model, the size of the sensory model, between the sensory model and the object outside the sensory model At least one of the contact area, the intensity of manipulating the sensory model, and the position of sound generated when manipulating the sensory model in virtual reality is preferable.

According to an embodiment of the present invention, "event type" and "material" are sound elements to be included in the acoustic signal of the virtual reality generated to effectively provide the auditory effect when the virtual reality is realized through the sensory model. "," Surface "," hardness "," mass "," contact area "," intensity ", and" direction " . An integer value is assigned to the sound elements, and the integer value is data representing a specific value of the sound element when the sound signal is expressed in the form of a sound file.

The event type defines the situation when using the sensory model. First, classify according to the function of the sensory model, second classify according to the sound generated when operating the sensory model itself, and finally classify according to the use of accessories additionally connected to the main body of the sensory model. The following table is a part of an example of defining an event type.

Table 1 Example of Integer Value Definitions for Event Types.

event type Integer value  Sound by function When you press the Power on button 000 When pressing the power off button 001 When you press the Play button 002  Encouragement When we drop sense model 100 When rubbing the realistic model 101 When you scratch the realistic model 102  Sound by accessories When the wire of the earphone rubs on another object 200 When dropping earphone 201 When connecting a microphone 202

The material defines what the sensory model is made of. The following table is a part of an example of defining materials.

Table 2 Example of defining integer values for materials.

material metal plastic tree Glass Integer value 0 One 2 3

The surface represents the surface state of the sensory model, the hardness represents the hardness and hardness of the sensory model, respectively, and the mass defines the size of the sensory model. The contact area represents the contact area when there is contact between the sensory model and the object or the sensory model and the user. The intensity represents the intensity with which the user manipulates the sensory model. The direction specifies the location of sound generated when manipulating the sensory model in virtual reality.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In adding reference numerals to components of the following drawings, it is determined that the same components have the same reference numerals as much as possible even if displayed on different drawings, and it is determined that they may unnecessarily obscure the subject matter of the present invention. Detailed descriptions of well-known functions and configurations will be omitted.

With the existing sound file format, it is difficult to control the machine operation sound according to the values of sound elements, so it is necessary to use the new sound file format. One embodiment of the present invention uses a file format modified from Microsoft's WAVE sound file format, which is a sound file format capable of multi-channel support in order to provide stereoscopic sound in virtual reality.

FIG. 2 illustrates an example of a structure of a new sound file format in which the sound file format of FIG. 1 used in one embodiment of the acoustic signal generating apparatus of the present invention is modified.

In the example of FIG. 2, the chunk 201 indicating that the sound file is in the RIFF file format corresponds to the chunks 101 to 103 of FIG. 1. In the example of FIG. 2, the "fmt" subchunk ("fmt" subchunk 202) and the "data" subchunk ("data" subchunk 204) are respectively the "fmt" subchunk 104-111 of FIG. ) And "data" subchunks (112-114).

In the example of FIG. 2, a part different from the example of FIG. 1 is that an "inf" sub-chunk 203 is placed between the "fmt" subchunk 202 and the "data" subchunk 204. In addition to being located. The " inf " subchunk is for indicating a sound element to be used for effective generation of an acoustic signal and storing the value thereof. In one embodiment of the present invention, the size of the byte is 32 bytes.

The "inf" subchunk 203 is a 4-byte subChunk3ID (SubChunk3ID) 211 in which a string of the subchunk name "inf" is stored to identify the subchunk, and the size of subsequent data of the subchunk. SubChunk3Size 212 having a 4-byte size, and portions 213 to 220 specifying sound elements. In the above example, the sound element is an event type 213 in 2 bytes, a material 214 in 2 bytes, a surface 215 in 2 bytes, 2 bytes as described above. Low hardness (216), 4 bytes of mass (217), 4 bytes of contact area (218), 4 bytes of intensity (219), and 4 bytes of direction Specify (direction) (220).

3 illustrates an embodiment of a process of converting a sound signal in the sound signal generating apparatus of the present invention.

The acoustic signal generating apparatus of the embodiment of FIG. 3 has a "operational intensity" indicating the intensity at which the user manipulates the sensory model among the controllable sound elements defined above, and the contact when there is a contact between the sensory model and the object or the sensory model and the user. And a module for controlling the "contact area" representing the area.

In the embodiment of Fig. 3, the sound elements 304 of the acoustic signal include "button type" 301, "intensity" 302, and "contact area" ( 303) and the like, and control signals respectively representing the sound elements are detected in each unit of "sound type selection" 305, "amplitude conversion" 306, and "spectral conversion" 307, respectively, In the unit, an acoustic signal is converted by the control signal. In the embodiment, the module in the sound signal generating apparatus converts the sound signal by the detected control signal to the input 308 and the "operation intensity" of the basic sound source by the control signal indicating the "sound type". The conversion 309 by the control signal shown, the conversion 310 by the control signal indicating the "contact area", and the like are made, and finally, the sound signal 311 which has undergone all the conversion is output.

In this embodiment, the sound elements 304 of the acoustic signal include "sound type", "operation intensity", and "contact area". The " sound type " 301 designates a prototype (prototype) of a signal according to the type of sound signal to be generated, and the characteristics of sound vary according to the prototype of the signal. The "operation intensity" 302 designates an intensity for operating a machine, such as a sensory model. The "contact area" 303 designates the area in contact when operating the machine.

In the above embodiment, the sound type selection unit 305 designates the prototype of the sound signal in accordance with the type of the sound signal specified in the " sound type " That is, instead of receiving the basic sound source, a suitable one among the basic sound sources already recognized or stored in the storage unit and the like is selected according to the control signal indicating the "sound type", and the basic sound source to be converted in subsequent units. Output as 308.

Subsequently, the amplitude conversion unit 306 detects a control signal indicating the “operation intensity” 302 and, according to the value of the detected control signal, of the prototype acoustic signal 308 output from the unit 305. The sound signal obtained by converting the magnitude or the amplitude is output (309).

Subsequently, the spectrum conversion unit 307 detects a control signal indicating the "contact area" 303, and the acoustic signal 309 whose amplitude output from the unit 306 is converted according to the value of the detected control signal. A sound signal obtained by converting the spectrum of the sound signal is output (310), and thus the finally converted sound signal 311 is output.

FIG. 4 is a view briefly showing an embodiment of a process of converting an acoustic signal by a sound element of "operation intensity" in the example of FIG.

In the embodiment of Fig. 4, the unit performing the conversion of the acoustic signal by the sound element of the "operational intensity" receives an input signal 401, receives a control signal 402 of the "operational intensity" and receives the value thereof. And outputs an output signal 404 whose amplitude is converted as a result of the conversion by the amplitude conversion unit 403 for converting the amplitude of the input signal in accordance with the value of the detected control signal.

FIG. 5 is a view schematically illustrating an embodiment of a process of converting an acoustic signal by a sound element having a "contact area" in the example of FIG. 3.

In the embodiment of Fig. 5, the unit which performs the conversion of the acoustic signal by the sound element of the "contact area" receives an input signal 501, receives a control signal 502 of the "contact area" and receives its value. And outputs the output signal 504 whose spectrum is converted as a result of the conversion by the spectrum conversion unit 503 for converting the spectrum of the input signal in accordance with the value of the detected control signal.

The module in the acoustic signal generating apparatus of the present invention is preferably configured with one or more units for receiving a control signal for each sound element and converting the signal as shown in the embodiments of FIGS. 4 and 5. . In addition, it is preferable that one or more units provided in the module convert the basic sound source signals sequentially as shown in FIG. 3.

6 illustrates a process in which an embodiment of a sound reproducing method in a virtual reality implementing a sensory model is performed. In addition, the embodiment of the sound reproduction method is made through an embodiment of the sound reproduction apparatus of the present invention.

In the embodiment of FIG. 6, first, a sound having a specific sound element is transferred from the virtual reality 601 implementing the sensory model to the sound signal generating apparatus 603 or the sound reproducing apparatus 602 including the sound signal generating apparatus. Request (S1). Subsequently, it is determined whether or not corresponding to the requested sound is stored in the storage unit provided in the sound reproducing apparatus (S2). If there is a thing corresponding to the requested sound in the storage unit, the corresponding sound may be output. Otherwise, the sound signal generating device 603 included in the sound reproducing apparatus 602 generates a sound signal. To receive the basic sound source signal is received (S3). Subsequently, the basic sound source signal is converted (S4), and finally, the converted sound signal is output as sound (S5).

In the above embodiment, step S1 is a process performed in the virtual reality 601, but all other remaining steps S2 to S5 are performed in the sound reproducing apparatus 602. Even in the sound reproducing apparatus, steps S3 and S4 are performed by the sound signal generating apparatus 603.

In the above embodiment, in step S1, the virtual reality 601 requests the sound reproducing apparatus 602 for sound with a particular sound element. The specific sound element may be a real environment using the sensory model, the material of the sensory model, the surface state of the sensory model, the hardness of the sensory model, the size of the sensory model, the contact area between the sensory model and the objects outside the sensory model, and the sensory experience. It is related to the state of reality of the realistic model, such as the intensity of manipulating the model and the position of sound generated when manipulating the sensory model in virtual reality. It is desirable to feel the sense. Preferably, the virtual reality specifies at least one or more of the specific sound elements and values required for each element, and requests the corresponding sound. The value of the sound element required by the virtual reality is then input as a control signal to the sound signal generating device in the sound reproducing apparatus.

In step S2, the storage unit provided in the sound reproducing apparatus 602 searches whether there is a sound corresponding to the sound requested by the virtual reality.

The sound reproducing apparatus 602 is preferably sound files obtained through a preprocessing process for classifying, defining, and collecting sound in advance, in particular, when converting a sound signal in a simple or frequently required sound signal and a sound signal generating device. It is preferable to have a storage unit for storing the basic sound source signal or the like to be used. The sound file and the basic sound source signal stored in the storage unit have a chunk for storing data of at least one sound element related to the state of reality of the sensory model, for example, as shown in FIG. Preferably, the file is converted into a file format, and the file format is a file format suitable for conversion by the sound signal generating apparatus in the sound reproducing apparatus.

In the step S2, the sound file of the storage unit stored as a file having a chunk for storing data of the sound element as shown in FIG. 2 determines whether there is a sound element as required by the virtual reality. If a sound file as described above exists, the sound of the file is output.

In step S3, when there is no sound file with a particular sound element requested by the virtual reality as a result of the determination in step S2, generating a sound as required by the virtual reality from a basic sound source signal. The sound signal generating device 603 receives a basic sound source signal.

Subsequently, in step S4, the sound signal generating device 603 converts the basic sound source signal according to the data of the specific sound element requested by the virtual reality, and in step S5, the sound is output. In the conversion process, when the conversion is performed by two or more sound elements, the conversion is sequentially performed by each sound element, and an embodiment thereof is illustrated in FIG. 3.

The sound signal generating apparatus 603 converts the input of the basic sound source signal according to the data (control signal) of a specific sound element received from the virtual reality, and the sound signal generating apparatus converts the basic sound source signal into a sound. In the form of a file, the control signal is preferably input in the form of data stored in a chunk added to the sound file.

In relation to the input of the basic sound source signal and the control signal, as described above, the basic sound source signal is preferably stored in the storage unit, and the basic sound source signal stores data of the specific sound element. It is preferably stored as a file having a chunk. The data (control signal) of the sound element requested by the virtual reality is recorded in the chunk for storing data of the sound element in the sound file of the basic sound source signal before the step of converting the sound signal (S4). The sound file of the basic sound source signal is modified, and the sound signal generating device reads the sound file of the modified basic sound source signal and receives the basic sound source signal and the control signal.

As described above, it has been described with reference to a preferred embodiment of the present invention, but those skilled in the art various modifications and variations of the present invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

As described above, according to the sound signal generating apparatus or the sound reproducing method of the present invention, when the virtual reality is realized through the sensory model, sound can be more efficiently generated or converted and reproduced, and various sounds can be generated from a relatively small number of basic sound sources. The ability to create and play back also reduces the cost and storage space for creating and storing music.

Claims (13)

delete delete delete delete delete delete delete delete delete delete delete delete In the virtual reality that implements the sensory model, one or more specific sound files that are sequentially changed using a control signal representing one or more specific sound elements by receiving a basic sound source signal or a sound file representing the basic sound source signal as an integer value stored in a chunk. Requesting an acoustic signal having a sound element of the; Receiving data of one or more specific sound elements stored in the sound file as a control signal and converting the sound file into a requested sound signal; And And outputting the converted sound signal to the virtual reality.

Images