WO2016148553A2

WO2016148553A2 - Method and device for editing and providing three-dimensional sound

Info

Publication number: WO2016148553A2
Application number: PCT/KR2016/002826
Authority: WO
Inventors: 구본희; 이종석; 김대진; 김동준
Original assignee: (주)소닉티어랩
Priority date: 2015-03-19
Filing date: 2016-03-21
Publication date: 2016-09-22
Also published as: WO2016148553A3

Abstract

Provided is a method and device for editing and reproducing a three-dimensional sound. A three-dimensional sound providing device provides an edition function related to values of one or more parameters for generating the three-dimensional sound. A user of the three-dimensional sound providing device may set the values of the parameters through a graphic interface. A property of a source of a sound positioned in a three-dimensional space may be set through the values of the parameters. The three-dimensional sound providing device generates data of three-dimensional sound on the basis of the values of the parameters.

Description

Method and apparatus for editing and providing three-dimensional sound

The following embodiments relate to three-dimensional sound, and more particularly, a method and apparatus for providing editing of a three-dimensional sound and providing data of the three-dimensional sound generated in accordance with the edit are disclosed.

Recently, three-dimensional (Dimension) images have been screened in movie theaters for more realistic movie screening. For more realistic movie screening, the sound must be output as a three-dimensional sound. The three-dimensional sound is a sound that makes the user feel as if he or she is actually in the situation of the image as the intensity and direction of the sound is set to correspond to the image, more than simply output from the speaker.

In order to realize such a three-dimensional sound, a sound image externalization technology is required to cause sound images to form on the outside of the head of a user seated in a movie theater.

Sound image externalization techniques may be implemented by performing a convolution of a sound signal and a Head Related Transfer Function (HRTF). HRTF represents the impulse response to the sound heard at each location around the user's head.

At present, an environment for forming and controlling a sound field using a plurality of speakers has been provided. Multiple channels are used for the formation and control of the sound field. The sound field is becoming more sophisticated as more channels are used. For example, beyond the existing 5.1 and 7.1 channels, 11.1 channels, 15.1 channels and 31.1 channels are used for the formation of the sound field. In addition, the image externalization technology is creating an environment in which a user can enjoy a three-dimensional sound without a speaker through a plurality of channels through the headphones.

However, the prior art still reproduces only the ambient sound as the surround sound based on the speaker. Prior arts, unlike the three-dimensional image of a three-dimensional movie, have limitations in bringing the sound image to the front or back of the screen. In particular, according to the prior art, unlike a three-dimensional image protruding to the front of the screen, the sound image does not protrude to the front of the screen, thereby providing a complete three-dimensional sound.

Therefore, in light of the trend of rapid popularization of 3D movies, there is a need for a technology of forming sound in full 3D.

One embodiment may provide a method and apparatus for providing editing of three-dimensional sound using parameters.

One embodiment may provide a method and apparatus for adjusting a parameter of three-dimensional sound via a graphical interface.

In one aspect, the method comprises: determining values of one or more parameters for a three-dimensional sound; Generating data of the three-dimensional sound based on the one or more parameters; And outputting a signal comprising the data, wherein the one or more parameters are provided to provide a three dimensional sound associated with a source of sound located in a three dimensional space.

The one or more parameters may include a maximum distance parameter indicative of a maximum distance of an area behind the screen and a maximum threshold parameter indicative of a maximum distance of an area ahead of the screen.

The one or more parameters may include a damper parameter representing a damper of the source.

The one or more parameters may include a source start point parameter indicative of the position of the start point of the source and a source end point parameter indicative of the position of the end point of the source.

The one or more parameters may include a source trace line parameter indicating a line through which the source from the start point to the end point travels.

The source trace line parameter may indicate a plurality of lines connecting the start point and the end point.

The one or more parameters may include speed and acceleration parameters indicative of the speed and acceleration of the source.

The one or more parameters may have different values over time.

The data of the 3D sound may include data of a plurality of channels.

The one or more parameters may be used for each channel of a plurality of channels.

The values of the one or more parameters can be edited via a graphical interface.

On the other side, Determining unit for determining the values of one or more parameters for the three-dimensional sound; A generator configured to generate data of the 3D sound based on the one or more parameters; And an output unit for outputting a signal including the data, wherein the one or more parameters are provided with a three-dimensional sound providing apparatus associated with a source of sound located in a three-dimensional space.

The one or more parameters may have different values over time.

The data of the 3D sound may include data of a plurality of channels.

A method and apparatus are provided for providing editing of three-dimensional sound using parameters.

Methods and apparatus are provided for adjusting parameters of three-dimensional sound through a graphical interface.

1 is a structural diagram of a 3D sound reproducing apparatus according to an example.

2 is a flowchart illustrating a 3D sound reproduction method according to an example.

3 is a block diagram illustrating a primary signal processor according to an example.

4 is a flowchart of object extraction according to an example.

5 is a structural diagram of a 3D sound providing apparatus according to an embodiment.

6 is a flowchart of a 3D sound providing method according to an exemplary embodiment.

7 illustrates a sound mix according to an example.

8 shows a first process of a mixing method according to an example.

9 shows a second process of the mixing method according to an example.

10 shows a third process of a mixing method according to an example.

11 illustrates an overall interface for determining values of one or more parameters for three-dimensional sound according to an example.

12 illustrates a distance setting interface according to an example.

13 illustrates a source setting interface according to an example.

14 illustrates a distance interface according to an example.

15 illustrates a selected point according to an example.

16 illustrates editing a source trace line according to an example.

17 illustrates a screen interface according to an example.

18 illustrates a speed and acceleration interface according to an example.

19 illustrates a sequence interface according to an example.

20 illustrates a locator area of a project according to one embodiment.

21 illustrates an enlarged speed graph according to an example.

22 illustrates an electronic device implementing a 3D sound reproducing apparatus according to an embodiment.

23 is a diagram illustrating an electronic device that implements a 3D sound providing apparatus according to an embodiment.

DETAILED DESCRIPTION For the following detailed description of exemplary embodiments, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled.

Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects. Shape and size of the elements in the drawings may be exaggerated for clarity.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the embodiment, the singular also includes the plural unless specifically stated otherwise in the text. As used herein, “comprises” and / or “comprising” refers to the presence of one or more other components, steps, operations and / or elements. Or it does not exclude the addition, it means that the additional configuration may be included in the scope of the spirit of the exemplary embodiment or the exemplary embodiments. When a component is said to be "connected" or "connected" to another component, the above two components may be directly connected to or connected to each other, It is to be understood that other components may exist in the middle of the two components.

Terms such as first and second may be used to describe various components, but the above components should not be limited by the above terms. The above terms are used to distinguish one component from another component. For example, without departing from the scope of rights, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

In addition, the components shown in the embodiments are shown independently to represent different characteristic functions, and do not mean that each component is composed of only separate hardware or one software component unit. That is, each component is listed as each component for convenience of description. For example, at least two of the components may be combined into one component. In addition, one component may be divided into a plurality of components. The integrated and separated embodiments of each of these components are also included in the scope of the present invention without departing from the essence.

In addition, some of the components may not be essential components for performing essential functions, but may be optional components for improving performance. Embodiments may be implemented including only components necessary to implement the nature of the embodiments, and structures including the optional components, such as, for example, components used only for performance improvement, are also included in the scope of rights.

Hereinafter, in order to enable those skilled in the art to easily implement the embodiments, embodiments will be described in detail with reference to the accompanying drawings. In describing the embodiments, when it is determined that the detailed description of the related well-known configuration or function may obscure the subject matter of the present specification, the detailed description thereof will be omitted.

The three-dimensional sound reproducing apparatus 100 includes a signal receiver 105, a signal detector 110, a primary signal processor 120, a channel allocator 130, a secondary signal processor 140, and a function adjuster 150. The sound image externalization implementer 160, the bypass adjuster 170, the bypass switch 171, the far-range speaker sensing / reproducing unit 180, and the proximity speaker sensing / reproducing unit 190 may be included.

The signal receiver 105, the signal detector 110, the primary signal processor 120, the channel allocator 130, the secondary signal processor 140, the function adjuster 150, and the sound image externalization implementer 160. The functions and operations of the bypass adjusting unit 170, the bypass switching switch 171, the remote speaker sensing / reproducing unit 180, and the proximity speaker sensing / reproducing unit 190 will be described in detail below.

The 3D sound reproducing apparatus 100 may include a remote speaker 181 and a proximity speaker 191. Alternatively, the 3D sound reproducing apparatus 100 and the remote speaker 181 may be connected by wire or wirelessly. The 3D sound reproducing apparatus 100 and the proximity speaker 191 may be connected by wire or wirelessly.

The remote speaker 181 may be a speaker that is not used for sound externalization.

The remote speaker 181 may be a speaker physically separated from the listener of the sound generated by the 3D sound reproducing apparatus 100. The remote speaker 181 may be a loud speaker. The remote speaker 181 may be a main speaker, a surround speaker, or a sealing speaker connected to a receiver. The remote speaker 181 may be a speaker attached to a TV or a desktop speaker attached to a computer. Alternatively, the remote speaker 181 may be a speaker for transative reproduction.

The proximity speaker 191 may be a speaker used for sound externalization.

The proximity speaker 191 may be a speaker in close contact with the listener's ear of the sound generated by the 3D sound reproducing apparatus 100. The proximity speaker 191 may be headphones or earphones. The proximity speaker 191 may be in-ear headphones, on-ear headphones, or over-ear headphones.

In operation 210, the signal receiver 105 may receive a signal transmitted from a sound transmission device.

In operation 220, the signal detector 110 may detect a type of the received signal.

For example, the signal detector 110 may detect whether the received signal is in a format that supports 3D sound, and may detect whether the received signal includes object data. In addition, the signal detector 110 may detect that the received signal is a signal or content such as mono, stereo, 5.1 channel, and 7.1 channel.

As the signal detector 110 detects the type of the transmitted signal, the primary signal processor 120 may determine how to perform processing on the transmitted signal.

The signal detector 110 may detect the type of the received signal using the number of audio channels of the received signal.

For example, when the received signal has two audio channels, the signal detector 110 may detect that the received signal is a stereo signal. If the received signal has six audio channels, the signal detector 110 may detect that the received 5.1 channel signal is received. When the received signal is a signal using a codec, the signal detector 110 may check the information of the channel using the information of the head of the signal, and the type of the received signal using the confirmed channel information. Can be detected.

In operation 230, the signal detector 110 may determine whether to bypass the received signal. Alternatively, the signal detector 110 may determine whether to generate 3D sound or stereo sound. When bypassing the received signal, stereo sound may be produced. If the received signal is not bypassed, three-dimensional sound may be generated. The stereo sound may be a general 2D sound rather than a 3D sound.

For example, when the signal detector 110 detects two audio channels, the signal detector 110 may determine whether to generate three-dimensional sound or stereo sound.

In addition, regardless of the type of the signal detected by the signal detector 110 may determine to generate a stereo sound using the bypass function. The signal detector 110 may determine whether to generate 3D sound or stereo sound based on the setting of the bypass switch 171.

For example, the user of the 3D sound reproducing apparatus 100 may determine whether to generate 3D sound or stereo sound by setting the bypass switch 171.

For example, the bypass changeover switch 171 may be set to on or off. When the bypass switching switch 171 is set to on, the signal detector 110 may generate stereo sound. When the bypass switching switch 171 is set to off, a three-dimensional sound to be described later may be generated.

For example, the bypass changeover switch 171 may be set to "far output", "close output" or off. When the bypass switching switch 171 is set to "far output" or "close output", the signal detector 110 may generate stereo sound. When the bypass switching switch 171 is set to off, a three-dimensional sound to be described later may be generated.

If bypass of the received signal is determined (ie, it is determined that stereo sound is to be produced), step 235 can be performed. If it is determined not to bypass the received signal (ie, it is determined that three-dimensional sound is to be produced), step 245 may be performed.

In operation 235, when the bypass of the received signal is determined, the bypass adjuster 170 may select a speaker to which the received signal is immediately transmitted among the remote speaker 181 and the proximity speaker 191. The bypass adjuster 170 may directly transmit the received signal to one of the one or more remote speakers and the adjacent speaker 191.

For example, when the bypass change switch 171 is set to "far output", the bypass adjuster 170 may select the far speaker 181 and transmit the received signal directly to the far speaker 181. have. When the bypass switching switch 171 is set to "proximity output", the bypass adjuster 170 may select the proximity speaker 191 and transmit the received signal directly to the proximity speaker 191.

In operation 240, the function adjusting unit 150 may perform setting according to the taste of the listener. The setting may affect the operation of the primary signal processor 120. The function adjuster 150 may perform settings for the primary signal processor 120.

For example, the function adjusting unit 150 may set a downmix according to the taste of the listener.

For example, the function adjusting unit 150 may set an upmix according to the taste of the listener.

In operation 245, the primary signal processor 120 may generate one or more separated signals by performing channel separation and object separation on the signal detected by the signal detector 110.

The primary signal processor 120 may extract channel data and object data from the detected signal by separating the channel data and the object data from the detected signal.

Channel data and object data may constitute three-dimensional data. Alternatively, the 3D data may include channel data and object data.

The channel data may be data of channel components recognized by the primary signal processor 120 among the detected signals.

The object data may be data of an object component recognized by the primary signal processor 120 among the detected signals. The object data may include location information about the space, direction information about the motion, size information of the sound, and the like.

The primary signal processor 120 may separate the channel data of the detected signal according to the surround standard.

The primary signal processor 120 may separate and extract automation data of each object for each object of one or more objects.

The primary signal processor 120 may apply processing according to the sensed signal type. In addition, the primary signal processor 120 may perform channel separation and object separation on the detected signal according to the mode of processing determined by the user. For example, the mode may include a stereo mode, a multichannel surround mode, and a three dimensional audio mode including an object. When the stereo mode is set, the primary signal processor 120 may provide channel data of the sensed signal to the channel allocator 130 without additional processing. When the multi-channel surround mode is set, the primary signal processor 120 may separate the channel data and the object data using the object extraction method, and provide the separated channel data and the object data to the channel allocator 130. Can be. When the 3D audio mode including the object is set, the primary signal processor 120 may separate the channel data and the object data according to the channel information and the object information of the detected signal, and separate the separated channel data and the object data. The channel allocator 130 may provide the channel allocation unit 130.

The primary signal processor 120 may separate and extract channel data, object data, and spatial image data by using a decoder when the sensed signal corresponds to 3D sound.

One or more separated signals may include at least some of channel data, object data, and spatial image data.

The primary signal processor 120 may transmit one or more separated signals to the channel allocator 130.

In operation 250, the channel allocator 130 may acquire information about the remote speaker 181 and the proximity speaker 191.

The channel allocator 130 may detect whether the remote speaker 181 is connected to the 3D sound reproducing apparatus 100, and determine the number of remote speakers 181 connected to the 3D sound reproducing apparatus 100. Can be.

The remote speaker detecting / reproducing unit 180 may detect whether the remote speaker 181 is connected or detect the number of the remote speakers 181 connected to the 3D sound reproducing apparatus 100.

The remote speaker detecting / reproducing unit 180 may detect the remote speaker 181 actually connected to the 3D sound reproducing apparatus 100. The remote speaker detecting / reproducing unit 180 may provide the number of the detected remote speakers 181 to the channel allocator 130. The remote speaker detection / reproducing unit 180 may detect the number of remote speakers 181 currently connected to the preamplifier and / or the power amplifier.

The remote speaker detecting / reproducing unit 180 may provide the channel allocating unit 130 with the number of detected remote speakers 181 set by the user.

The channel allocator 130 may detect whether the proximity speaker 191 is connected to the 3D sound reproducing apparatus 100.

The proximity speaker detecting / reproducing unit 190 may detect the proximity speaker 191 actually connected to the 3D sound reproducing apparatus 100. The proximity speaker detecting / reproducing unit 190 may provide the channel allocating unit 130 with information indicating whether the proximity speaker 191 is actually connected to the 3D sound reproducing apparatus 100.

In step 255, allocation of each signal of one or more separate signals may be performed. The channel allocator 130 may allocate each signal of one or more separated signals according to the number of detected one or more remote speakers and whether the proximity speaker 191 is connected. . Here, the assignment may be to classify the signal into one of channel data and object data.

The channel allocator 130 may classify each signal of one or more separated signals into one of channel data and object data. The channel allocator 130 may classify each signal of the one or more separated signals into one of channel data and object data according to the number of detected one or more remote speakers and whether the proximity speaker 191 is connected.

The channel allocator 130 may allocate channel information according to a channel supported by the 3D sound reproducing apparatus 100 and arrange object data.

The channel allocator 130 may separate the channel data into one or more remote speaker channels of the one or more remote speakers, and may separate the object data into a sound externalization channel of the proximity speaker 191.

The channel allocator 130 converts each signal of one or more separated signals according to the number of one or more remote speakers into one or more channels of one or more remote speaker channels of the one or more remote speakers and the sound externalization channel of the proximity speaker 191. Can be sent to.

The far speaker channel may be a channel of the far speaker 181. The sound externalization channel may be a channel of the proximity speaker 191. The far speaker channel may be a channel provided with channel data. The sound externalization channel may be a channel provided with object data. In other words, the remote speaker channel may actually represent channel data, and the audio externalization channel may actually represent object data.

The remote speaker channel can provide separate data in surround formats such as 5.1 channel, 7.1 channel, 8.1 channel, 9.1 channel, 10.2 channel, 11.1 channel, 13.1 channel, 14.2 channel, 15.1 channel, 22.2 channel, 30.2 channel and 31.1 channel. Can be. The remote speaker channel can support all existing surround formats.

The acoustic externalization channel may provide information about data, spatial coordinates, vectors, levels, and the like of the object.

However, the remote speaker channel and the sound image externalization channel may be exchanged with each other. In other words, the channel allocator 130 may use the remote speaker channel as a channel for reproducing object data, and use the sound externalization channel as a channel for reproducing channel data, if necessary.

The channel allocator 130 may transmit one or more separated signals to one or more remote speaker channels using the following allocation rule.

N may represent the number of one or more remote speakers, and n may represent the number of one or more effective channels of one or more separate signals.

1) When N and n are the same, the channel allocator 130 may assign one or more effective channels to one or more remote speaker channels one-to-one.

2) When N is less than n, the channel allocator 130 assigns some effective channels as many as one or more remote speakers among one or more effective channels to one or more remote speaker channels of the one or more remote speakers, respectively. And the remaining valid channels that are not assigned to one or more far channels of the one or more effective channels can be assigned to the audio externalization channel.

3) When N is smaller than n, the channel allocator 130 may perform downmixing of one or more effective channels according to a user's setting.

For example, the channel allocator 130 may perform downmixing on a plurality of effective channels according to the number of one or more remote speakers. The channel allocator 130 may allocate all of the one or more effective channels to only one or more remote speaker channels of the one or more remote speakers through the downmix, but may not allocate the sound externalization channel.

For example, the channel allocator 130 may allocate all of one or more effective channels only to the sound externalization channel through the downmix, but may not assign the one or more remote speaker channels.

4) When N is greater than n, the channel allocator 130 assigns one or more effective channels to some remote speaker channels as many as one or more effective channels of one or more remote speaker channels of the one or more remote speakers, respectively. The effective channel may not be allocated to the remaining remote speaker channel to which each effective channel of the one or more effective channels of the one or more remote speaker channels is not assigned.

5) When N is greater than n, the channel allocator 130 may perform upmix on one or more effective channels according to a user's setting.

For example, the channel allocator 130 may perform upmix on one or more effective channels according to the number of one or more remote speakers. The channel allocator 130 may allocate all of one or more effective channels to one or more remote speaker channels of the one or more remote speakers through the upmix, but may not allocate the sound externalization channel.

For example, the channel allocator 130 may assign all of one or more effective channels to only the sound externalization channel, but may not assign to one or more remote speaker channels.

In operation 255, the function adjusting unit 150 may perform a setting for the secondary signal processing unit 140.

The function adjuster 150 may set a parameter used by the secondary signal processor 140 to perform processing on one or more separated signals. The function adjusting unit 150 may receive a user input for setting a parameter.

For example, the function adjusting unit 150 may adjust the size of the space according to the environment of the listener, and may adjust the size of the virtual space according to the taste of the listener.

For example, the function adjusting unit 150 may set sound effects such as an equalizer, a compressor, and a delay for correcting the listener's environment.

For example, the function adjuster 150 may set an adjustment of the level of each signal of one or more separate signals for correction of the listener's listening environment.

For example, the function adjuster 150 may set positions of one or more remote speakers for correction of the listener's listening environment.

In operation 260, the secondary signal processor 140 may perform processing on one or more separated signals.

The secondary signal processor 140 may set the tone and level of each signal of one or more separated signals according to the taste of the listener through processing, and may add a virtual space image to each signal.

The secondary signal processor 140 may process each signal of one or more separated signals in accordance with a space according to the environment of the listener or a virtual space according to the taste of the listener. Each signal may be one of channel data and object data.

When the proximity speaker 191 is not detected, the secondary signal processor 140 may mix object data into channel data.

For example, the secondary signal processor 140 may give a spatial image to each signal of one or more separated signals. The secondary signal processor 140 may perform a function of a room simulator through the provision of a spatial image.

For example, the secondary signal processor 140 may apply a sound effect to each signal of one or more separated signals according to the listener's listening environment.

For example, the secondary signal processor 140 may adjust the level and tone of each signal of one or more separated signals according to the listener's listening environment.

For example, the secondary signal processor 140 may adjust the levels of each of the one or more far speakers and the near speaker of the one or more far speakers.

For example, the secondary signal processor 140 may adjust positions of one or more remote speakers.

For example, the secondary signal processor 140 may adjust an artificial distance between one or more remote speakers and a proximity speaker.

For example, the secondary signal processor 140 may mix levels of one or more remote speakers and a proximity speaker.

In operation 270, the remote speaker sensing / reproducing unit 180 may output the channel data processed by the secondary signal processing unit 140 to one or more remote speakers.

In operation 280, the sound externalization implementer 160 may implement sound externalization of the object data processed by the secondary signal processor 140. The sound externalization implementer 160 may generate data on which sound externalization is implemented by implementing sound externalization.

The sound externalization implementer 150 may implement sound externalization by rearranging information represented by the object data processed by the secondary signal processor 140 according to the spatial image represented by the spatial image data.

The channel of the object data may not be limited. In other words, the object data may be input to the sound externalization implementer 160 through one or more infinite channels.

Sound externalization data generated by the sound externalization implementation unit 160 may include two or more channels.

In operation 290, the proximity speaker detection / reproducing unit 190 may reproduce data in which sound image externalization is implemented using the proximity speaker 191. The proximity speaker detection / reproducing unit 190 may output data on which sound image externalization is implemented to the proximity speaker 191.

The proximity speaker sensing / reproducing unit 190 may process the processing of the sound externalization of the data on which the sound externalization is implemented.

The proximity speaker sensing / reproducing unit 190 may output a result of the processing of the external sound image to the proximity speaker 191.

The primary signal processor 120 includes a channel detector 310, a level detector 320, a threshold controller 330, a channel comparator 340, an object data allocation controller 350, and channel data. The generator 360 may include an object data generator 370, a channel mixer 380, and an object data mixer 390.

Channel detector 310, level detector 320, threshold controller 330, channel comparator 340, object data allocation controller 350, channel data generator 360, object data generator 370 ), Functions and operations of the channel mix unit 380 and the object data mix unit 390 will be described in detail below.

4 is a flowchart of object extraction according to an example.

In the following

steps

410, 415, 420, 425, 430, 440, 445, 450 and 455, the signal detector 110 detects a general surround signal rather than a 3D audio signal, and the bypass adjuster 170. ) May be performed when the bypass function is not used. Step 245 described above with reference to FIG. 2 may include

steps

410, 415, 420, 425, 430, 440, 445, 450, and 455.

Through the following

steps

410, 415, 420, 425, 430, 440, 445, 450, and 455, the primary signal processor 120 may provide compatibility with existing content. Can be extracted.

In operation 410, the channel detector 310 may divide the signal detected by the signal detector 110 into one or more channels according to the surround format of the detected signal. Each channel of the one or more channels may have a unique channel number.

When the signal detected by the signal detector 110 includes a surround channel, the channel detector 310 may separate the detected signal into one or more channels according to the surround format of the surround channel.

For example, the surround channel may include a 5.1 channel, a 7.1 channel, and the like. For example, a surround channel may include all channels that do not have a height component. For example, the surround channel may include 6.1 channels, 8.1 channels, and 9.1 channels.

If the signal detected by the signal detector 110 does not include a surround channel, the channel detector 310 may bypass the detected signal to the channel allocator 130. For example, if the sensed signal is a signal of a stereo channel, the sensed signal may not include a surround channel.

In step 415, the threshold controller 330 may determine the threshold level.

In operation 420, the level detector 320 may detect levels of one or more channels, and may divide one or more channels into one or more high level channels and one or more low level channels.

Each channel of the one or more high level channels may be a channel having a level above the threshold level. Each channel of the one or more low level channels may be a channel having a level less than the threshold level. The level detector 320 may classify a channel having a level higher than or equal to a threshold level among the one or more channels as a high level channel, and classify a channel having a level smaller than the threshold level as a low level channel. Basically, high level channels can be used as object data and low level channels can be used as channel data.

The level detector 320 may transmit one or more high level channels and one or more low level channels to the channel comparator 340.

In operation 425, the object data allocation controller 350 may set whether to use dialogue as channel data or object data.

In operation 430, the channel comparator 340 may set whether to use data of each channel of one or more channels as channel data or object data.

The channel comparator 340 may set one or more low level channels to be used as channel data.

The channel comparator 340 may finally determine whether to use data of one or more high level channels as object data or channel data through channel comparison with respect to one or more high level channels.

The channel comparator 340 may be configured to use the dialogue channel among the one or more high level channels as one of the object data and the channel data according to the setting of the object data allocation controller 350. In other words, the channel comparator 340 may determine which channel among the one or more high level channels is used as the remote speaker channel or the audio externalization channel according to the setting of the object data allocation controller 350.

For example, when dialogue is set to be used as channel data, the channel comparator 340 may set the channel of dialogue among the one or more high level channels to be used as channel data. When the dialogue is set to be allocated to the object data, the channel comparator 340 may set the dialogue channel among the one or more high level channels to be used as the object data.

For example, when data is extracted only from a center channel among one or more channels, the center channel may be a metabolic channel with a probability of 90% or more.

For example, when the same data is extracted from a plurality of channels among one or more high level channels, the channel comparator 340 may set the plurality of channels from which the same data is extracted to be used as object data.

For example, when data for one channel is extracted from a channel other than the center channel among the one or more high level channels, the extracted data may be a special effect. The special effect may be object data. Therefore, when data for one channel is extracted from a channel other than the center channel among one or more high level channels, the channel comparator 340 may set the channel from which the data for one channel is extracted as object data. Can be.

For example, when data is extracted only from a left channel and a right channel of one or more high level channels, the extracted data may be music data or ambience with a large volume. Music data or ambience with large volume may be channel data. Therefore, when data is extracted only from the left channel and the right channel among the one or more high level channels, the channel comparator 340 may set the left channel and the right channel to be used as the channel data.

In operation 440, the channel data generator 360 may generate channel data using data of a channel set to be used as channel data among one or more channels.

The channel data generator 360 may prevent data of a channel having a level higher than or equal to the threshold level set by the threshold controller 330 from being reproduced in the remote speaker 181.

The channel data generator 360 may allow the data of the channel set to be used as the channel data to be reproduced through the remote speaker channel according to the setting method of the channel comparator 340.

In operation 445, the channel mixer 380 may perform level compensation on the channel data, which is reduced according to the extraction of the object data.

The channel mix unit 380 may apply a release parameter in level compensation. By applying the release parameter, a break between the object data and the sound in the threshold region may be prevented.

In operation 450, the object data generator 370 may generate object data using data of a channel set to be used as object data among one or more channels.

The object data generator 370 may allow only the data of the channel having a level equal to or greater than the threshold level set by the threshold controller 330 to be reproduced in the proximity speaker 191.

The channel data generator 360 may allow the data of the channel set to be used as the object data to be reproduced through the sound externalization channel according to the setting method of the channel comparator 340.

In operation 455, the object data mixing unit 390 may perform level compensation on the object data, which is reduced according to the extraction of the object data.

Object data may be extracted from channel components. Accordingly, the object data mixing unit 390 may adjust the object data such that only pure object data can be reproduced by using a parameter such as a gate.

In addition, in order to reproduce natural object data, the object data mix unit 390 may apply a gate time to adjust the object data so that a mix between the channel data and the object data occurs naturally.

The 3D sound providing apparatus 500 may include a setting unit 510, a generating unit 520, and an output unit 530. Functions and operations of the setting unit 510, the generating unit 520, and the output unit 530 will be described in detail below.

In operation 610, the setting unit 510 may determine values of one or more parameters for the 3D sound. One or more parameters may be associated with a source of sound located in three-dimensional space.

The values of the one or more parameters can be edited via the graphical interface described below.

Step 610 may be performed repeatedly as the values of one or more parameters change.

In operation 620, the generator 520 may generate data of the 3D sound based on one or more parameters.

The generator 520 may generate data of the 3D sound by reflecting one or more parameters.

The data of the 3D sound may include data of a plurality of channels. One or more parameters may be used for each channel of the plurality of channels.

In operation 630, the output unit 530 may output a signal including data of 3D sound. The output signal may be received and used by the 3D sound reproducing apparatus 100.

Data of the 3D sound of the output signal may correspond to the object data described above with reference to FIGS. 2 to 4. In other words, the one or more parameters may correspond to positional information on the space of the object data, direction information on the movement, and size information of the sound, respectively. For example, one of the one or more parameters related to the location of the source may correspond to location information about the space. The parameter related to the movement of the source among the one or more parameters may correspond to the direction information about the movement. Among the one or more parameters, parameters representing damper, mix, diffuse, gain, and tail of the source may correspond to loudness information.

For each of the one or more parameters, it is described in detail below.

7 illustrates a sound mix according to an example.

In FIG. 7, screens and thresholds are shown. The threshold may represent a bulge limit line. D may represent a difference between a general mixing line and a mixing line according to an example of the present invention. The screen may be a location where the image is formed.

In a typical sound mix, a mix based on the screen can be made. In other words, the delay at the point of the screen may be zero.

Through the mix, adding a delay of threshold d to the overall signal of the sound, the sound can be represented substantially behind the distance d 'corresponding to the threshold from the screen. In operation 610, the setting unit 510 may set the threshold d as a parameter. In operation 620, the generator 520 may apply the delay d as much as the threshold d to the entire signal of the three-dimensional sound so that the three-dimensional sound is represented by the distance corresponding to the threshold d.

Physically, the delay cannot be pulled forward. However, when the viewer's hearing recognizes the distance d ', the viewer may recognize the sound according to the position of the screen according to the psychological factor of the viewer. That is to say, as a result, the viewer can perceive the position of the screen as if a sound was produced.

Using the mixing method as described above, the sound can be projected toward the seat where the viewer is located by the distance of D. However, in order for the sound to protrude, the source of the sound may have a velocity component.

Hereinafter, the source may represent a source of sound. In addition, the source may correspond to the above-described object.

8 shows a first process of a mixing method according to an example.

As shown in FIG. 8, in step 620, the generation unit 520 may enlarge the spatial image by applying a threshold d delay to the entire signal of the 3D sound.

In operation 620, the generation unit 520 may reduce the pre-delay value by a distance d corresponding to the increased space in the reverb process. By reducing the value of the pre-delay, the generator 520 may perform mixing to prevent the viewer from recognizing the increased space.

9 shows a second process of the mixing method according to an example.

The delay may not be applied to the rear of the screen in order to minimize distortion of the space and recognition of the distortion of the viewer.

If the viewer is sufficiently adapted to the space, the delay applied by d may be ignored and a sound image may be formed relative to the screen.

10 shows a third process of a mixing method according to an example.

As the sound image is formed based on the screen, the sound protruding from the screen can be expressed as a result.

When sound correction for the back side of the screen is performed, a superior effect can be obtained.

In the following, a graphical interface for front screen panning is described. The parameters can be edited in nonlinear form by the graphical interface below. In addition, the speed and acceleration of the sound source can be imparted by the graphic interface below, and sound effects on the sound source can be processed. Sound effects can include Doppler processing.

The overall interface 1100 may include at least some of the distance setting interface 1110, the source setting interface 1120, the distance interface 1130, the screen interface 1140, the speed interface 1150, and the sequence interface 1160. Can be.

In each interface, the starting point of the source can be represented by S. The end point of the sound source may be represented by E. In the following, the starting point of the source may be outlined as the starting point. The end point of the source can be outlined as the end point.

12 illustrates a distance setting interface according to an example.

The distance setting interface 1110 includes a distance knob 1210, a distance value 1215, a threshold knob 1220, a threshold value 1225, a damper knob 1230, a damper value 1235, and a mix knob 1240. , A mix value 1245, and a limit reset button 1250.

The sound source may be formed behind the screen via the distance setting interface 1210.

The distance value 1215 can represent the maximum distance of the area behind the screen. The distance knob 1210 can adjust the distance value 1215.

One or more parameters may include a maximum distance parameter that indicates a maximum distance of the area behind the screen.

For example, the maximum distance parameter may have a value of 0 to 1000. The default value of the maximum distance parameter may be zero.

The range of values of the start point and the end point of the source may be changed according to the maximum distance parameter. For example, when the start and end points of the source are located behind the screen, the distance from the screen of the start and end points of the source cannot be greater than the value of the maximum distance parameter.

The threshold value 1225 may represent the distance of the area in front of the screen. Threshold knob 1220 can adjust threshold value 1225.

One or more parameters may include a maximum threshold parameter that indicates a maximum distance of the area in front of the screen.

For example, the maximum threshold parameter may have a value between 0 and 30. The default value of the maximum threshold parameter may be zero.

The range of values of the start point and the end point of the source may be changed according to the maximum threshold parameter. For example, when the start and end points of the source are located in front of the screen, the distance from the screen of the start and end points of the source cannot be greater than the value of the maximum threshold parameter.

The damper value 1235 may represent the damper of the source. The damper knob 1230 can adjust the damper value 1235.

The damper may exhibit a velocity effect.

One or more parameters may include a damper parameter indicating a damper of the source.

For example, the damper parameter may have a value of 0 to 1. The default value of the damper parameter may be zero.

In operation 620, the generator 520 may maximize the level effect and the reverberation effect of the 3D sound in consideration of the speed of the sound by using the damper parameter. In addition, the generator 520 may perform the compression of the 3D sound by using the damper parameter, and may harden the 3D sound through the compression. The generator 520 may cut the upper part and the lower part of the 3D sound by applying a filter to the 3D sound by using the damper parameter.

The size of the damper may be indicated as one or more concentric circles at speed interface 1150.

The mix value 1245 may represent a balance between the sound generated by the processing and the original sound. The mix knob 1240 may adjust the mix value 1235.

One or more parameters may include a mix parameter indicative of a balance between the sound produced by the processing and the original sound.

For example, the mix parameter may have a value between 0 and 100. The default value of the damper parameter may be 100. The mix parameter having a minimum value may indicate that only the sound generated by the processing is output. A mix parameter having a maximum value may indicate that only the original, unprocessed sound is output. That is to say that the mix parameter has a maximum value may indicate that the sound is bypassed.

In operation 620, the generator 520 may adjust a balance between the sound generated by the processing and the original sound according to the value of the mix parameter.

The limit reset button may be used to return one or more parameters of the distance setting interface 1110 to the default value.

13 illustrates a source setting interface according to an example.

The source setting interface 1120 includes a source start point knob 1310, a source start point value 1315, a source start point diffuse knob 1320, a source start point diffuse value 1325, a source end point knob 1330, a source It may include an endpoint value 1335, a source endpoint diffuse knob 1340, a source endpoint diffuse value 1345, and a source reset button 1350.

Through the source setting interface 1120, the position of the start point of the source, the position of the end point of the source, the diffuse of the start point of the source and the diffuse of the end point of the source may be determined. The diffuse may indicate the degree of diffusion. The value of diffuse can represent the radius of diffusion.

The source start point value 1315 may indicate the position of the start point of the source. The source start point knob 1310 may adjust the source start point value 1315.

One or more parameters may include a source start point parameter that indicates the location of the start point of the source. The location of the start point of the source may include the distance from the listening area to the start point of the source and / or the coordinates of the start point of the source.

The source start point parameter can optionally be activated. For example, the source start point knob 1310, the source start point value 1315, and the source start point parameter may be activated when the source start point is generated. The starting point of the source may be generated by user manipulation at one of the distance interface 1130, the screen interface 1140, and the speed interface 1150.

For example, the minimum value of the source start point parameter may be zero. The maximum value of the source start point parameter may be the sum of the maximum distance parameter and the maximum threshold parameter.

The source start point diffuse value 1325 may represent the diffuse of the source start point. The source start point diffuse knob 1320 may adjust the source start point diffuse value 1325.

One or more parameters may include a source start point diffuse parameter that indicates a diffuse of the start point of the source. The diffuse at the beginning of the source may represent the degree of dispersion of the sound at the beginning of the source.

The source start point diffuse parameter can optionally be activated. For example, the source start point diffuse knob 1320, the source start point diffuse value 1325, and the source start point diffuse parameters may be activated when the source start point is generated. The starting point of the source may be generated by user manipulation at one of the distance interface 1130, the screen interface 1140, and the speed interface 1150.

For example, the source start point diffuse parameter may have a value of 0 to 100. The default value of the source start point diffuse parameter may be zero.

In operation 620, the generator 520 may set the degree of dispersion of the sound at the start point of the source using the source start point diffuse parameter. The same delay can be applied to the distributed sound. The distributed sound may be reproduced by the proximity speaker 191.

The degree of dispersion by the source start point diffuse parameter may be displayed in the screen interface 1140.

The source endpoint value 1335 may indicate the location of the endpoint of the source. The source endpoint knob 1330 may adjust the source endpoint value 1335.

One or more parameters may include a source endpoint parameter that indicates the location of the endpoint of the source. The location of the end point of the source may include the distance from the listening area to the end point of the source and / or the coordinates of the end point of the source.

The source endpoint parameter can optionally be activated. For example, the source endpoint knob 1330, the source endpoint value 1335, and the source endpoint parameter may be activated when the endpoint of the source is generated. The end point of the source may be generated by user manipulation at one of the distance interface 1130, the screen interface 1140, and the speed interface 1150.

For example, the minimum value of the source endpoint parameter may be zero. The maximum value of the source endpoint parameter may be the sum of the maximum distance parameter and the maximum threshold parameter.

The source endpoint diffuse value 1345 may represent the diffuse of the source endpoint. The source endpoint diffuse knob 1340 may adjust the source endpoint diffuse value 1345.

One or more parameters may include a source endpoint diffuse parameter that indicates a diffuse of the endpoint of the source. The diffuse at the end of the source may indicate the degree of dispersion of the sound at the end of the source.

The source endpoint diffuse parameter can optionally be activated. For example, the source endpoint diffuse knob 1340, the source endpoint diffuse value 1345, and the source endpoint diffuse parameters can be activated when the source endpoint is generated. The end point of the source may be generated by user manipulation at one of the distance interface 1130, the screen interface 1140, and the speed interface 1150.

For example, the source endpoint diffuse parameter may have a value between 0 and 100. The default value of the source endpoint diffuse parameter may be zero.

In operation 620, the generator 520 may set the degree of dispersion of the sound at the end point of the source using the source end point diffuse parameter. The same delay can be applied to the distributed sound. The distributed sound may be reproduced by the proximity speaker 191.

The degree of dispersion by the source endpoint diffuse parameter may be displayed in the screen interface 1140.

14 illustrates a distance interface according to an example.

The distance interface 1130 includes a maximum distance value 1410, a distance value control 1415, a maximum threshold value 1420, a threshold value control 1425, a unit 1430, a source start point 1440, a source start It may include a point distance 1445, a source end point 1450, a source end point distance 1455, a source trace line 1460, a screen line 1470, and a listening area 1480.

The distance interface 1130 may visually show the parameters set in the distance setting interface 1110. The distance interface 1130 may reflect the increase or decrease of the unit without changing the appearance.

The maximum distance value 1410 may represent a maximum distance value inside the screen. Here, the inside of the screen may represent an area behind the screen based on the viewer. The maximum distance value 1410 may represent a value of the maximum distance parameter. The distance value into the interior of the screen of the source start point 1240 and the distance value into the interior of the screen of the source end point 1250 may be limited to below the maximum distance value 1410.

The distance interface 110 may increase or decrease the unit of the maximum distance value 1410. The unit of the maximum distance value 1410 may be meters or feet.

The distance value control 1415 may be appropriately spaced scales generated according to the maximum distance value 1410.

The maximum threshold value 1420 may represent a maximum distance value outside of the screen. Here, the outside of the screen may represent an area in front of the screen based on the viewer. The maximum threshold value 1420 may represent a value of the maximum threshold parameter. The distance value to the outside of the screen of the source start point 1440 and the distance value to the outside of the screen of the source end point 1450 may be limited to below the maximum distance value 1420.

The distance interface 1130 may increase or decrease the unit of the maximum threshold value 1420. The unit of the maximum threshold value 1420 may be meters or feet.

Threshold value control 1425 may be appropriately spaced scales generated according to maximum threshold value 1420.

The unit 1430 may indicate a value displayed on the distance interface 1130 or a unit of parameters displayed on the distance interface 1130. Through a global setting, one of the meters and feet can be selected as the unit. The default value of the unit may be meters.

The source start point 1440 may indicate the start point of the source. The source start point 1440 may indicate a value of a source start point parameter.

The user may generate the source start point 1440 by manipulating the input device within the dotted line of the area of the distance interface 1130. The location specified by the manipulation of the input device may represent the source start point 1440. According to the generation of the source starting point 1440, a starting point of the source may be generated, and a source starting point knob 1310, a source starting point value 1315, a source starting point diffuse knob 1320, a source starting point diffuse value 1325 and the source start point diffuse parameter may be activated.

For example, the input device can include a keyboard and / or a mouse. Operation of the input device may include clicking, double-clicking, dragging, and dragging and dropping of a mouse, and may include pressing a specific key of the keyboard.

The source start point 1440 may display an X coordinate and a Y coordinate among coordinates of the start point of the source. The value of the Z coordinate of the start point of the source may not be reflected at the source start point 1440.

The user may move the source start point 1440 to a desired position through manipulation of the input device. The user may delete the source start point 1440 through an operation of the input device.

The source start point distance 1445 can represent the distance between the screen and the source start point 1440. The source start point distance 1445 may represent a value of a source start point parameter.

The source start point distance 1445 may be displayed as a horizontal line and may be displayed as a value located at the end of the horizontal line.

The source endpoint 1450 may represent an endpoint of the source. The source endpoint 1450 may represent a value of the source endpoint parameter.

The user may generate the source endpoint 1450 by manipulating the input device within a dashed line of the area of the distance interface 1130. The location specified by the manipulation of the input device may represent the source end point 1450. Source end point 1450 may be generated after generation of source start point 1440. The user may generate a source start point 1440 and a source end point 1450 through manipulation of the input device. Also, after the source start point 1440 is generated, if the value of another parameter is adjusted, the source end point 1450 may be automatically generated. In this case, the location of the generated source endpoint 1450 may be the center of the screen line 1470.

Depending on the generation of the source end point 1450, an end point of the source may be generated, the source end point knob 1330, the source end point value 1335, the source end point diffuse knob 1340, and the source end point diffuse value. 1345 and the source endpoint diffuse parameter may be activated.

The source end point 1450 may display an X coordinate and a Y coordinate among coordinates of the end point of the source. The value of the Z coordinate of the end point of the source may not be reflected in the source end point 1450.

The user may move the source end point 1450 to a desired position through manipulation of the input device. The user may delete the source endpoint 1450 through manipulation of the input device.

The source endpoint distance 1455 may represent the distance between the screen and the source endpoint 1450. The source endpoint distance 1455 may represent a value of the source endpoint parameter.

The source end point distance 1455 may be displayed as a horizontal line and may be displayed as a value located at the end of the horizontal line.

The source trace line 1460 may represent a line along which the source moves from the start point of the source to the end point of the source.

One or more parameters may include a source trace line parameter indicating the line through which the source travels from the start point of the source to the end point of the source.

The line through which the source moves may include a plurality of lines. The source trace line parameter may indicate a plurality of lines connecting the start point of the source and the end point of the source.

In operation 620, the generator 520 may move the position of the source using the source trace line parameter.

The user may change the shape of the source trace line 1460 through manipulation of the input device. For example, the user may divide one line of the source trace line 1460 into two connected lines through manipulation of the input device.

The distance interface 1130 may indicate a point moving along the source trace line 1460 according to the processing length set in the sequence interface 1160.

The screen line 1470 may indicate a point where a screen on which an image is displayed is located. The screen line 1470 may be a reference for the distance of the source start point 1440 and the distance of the source end point 1450.

The listening area 1480 may represent an area where the viewer is located. Also, the listening area may indicate the direction of the viewer.

15 illustrates a selected point according to an example.

At distance interface 1330, the user can select one point of source trace line 1460.

The selected point 1510 may represent a point selected by the user. When a point is selected by the user, a distance value of the selected point may be displayed. The distance value 1520 may represent a distance value of a point selected by the user. In addition, the distance value may represent the distance between the selected point and the screen.

16 illustrates editing a source trace line according to an example.

At distance interface 1130, a user can edit source trace line 1460. The value of the source trace line parameter may be set by editing the source trace line 1160.

The user may move the position of the selected point 1510 through manipulation of the input device. In FIG. 16, the moved selected point 1610 is shown.

As the position of the selected point moves, the source trace line 1460 may change. For example, the line in which the selected point 1510 of one or more lines of the source trace line 1460 is located is two lines in which the moved selected point 1610 is the end point as the selected point 1510 moves. It can be divided into Here, the first of the two lines may be a line from the start point of the line where the selected point 1510 is located to the selected point 1610 moved. The second of the two lines may be a line from the moved selected point 1610 to the end point of the line where the selected point 1510 is located.

When one of the one or more lines of the source trace line is divided, in step 610, the setting unit 510 may set the value of the source trace line parameter to reflect the division of the line.

17 illustrates a screen interface according to an example.

The screen interface 1140 may provide editing of the value of the X coordinate and the value of the Y coordinate for the start point and the end point of the source.

The screen interface 1140 may display the value of the X coordinate and the value of the Y coordinate among the coordinates of the start point of the source, and display the value of the X coordinate and the value of the Y coordinate among the coordinates of the end point of the source. have.

The screen interface 1140 may include a panning area 1710, a source start point 1720, a source end point 1730, a diffuse circle 1740, and a source trace line 1750.

The panning area 1710 may represent an area required for displaying the source start point 1720 and the source end point 1730.

The source start point 1720 may represent the coordinates of the start point of the source. The source start point 1720 may indicate an X coordinate and a Y coordinate among coordinates of the start point of the source. The function of the source start point 1720 may correspond to the function of the source start point 1440 described above.

The source end point 1730 may represent the coordinates of the end point of the source. The source end point 1730 may indicate an X coordinate and a Y coordinate among coordinates of the end point of the source. The function of the source endpoint 1730 may correspond to the function of the source endpoint 1450 described above.

The radius of the diffuse circle 1740 may represent the value of the diffuse of the start point of the source and / or the end point of the source. When the source start point 1720 is selected by a user's manipulation, the radius of the diffuse circle 1740 may represent only the value of the diffuse of the start point of the source. The value of the diffuse of the start point of the source may represent the value of the source start point diffuse parameter. When the source end point 1730 is selected by a user's manipulation, the radius of the diffuse circle 1740 may represent only the value of the diffuse of the end point of the source. The value of the diffuse of the end point of the source may represent the value of the source endpoint diffuse parameter.

The source trace line 1750 may indicate a line from which the source moves from the start point of the source to the end point of the source. The function of the source trace line 1750 may correspond to the function of the source trace line 1460 described above.

18 illustrates a speed and acceleration interface according to an example.

Velocity interface 1150 may provide editing of the value of the Y coordinate and the value of the Z coordinate for the start point of the source and the end point of the source.

The velocity interface 1150 may display the value of the Y coordinate and the value of the Z coordinate among the coordinates of the start point of the source, and display the value of the Y coordinate and the value of the Z coordinate among the coordinates of the end point of the source. have.

Speed interface 1150 includes screen line 1810, maximum distance value 1820, distance value control 1825, maximum threshold value 1830, threshold value control 1835, listening area 1840, damper value 1850, source start point 1860, source end point 1870, and source trace line 1880.

Screen line 1810 may correspond to screen line 1470 described above. However, screen line 1470 may be displayed for the X and Y coordinates, while screen line 1810 may be displayed for the Y and Z coordinates.

The maximum distance value 1820 may correspond to the maximum distance value 1410 described above. However, the maximum distance value 1410 may be displayed for the X coordinate and the Y coordinate, while the maximum distance value 1820 may be displayed for the Y coordinate and the Z coordinate.

Distance value control 1825 may correspond to distance value control 1415 described above. However, distance value control 1415 may be displayed for X and Y coordinates, while distance value control 1825 may be displayed for Y and Z coordinates.

The maximum threshold value 1830 may correspond to the maximum threshold value 1420 described above. However, the maximum threshold value 1420 may be displayed for the X coordinate and the Y coordinate, while the maximum threshold value 1830 may be displayed for the Y coordinate and the Z coordinate.

Threshold value control 1835 may correspond to threshold value control 1425 described above. However, the threshold value control 1425 can be displayed for the X and Y coordinates, while the threshold value control 1835 can be displayed for the Y and Z coordinates.

The listening area 1840 may correspond to the aforementioned listening area 1480. However, while the listening area 1480 is displayed with respect to the X coordinate and the Y coordinate, the listening area 1840 may be displayed with respect to the Y coordinate and the Z coordinate.

The damper display unit 1850 may indicate the above-described damper value. In other words, the shape of the damper display unit may indicate a value of the damper parameter. The damper indicator 1850 may include one or more concentric circles. One or more concentric circles may be shown only in part. The number, shape and radius of one or more concentric circles may represent a damper applied to the source and may vary depending on the value of the damper parameter. For example, an ellipse may be formed based on the threshold line as the value of the damper parameter increases.

The source start point 1860 may correspond to the source start point 1440 described above. However, the source start point 1440 may be displayed for the X and Y coordinates, while the source start point 1860 may be displayed for the Y and Z coordinates.

Source endpoint 1870 may correspond to source endpoint 1450 described above. However, the source end point 1450 may be displayed for the X and Y coordinates, while the source end point 1870 may be displayed for the Y and Z coordinates.

Source trace line 1880 may correspond to source trace line 1460 described above. However, the source trace line 1460 may be displayed for the X and Y coordinates, while the source trace line 1880 may be displayed for the Y and Z coordinates.

19 illustrates a sequence interface according to an example.

The sequence interface 1160 may provide editing of the value of the Y coordinate and the value of the Z coordinate for the start point and the end point of the source.

The sequence interface 1160 may be linked with the source start point 1440 and the source end point 1450 of the distance interface 1130. In other words, the information about the source displayed in the sequence interface 1160 may be displayed from the Y coordinate and the Z coordinate of the source start point 1440, and may be displayed from the Y coordinate and the Z coordinate of the source end point 1450. .

In addition, the sequence interface 1160 may indicate a damper at an end point of the source.

The sequence interface 1160 includes a timeline 1910, a project cursor 1915, a sequence marker 1920, a time stretch switch 1925, a speed length 1930, a time stretch grid line 1935, a speed graph 1940. , Gain graph (1945), tail (1950), tail (1950), velocity length (1955), timeline speed (1960), timeline gain (1965), tail length (1970), tail end A point 1975 and sequence editing area 1980 may be included.

The timeline 1910 may represent a timeline according to a length set in the sequence interface 1160. Here, the timeline may represent the flow of time from the time of the start point of the source to the time of the end point of the source.

The start point and the end point of the timeline 1910 may be set to be the same as the start point and the end point of the locator area of the projector.

Timeline 1910 may represent an application for pre-roll and / or post-roll. The time at which the preroll and / or postroll is applied can be set globally.

The project cursor 1915 may operate in synchronization with the cursor of the host program. For example, when the cursor of the host program moves, the project cursor 2015 may also move along the cursor of the host program.

One or more parameters may have different values over time. The point indicated by the project cursor 1915 in the timeline 1910 may indicate a reference time in displaying a parameter value.

The sequence marker 1920 may be a button for setting a start point and an end point of the timeline 1910. When the user selects the sequence marker 1920 through manipulation of the input device, the timeline 1910 may be set in the same manner as the locator of the project. In other words, when the sequence marker 1920 is selected, the time of the start point locator may be set to the start point of the timeline 1910, and the time of the end point locator may be set to the end point of the timeline 1910. .

If the locator is not set in the host program, the sequence marker 1920 may be deactivated.

The time stretch switch 1925 can create an additional timeline on the line of velocity length 1930. The user can set the time stretch switch 1925 to ON through manipulation of the input device. If time stretch switch 1925 is set to on, an additional time line may be created in the line of speed length 1930.

The user can adjust the interval of the additional timeline by operating the input device. As the interval of additional timelines becomes wider, the passage of time can be faster. As the intervals of additional timelines become narrower, the passage of time may be slower.

When an additional timeline is created, the project cursor 1915 can reflect the passage of time based on the additional timeline.

The user may set the time stretch switch 1925 to OFF through manipulation of the input device. If time stretch switch 1925 is set to off, an additional time line may disappear from the line of speed length 1930. If the additional timeline disappears, the project cursor 1915 may reflect the passage of time relative to the timeline 1910.

The velocity length 1930 may represent the length of time that the process takes. The length of velocity length 1930 may be the same as the length of timeline 1910. For example, the starting point of velocity length 1930 may correspond to the starting point locator of the project. The end point of velocity length 1930 may correspond to the end point locator of the project. The velocity length 1930 may have a value corresponding to the length generated by the sequence marker 1920.

The time stretch grid line 1935 can indicate the extent of stretch in time.

The time stretch grid line 1935 may be generated by manipulation of a user's input device. The user may manipulate the input device to change the position of the points of the time stretch grid line 1935. As the location of the point changes, the spacing of the grid lines may change.

The speed graph 1940 may represent the speed of the source and the acceleration of the source along the timeline. The speed graph 1940 may represent a change in the speed of the source from the start point of the source to the end point of the source.

One or more parameters may include a speed parameter indicative of the speed of the source and may include an acceleration parameter indicative of the acceleration of the source. The speed parameter may indicate the speed of the source over time. The acceleration parameter may represent the acceleration of the source over time. Alternatively, the one or more parameters may include speed and acceleration parameters indicative of the speed and acceleration of the source.

In operation 620, the generator 520 may generate the 3D sound by reflecting the speed of the source using the speed parameter. In addition, the generator 520 may generate the 3D sound by reflecting the acceleration of the source using the acceleration parameter. Alternatively, the generator 520 may generate the 3D sound by reflecting the speed and acceleration of the source using the speed and acceleration parameters.

The user may enlarge the speed graph 2040 by manipulating the input device.

The speed graph 1940 may automatically calculate the speed of the source linearly over time once the start and end points are determined, and display the calculated speed of the source as a graph. In the speed graph 1940, the height of a point may represent the speed of the source at the time corresponding to the point. In the speed graph 1940, the slope of a point may represent the acceleration of the source at the time corresponding to the point.

The user may move one point on the speed graph to another position by manipulating the input device. As one point moves to another position, the shape of the graph may change, and the speed of the source may change according to the changed shape of the speed graph.

The gain graph 1945 may represent a change in gain of the source along the timeline. The gain graph 1945 may represent a change in gain from the start point of the source to the end point of the source.

One or more parameters may include a gain parameter indicative of the gain of the source. The gain parameter may represent a gain of a source over time.

In operation 620, the generator 520 may generate the 3D sound by reflecting the gain of the source using the gain parameter.

The gain graph 1945 may automatically calculate the gain of the source linearly over time when the start point and the end point are determined, and display the calculated gain of the source as a graph. In the gain graph 1945, the height of a point may represent the gain of the source at the time corresponding to the point.

The user can apply the value of the speed of the source to the gain of the source through the manipulation of the input device.

The tail graph 1950 may represent the tail of the source. The tail of the source may occur after the end point of the source. Basically, 3 seconds of reverberation can occur as the tail.

One or more parameters may include a tail parameter indicating a tail of the source. The tail parameter may indicate the tail of the source.

In operation 620, the generator 520 may generate the 3D sound by reflecting the tail of the source using the tail parameter.

The tail graph 1950 may operate regardless of the project cursor 1915.

The user may change the shape of the tail through manipulation of the input device. Depending on the manipulation of the input device by the user, the tail may change cyclically into a plurality of predefined shapes. When the shape of the tail changes, the setting unit 510 may set a value of the tail parameter according to the changed shape.

The velocity length value 1955 may represent the time used for the representation of velocity.

The timeline speed value 1945 may indicate a speed at a point indicated by the project cursor 1915 of the timeline 1910.

Timeline Gain Value 1965 The gain at the point indicated by the project cursor 1915 of the timeline 1910 may be represented.

Tail length 1970 may represent the reverberation time of the tail.

Tail end point 1975 may represent the end point of the tail.

The user may move the position of the end point of the tail through manipulation of the input device. In operation 610, the setting unit 510 may set the value of the tail parameter by reflecting the position of the moved end point.

If the position of the end point of the tail is moved, the length of the tail may change depending on the position of the end point to which the tail is moved. The changed tail length can be indicated at tail length 1970.

The sequence editing area 1980 may be an area that provides editing for the sequence. The user can manipulate the input device to select one of speed, gain and tail. As one of the speed, the gain and the tail is selected, the size of the edit window for editing the selected object may increase. If a specific object is not selected, an editing window of the same size may be provided for the objects.

20 illustrates a locator area of a project according to one embodiment.

The locator area 2000 may include a start point locator 2010, an end point locator 2020, and a project cursor 2040.

As described above, the time of the start point locator may be set to the start point of the timeline 1910 and the time of the end point locator may be set to the end point of the timeline 1910.

21 illustrates an enlarged speed graph according to an example.

In FIG. 21, an enlarged speed graph 2100 is shown enlarged in the sequence editing area 1980.

In FIG. 21, the point 2110 selected as the object of editing due to the manipulation of the input device by the user is shown, and the current speed 2120 is shown. The current speed 2120 can be displayed as the height of the graph at the point where the project cursor 1915 is located. Also, the current speed 2120 may be displayed at the speed length value 1955.

The 3D sound reproducing apparatus 100 may be implemented as the electronic device 2200 illustrated in FIG. 22. The electronic device 2200 may be a general purpose computer system that operates as the 3D sound reproducing apparatus 100.

As shown in FIG. 22, the electronic device 2200 may include at least a portion of a processor 2221, a network interface 2229, a memory 2223, a storage 2228, and a bus 2222. Components of the electronic device 2200, such as the processor 2221, the network interface 2229, the memory 2223, the storage 2228, and the like, may communicate with each other through the bus 2222.

The processor 2221 may be a semiconductor device that executes processing instructions stored in the memory 2223 or the storage 2228.

The processor 2221 may process a task required for the operation of the electronic device 2200. The processor 2221 may execute code of an operation or step of the processor 2221 described in the embodiments.

The network interface 2229 may be connected to the network 2230. The network interface 2229 may receive data or information required for the operation of the electronic device 2200, and may transmit data or information required for the operation of the electronic device 2200. The network interface 2229 may transmit data to and receive data from other devices via the network 2230. For example, the network interface 2229 may be a network chip or port.

Memory 2223 and storage 2228 may be various forms of volatile or nonvolatile storage media. For example, the memory 2223 may include at least one of a ROM 2224 and a RAM 2225. The storage 2228 may include internal storage media such as RAM, flash memory, hard disk, and the like, and may include removable storage media such as a memory card.

The electronic device 2200 may further include a user interface (UI) input device 2226 and a UI output device 2227. The UI input device 2226 may receive a user input required for the operation of the electronic device 2200. The UI output device 2227 may output information or data according to the operation of the electronic device 2200.

The function or operation of the electronic device 2200 may be performed as the processor 2221 executes at least one program module. The memory 2223 and / or the storage 2228 may store at least one program module. At least one program module may be configured to be executed by the processor 2221.

At least one program module may include a signal detector 110, a primary signal processor 120, a channel allocator 130, a secondary signal processor 140, a function controller 150, and a sound image externalization implementer 160. ), A bypass adjusting unit 170, a remote speaker detection / playback unit 180 and a proximity speaker detection / playback unit 190 may be included.

The UI input device 2226 can include a bypass switch 171.

The network interface 2229 may include a signal receiver 105.

The 3D sound providing apparatus 500 may be implemented as the electronic device 2300 illustrated in FIG. 23. The electronic device 2300 may be a general purpose computer system that operates as the 3D sound providing device 500.

As illustrated in FIG. 23, the electronic device 2300 may include at least a portion of a processor 2321, a network interface 2333, a memory 2323, a storage 2328, and a bus 2322. Components of the electronic device 2300, such as the processor 2321, the network interface 2329, the memory 2323, the storage 2328, and the like, may communicate with each other through the bus 2232.

The processor 2321 may be a semiconductor device that executes processing instructions stored in the memory 2323 or the storage 2328.

The processor 2321 may process a task required for the operation of the electronic device 2300. The processor 2321 may execute code of an operation or step of the processor 2321 described in the embodiments.

The network interface 2329 may be connected to the network 2330. The network interface 2329 may receive data or information required for the operation of the electronic device 2 # 00, and may transmit data or information required for the operation of the electronic device 2300. The network interface 2329 may transmit data to and receive data from other devices through the network 2330. For example, the network interface 2329 may be a network chip or a port.

The memory 2323 and the storage 2328 may be various forms of volatile or nonvolatile storage media. For example, the memory 2323 may include at least one of a ROM 2324 and a RAM 2325. The storage 2328 may include built-in storage media such as RAM, flash memory, hard disk, and the like, and may include removable storage media such as a memory card.

The electronic device 2300 may further include a user interface (UI) input device 2326 and a UI output device 2327. The UI input device 2326 may receive a user input required for the operation of the electronic device 2300. The UI output device 2327 may output information or data according to the operation of the electronic device 2300.

The function or operation of the electronic device 2300 may be performed as the processor 2321 executes at least one program module. The memory 2323 and / or the storage 2328 may store at least one program module. At least one program module may be configured to be executed by the processor 2321.

At least one program module may include a setting unit 510 and a generating unit 520.

The network interface 2329 may include an output unit 530.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Claims

Determining values of one or more parameters for the three-dimensional sound;

Generating data of the three-dimensional sound based on the one or more parameters; And

Outputting a signal comprising the data

Including,

Wherein said one or more parameters provide a three dimensional sound associated with a source of sound located in three dimensional space.
The method of claim 1,

Wherein said one or more parameters comprise a maximum distance parameter indicative of a maximum distance of an area behind the screen and a maximum threshold parameter indicative of a maximum distance of an area ahead of the screen.
The method of claim 1,

Wherein said one or more parameters comprise a damper parameter representing a damper of said source.
The method of claim 1,

Wherein said one or more parameters comprise a source start point parameter indicative of a position of a start point of said source and a source end point parameter indicative of a position of an end point of said source.
The method of claim 4, wherein

And wherein the one or more parameters include a source trace line parameter indicating a line through which the source from the start point to the end point travels.
The method of claim 5,

And wherein the source trace line parameter represents a plurality of lines connecting the start point and the end point.
The method of claim 1,

Wherein said one or more parameters comprise a velocity parameter indicative of a velocity of said source, said one or more parameters having a different value over time.
The method of claim 1,

And wherein the data of the three-dimensional sound comprises data of a plurality of channels, wherein the one or more parameters are used for each channel of the plurality of channels.
The method of claim 1,

Values of the one or more parameters are edited via a graphical interface.
A determination unit that determines values of one or more parameters for the three-dimensional sound;

A generator configured to generate data of the 3D sound based on the one or more parameters; And

An output unit for outputting a signal including the data

Including,

Wherein said one or more parameters are associated with a source of sound located in three-dimensional space.
The method of claim 10,

Wherein said one or more parameters comprise a maximum distance parameter indicative of a maximum distance of an area behind the screen and a maximum threshold parameter indicative of a maximum distance of an area ahead of the screen.
The method of claim 10,

And the one or more parameters include a damper parameter representing a damper of the source.
The method of claim 10,

And the one or more parameters include a source start point parameter indicative of the position of the start point of the source and a source end point parameter indicative of the position of the end point of the source.
The method of claim 13,

And the one or more parameters include a source trace line parameter indicating a line through which the source from the start point to the end point travels.
The method of claim 13,

And the source trace line parameter indicates a plurality of lines connecting the start point and the end point.
The method of claim 10,

And the one or more parameters include a velocity parameter indicative of the velocity of the source and the acceleration of the source, wherein the one or more parameters have different values over time.
The method of claim 10,

The data of the 3D sound includes data of a plurality of channels, wherein the one or more parameters are used for each channel of the plurality of channels.
The method of claim 10,

And the values of the one or more parameters are edited via a graphical interface.