KR100316769B1 - Audio encoder/decoder apparatus and method - Google Patents

Abstract

PURPOSE: An audio encoder/decoder apparatus and method is provided to perform a scale adjusting function by one structure and processing multiple contents at the same time. CONSTITUTION: The first filter(100) separates an input audio signal into low-frequency and high-frequency signals. The second filter(110) separates the low-frequency signal into a fine frequency band. An ADPCM encoder(120) encodes an output signal of the second filter(110) into a digital signal by an ADPCM type. A time-to-frequency conversion part(130) converts the high-frequency signal of a time domain into a frequency domain. A bit alignment and quantization part(140) bit-aligns and quantizes an output signal of the conversion part(130). The first bit stream forming part(150) forms a bit stream using the encoded signal, the quantized bits, position information of contents, and a process mode. A sound processing part(160) processes the low-frequency and high-frequency signals according to a sound psychological model to adjust a delta value of a quantizer used in the encoder(120). A control part(170) provides the process mode and the position information of contents. A prediction part(180) gets correlation between previous frame information of the part(150) and present frame information, and a bit stream forming part(190) reduces repeated data according to the calculated correlation information to form a bit stream.

Description

Audio encoding / decoding apparatus and method

The present invention relates to an audio encoding / decoding apparatus and method, and more particularly, to an audio encoding / decoding apparatus and method capable of scaling in one structure and simultaneously processing multiple contents.

Recently, various interactive services such as video conferencing and video shopping have been provided. In such an interactive service, meaningful video units (content) are gathered to form a screen. Each of the meaningful image units (contents) is one processing unit, and are individually moved, enlarged, reduced, and deleted. Such a system that processes a plurality of contents individually or simultaneously is called a multiple concurrent system.

In addition, in order to effectively use the data transmission line, it is necessary to adjust the bit rate used for reproduction according to the importance of the information or the user's request for the bits used for the representation of the information. That is, a process capable of scaling the number of bits used is required.

In general, when encoding or decoding audio data and video data, each content is encoded and decoded without being distinguished from each other. Therefore, only a specific audio signal is extracted from the existing audio signals and reproduced, or specific among existing video signals. It is not easy to extract only the parts and process them such as moving, deleting and transforming them. This problem can be solved when independent control of each content becomes possible. If each content is controlled independently, it can be eliminated when it is not desired to listen to content such as the voice of a specific person, and the output audio data can be modified in consideration of the changed position when the position of the specific person's screen is changed.

However, in this case, since the sound of each person is transmitted to an independent channel, it can be easily achieved by transmitting independent channel data or not to remove the specific person's sound, but the independent channel for each content to be processed. There is a problem that increases the complexity of the system by assigning. In addition, when the system is not used for a specific purpose, such as a video conference using multiple contents, there is a problem that effective use of the system is difficult because there are many unused parts of the system components.

On the other hand, the reason for the scalable (scalable) in the encoder and decoder is as follows. When there is video and audio information, only video information is important in some cases, and only audio information is important in some cases. In such a case, if a fixed bit rate is used for the video information and the audio information, it may not be possible to effectively use the channel transmission capability in transmitting the information. In this case, if the data transmission bit rate used for processing is adjusted according to the importance of information, a channel having a limited transmission capacity can be used more effectively. In addition, it is important to know which programs are served in cases such as video channel search and audio channel search. Therefore, the information used for the service is scalable, so that even if the sound quality or the image quality is deteriorated, information on many channels is sent at the same time to enable effective channel search. However, in the implementation of a device capable of scaling, the conventional method obtains error signals by encoding and decoding, and then creates a bitstream that can be scaled. Therefore, as the number of steps required for scaling is increased, the complexity is multiplied by the number of steps. There is an increasing problem.

On the other hand, the reasons for multiple concurrent processing are as follows. In the case of a video conference or a multi-party call, if the processing is possible for each person or according to each content, the content such as the voice of a specific person can be deleted if the user does not want to hear it. When changing, the output audio data can be modified in consideration of the changing position to move and process the position of the sound source. If all this does not happen at the same time, the sound you hear and the shape of your mouth will be wrong, and you will be unnatural because you don't seem to be talking in real time. Thus, to handle multiple contents, it must be a multiple concurrent processing system.

At this time, the positional movement of the sound source can be improved by applying the results of research on human hearing and feeling the sound existing in the three-dimensional space. In other words, based on the results of the difference between the magnitude of the sound signal sensed by the right ear and the left ear and the time of sound transmission, the recognition characteristics of the sound source that exist at a point in space are modeled. function). The HRTF functions are expressed as an impulse response or transfer function at the middle ear for a characteristic when there is sound at a point in space, when the signal is transmitted to both ears. By applying the HRTF, it becomes possible to move the place where sound exists to an arbitrary position in the three-dimensional space.

However, in the past, it was difficult to process only the sound generated in a specific area on the screen, that is, a meaningful part such as a person or an animal. For example, it was difficult to process video and audio to change or eliminate the position of a particular person on the screen. As a result, it was not possible to process each of the independent contents, and it was not easy to transform only a part of the data transmitted or stored in the decoding step. In conclusion, in the conventional encoding and decoding system, there is no consideration of a method of scaling and simultaneously processing multiple contents.

The present invention was created to solve the above-described problems, and scaled in a single structure for processing by a scale adjustment that can be used for channel search or the like for processing various contents that can be used for multi-party calls or video conferences. It is an object of the present invention to provide an audio encoding / decoding apparatus and method capable of adjusting and simultaneously processing multiple contents.

1 is a block diagram illustrating a configuration of an audio encoder capable of scaling and multi-content processing according to the present invention.

2 is a block diagram illustrating a configuration of an audio decoder capable of scaling and multi-content processing according to the present invention.

3A and 3B show block diagrams of a conventional ADPCM encoder and an ADPCM decoder.

4 illustrates a method of representing content position information of an image screen of an encoder.

5A and 5B are diagrams for describing a position information expression method according to a content movement on a video screen in a decoder, and show a screen after the original screen and the content are moved.

6A and 6B show block diagrams of the ADPCM encoder and ADPCM decoder used in the present invention.

Figure 7 shows an example of the case of reproducing with headphones and speakers.

According to the present invention for achieving the above object, an audio encoding apparatus capable of scaling in one structure and capable of simultaneously processing multiple contents includes a first filter for dividing an input audio signal into a low frequency band signal and a high frequency band signal; A second filter dividing the low frequency band signal separated by the first filter into a finer frequency band; A T / F converter for converting a high frequency band signal separated by the first filter from a time domain to a frequency domain; An ADPCM encoder for encoding the output signal of the second filter into a digital signal by the ADPCM method; A bit allocation and quantization unit for allocating and quantizing the output signal of the T / F converter; An acoustic psychology unit which processes the low frequency and high frequency signals separated by the first filter according to a predetermined acoustic psychology model and provides information on quantization error control generated by the ADPCM encoder and the T / F converter; Controlling whether the input audio signal passes through the first filter and the second filter, controlling the processing frequency band of the T / F converter, and indicating information indicating a multi-content processing mode or a scale adjustable mode and position information of the content. Providing a control unit; And a primary bitstream forming unit configured to form a bitstream by using the signal encoded by the ADPCM encoder, the bits quantized by the bit allocation and quantization unit, and position information of the content of the controller.

In order to achieve another object of the present invention, an audio decoding apparatus capable of adjusting scale in one structure and simultaneously processing multiple contents may include: a bitstream decomposing unit for decomposing an input bitstream; An inverse quantizer for inversely quantizing the bitstream decomposed by the bitstream decomposing unit; An ADPCM decoder for decoding the bitstream decomposed by the bitstream decomposing unit; A first signal synthesizer for synthesizing the signals of the low frequency bands decoded by the ADPCM decoder; An F / T converter for converting a high frequency band signal into a time domain; A second signal synthesizer for synthesizing the low frequency band signal synthesized by the first signal synthesizer and the output signal of the F / T converter; A space control processor configured to extract position information in the space of contents from the signal disassembled by the bitstream decomposing unit and adjust the position of the sound source according to whether the speaker or the headphone is reproduced; Receiving the signal disassembled by the bitstream decomposing unit, it is determined whether the bitstream is a multi-content processing mode or a scalable control mode, and if the determined mode is a multi-content processing mode, an output signal of the F / T converter is output. A control unit controlling the scale adjustment in the space control processing unit according to a user's scale adjustment command; And a buffer output unit which temporarily stores and outputs signals output from the spatial control processing unit and the second signal synthesizing unit.

According to the present invention for achieving the above object, the audio encoding method capable of adjusting the scale in one structure and simultaneous processing of multiple contents includes a frequency band dividing an input audio signal into a low frequency signal and a high frequency band signal. Separation step; A low frequency separation step of dividing the low frequency band signal separated in the frequency band separation step into finer frequency bands; Determining whether to encode multiple contents at the same time or to encode scales; An encoding step of encoding a signal separated in the low frequency separation step into a digital signal by an ADPCM method when encoding multiple content at the same time; In order to encode the scale, T which encodes the signal separated in the low frequency separation step into a digital signal by the ADPCM method and converts the high frequency band signal separated in the frequency band separation step from the time domain to the frequency domain. / F conversion step; A quantization step of bit-allocating and quantizing the signal converted in the T / F conversion step; An acoustic psychology step of processing the low frequency and high frequency signals separated in the frequency band separation step according to a predetermined acoustic psychological model to provide information on the step difference value of the quantizer used in the encoding step; And forming a bitstream using the encoded signal, the quantized bits, and position information of the content.

In order to achieve another object of the present invention, an audio decoding method capable of scaling in one structure and simultaneously processing multiple contents may include: a bitstream decomposing step of decomposing an input bitstream; Determining whether the decoded bitstream in the bitstream decomposing step is a multi-content processing mode or a scalable control mode; An inverse quantization step of inversely quantizing the disassembled bitstream; A decoding step of decoding the bitstream decomposed in the bitstream decomposing step; A first signal synthesis step of synthesizing the decoded low frequency band-specific signals; An F / T conversion step of converting a high frequency band signal into a time domain when the determined mode is not a multiple content processing mode; A second signal synthesis step of synthesizing the low frequency band signal synthesized in the signal synthesis step and the signal converted in the F / T conversion step; Spatial processing step of adjusting the scale according to the user's scale adjustment command and adjusting the position of the sound source according to whether the speaker or headphone is reproduced by extracting the position information in the space of the contents from the signal disassembled in the bitstream decomposing step. ; And buffering and outputting the signal synthesized in the second signal synthesis step and the signal processed in the spatial processing step.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. 1 is a block diagram illustrating a configuration of an audio encoder capable of scaling and multi-content processing according to the present invention. The first filter 100, the second filter 110, the ADPCM encoder 120, and T / F conversion are shown in FIG. To the unit 130, the bit allocation and quantization unit 140, the primary bit stream forming unit 150, the acoustic psychological unit 160, the control unit 170, the prediction unit 180 and the bit stream forming unit 190 Is done.

The first filter 100 separates the input audio signal into a low frequency band signal and a high frequency band signal. The second filter 110 separates the low frequency band signal separated by the first filter 100 into a finer frequency band. The ADPCM encoder 120 encodes the output signal of the second filter 110 into a digital signal by the ADPCM method.

The T / F converter 130 converts the high frequency band signal separated by the first filter 100 from the time domain to the frequency domain. The bit allocation and quantization unit 140 bit-assigns and quantizes the output signal of the T / F converter 130. The primary bitstream forming unit 150 generates a bitstream using a signal encoded by the ADPCM encoder 120, position information of bits and content quantized by the bit allocation and quantization unit 40, and a processing mode. Form.

The acoustic psychological unit 160 processes the low frequency and high frequency signals separated by the first filter 100 according to an acoustic psychological model, and thus a delta value that is a step value of the quantizer used in the ADPCM encoder 120. And a measure of determining the number of bits used in the bit allocation & quantization unit 140.

The controller 170 controls whether the input audio signal passes through the first filter 100 and the second filter 110, controls the processing frequency band of the T / F converter 130, and multi-contents. It provides a processing mode indicating a processing mode or a scale adjustable mode and position information of the content.

The prediction unit 180 obtains an association between previous frame information of the primary bitstream forming unit 150 and current frame information. The bitstream forming unit 190 forms a bitstream by reducing overlapping data according to the frame correlation calculated by the prediction unit 180.

2 is a block diagram illustrating a configuration of an audio decoder capable of scaling and multi-content processing according to the present invention. The bitstream decomposing unit 200, the dequantizer 250, the ADPCM decoder 230, 1 signal synthesizing unit 240, F / T converting unit 260, second signal synthesizing unit 280, spatial control processing unit 270, control unit 290, buffer output unit 295, prediction unit 210 It comprises a primary bit stream decomposing unit 220.

The bitstream tearing unit 200 tears down the input bitstream. The predictor 210 determines whether the bitstream decomposed by the bitstream decompressor 200 is a bitstream using previous frame information.

The primary bitstream decomposing unit 220 reconstructs the bitstream using previous frame information when it determines that the determination in the prediction unit 210 is a bitstream using previous frame information.

The dequantizer 250 dequantizes the bitstream disassembled by the primary bitstream decomposing unit 220. The ADPCM decoder 230 decodes the bitstream decomposed by the primary bitstream decomposing unit 200. The first signal synthesizer 240 synthesizes the signals for each low frequency band decoded by the ADPCM decoder 230.

The F / T converter 260 converts a high frequency band signal into a time domain. The second signal synthesis unit 280 synthesizes the low frequency band signal synthesized by the first signal synthesis unit 240 and the output signal of the F / T converter 260.

The space control processor 270 extracts the position information in the space of the contents from the signal disassembled by the bitstream decomposing unit 200 and adjusts the position of the sound source according to whether the speaker is reproduced or the headphones are reproduced. The spatial control processor 270 also obtains new position coordinates of the image content according to the positional change of the image content having the positional information and adjusts the sound considering the positional movement of the sound source. The position information of the space control processor 270 uses the position of the speech engine as a reference position.

The control unit 290 receives the signal disassembled by the bitstream decomposing unit 220 to determine whether the bitstream is a multi-content processing mode or a scale adjustable mode, and if the determined mode is a multi-content processing mode, The output signal of the F / T converter 260 is not output, and the scale control in the space control processor 270 is controlled according to a user's scale adjustment command.

The buffer output unit 295 temporarily stores and outputs signals output from the space control processing unit 270 and the second signal synthesizing unit 280.

Then, the operation of the present invention will be described based on the above configuration. First, we look at the encoder. The control unit 170 selects one operation mode among multiple concurrent processing and scalable coding. If the selected operation mode is multiple concurrent processing, the T / F converter 130 does not operate by the controller 170, and each of the filters 100 and 110 and the ADPCM encoder 120 operate. To process the allocated content. Here, the filters 100 and 110 are anti-aliasing filters for considering the frequency characteristic of the content for processing, and band-limited signals are encoded by an ADPCM encoder as shown in FIG. 6A. At this time, the step delta of the quantizer (not shown) used in the ADPCM encoder 120 is controlled by the acoustic psychology unit 160 modeling the acoustic psychology. Here, the use of a filter and four ADPCMs in parallel enables concurrent processing of up to four pieces of content.

On the other hand, when the user selects the scalable coding in the operation mode in the control unit 170, the input signal is for one content, it is possible to process the input signal by five frequency bands. If the sampling frequency is Fs, it is processed into five bands: Fs / 4-Fs / 2, 0-Fs / 16, Fs / 16-Fs / 8, Fs / 8-3Fs / 16, and 3Fs / 16-Fs / 4. do. Since humans are less sensitive to the high frequency signal, the T / F conversion is performed by the T / F converter 120 by widening the frequency band used for processing on the high frequency side, and the configuration complexity is simplified on the low frequency side. While using the ADPCM unit 130.

In the low frequency band, ADPCM using a nonlinear predictor is used for resilience to errors that may occur during data transmission. An example of the error resilience of the ADPCM encoder 120 using the nonlinear predictor will be described later in more detail. The reason why the ADPCM is used without using the DPCM is to use a quantizer that is more suitable for the signal, and also calculates the power of the error signal generated by the ADPCM result as shown in FIG. The buffer control is made possible by considering whether it falls within the limit obtained by the psychoacoustic model.

The result of the processing of the data is transferred to the primary bitstream forming unit 150, and after the transfer, the difference is obtained by comparing with the bitstream previously configured in the prediction unit 180. At this time, when the association between the previous frame and the next frame is more than a predetermined threshold value, the prediction is on, and the bitstream is configured and transmitted through the bitstream forming unit 190, and the predetermined threshold value is obtained. In the following cases, prediction off is performed to configure the bitstream.

Meanwhile, the decoder is as follows. First, a bitstream is decomposed through the bitstream body 200. Then, the prediction unit 210 checks the prediction on / off information on the decomposed bitstream and reconstructs the bitstream based on whether the previous frame result is used for processing or not. If the prediction is on, the first bitstream decomposing unit 220 decomposes the bitstream using the previous frame result for processing and dequantizes the dequantizer through the dequantizer 250. If the prediction is off, the bitstream decomposed by the bitstream decomposing unit 200 is used as it is.

Then, the control unit 290 reads the information on the bitstream and detects whether the bitstream is performing multiple concurrent processing or serving as a scalable decoder. In the case of multiple concurrent processing, the bitstream decomposed by the controller 290 does not pass through the F / T converter 260 and performs ADPCM by the ADPCM decoder 230. Then, the signals are summed again to form a low frequency band for the low frequency part that is finely divided, as opposed to when encoded by the first signal synthesis unit 240. When used as a scalable encoder and decoder, signals are reproduced while being passed to the F / T converter 260 by the control of the controller 290.

The low frequency band signal reproduced by the first signal synthesizer 240 and the high frequency band signal converted by the F / T converter 260 are combined through the first signal synthesizer 280 to provide the buffer output unit 295. Will be printed).

By using the location information of each content on the bitstream, other more effective processing on the spatial location of each content is possible. In this case, when the position of a certain content is moved by the controller 290, the position to be moved is calculated by calculation, and the compensation is performed according to the position movement of the sound source. The reason that the encoder considers the position of the sound source rather than the position compensation of the encoder is that if the encoder is transformed by another movement when it is transformed in the encoder, the encoder must control the deformation after removing the transformed effect. This is because there is a problem that the complexity is doubled. By considering only the decoder, the complexity can be prevented from being doubled.

Meanwhile, the processing of the multiple content will be described in detail as follows. The controller 290 detects whether the bitstream is a multiple content processing input. In the case of a signal for multiple contents, each of the ADPCM decoding 230 used for processing low frequency signals is processed for independent content. In this case, the signal to be handled is different from the frequency bandwidth to be handled when the ADPCM encoder 120 and the T / F converter 130 are used when the scale is adjustable. Since each content is processed using an independent ADPCM encoder during encoding, the sound of a specific content can be completely eliminated under the control of a user, and the distribution characteristic in the space of the specific content can be modified.

And if the signal is scalable, the input signal is for one content. Processing is possible for each of five frequency bands, and if the sampling frequency is Fs, Fs / 4-Fs / 2, 0-Fs / 16, Fs / 16-Fs / 8, Fs / 8-3Fs / 16, 3Fs / 16-It is treated as 5 bands of Fs / 4. Since the human body is less sensitive to the high frequency signal, the frequency band used for processing is widened and the T / F is converted to the high frequency side, so that the F / T converter 260 restores the F / T conversion. In order to simplify the complexity of the configuration on the lower frequency side, the decoding is performed by the ADPCM decoder 230 because the encoding is performed by the ADPCM encoder. When fast retrieval is required, only part of the bitstream is read and decoded to increase processing efficiency.

Meanwhile, the new location information is obtained and processed using the location information of the content as follows. At the time of encoding, the position of the content of the video is included in the bitstream. 4 is a diagram illustrating a method of representing content location information of an image screen of an encoder, wherein the position information transmitted by the bitstream is information about x and y coordinate values in the image screen as shown in FIG. 4, and the value is image content. Use one end of as a reference. It is characterized by using the position of the mouth for processing because the position of the mouth is the position of the sound source from which the sound comes out. In the case of the mouth that does not appear in the image, the mouth is assumed to be a position where the mouth exists.

4 shows an example by the reference point. At this time, the screen used for processing is divided into the background, the content, and the border lines of the contents, and the combined content is played back to extract the corresponding content by using the border information when the content of the screen is moved by the decoder. Make sure to move it back to a new location.

At the time of decoding, the image information is a single content, and the user can move the content up, down, left and right or adjust the size by zoom in / out. When decoding, considering the coordinate values that change according to the movement up, down, left, and right, the processing is performed as if the sound comes out from the position shown on the screen during restoration. For example, when there is a person A and B as shown in Figs. 5A and 5B, if the user changes the person's position from the original position (Fig. 5A) as shown in Fig. 5B, playback is performed using the changed location information values (x, y). The processing to change the sound taking into account the position of the changed video content. In addition, when the image content is zoomed in / out, the information is used as (z) information, and the effect of speaking at a short distance and speaking at a long distance using a new standard for (x, y, z) is obtained. Process it. As a result, the up, down, left, and right movements of the image content can be processed to move back and forth to the water. The spatial movement technique of the sound source is described in detail later.

Meanwhile, the reason why the ADPCM encoders and decoders 120 and 230 used in the encoder and the decoder during the low frequency band and the multiple concurrent processing use the nonlinear predictor will be described. The influence of the error signal varies depending on whether the predictor constituting the DPCM unit or the ADPCM unit is a linear predictor or a nonlinear predictor. In the linear predictor, error signals accumulate and continue to be transmitted to the surrounding signals, whereas in the nonlinear predictor, the error is isolated so that the influence of the error signal does not propagate continuously in the surrounding signals. For example, consider a case where a linear predictor and a nonlinear predictor are used in the predictor portion of the ADPCM encoder / decoder of FIGS. 3A and 3B.

[Equation 1]

P_out [n] = mean {P_in [n], P_in [n-1], P_in [n-2]}

= integer [(P_in [n] + P_in [n-1] + P_in [n-2]) / 3.0]

The linear predictor adds three values of P_in [n], P_in [n-1], and P_in [n-2] as shown in Equation 1, divides the value by 3, and quantizes the integer value to P_out [n]. Is a predictor.

Meanwhile, as a nonlinear predictor, a median predictor such as Equation 2 is used.

[Equation 2]

P_out = median {P_in [n], P_in [n-1], P_in [n-2]}

That is, the nonlinear predictor arranges three samples, P_in [n], P_in [n-1], and P_in [n-2] in size order as above, and then places the values in the center order among the ordered values. Is a predictor that makes P_out [n] a value.

Examples of the input / output values of the encoder and the input / output values of the decoder for the linear predictor / nonlinear predictor for X_in and Cod_X, Cod_Y and Y_out are as follows.

TABLE 1

Encoder input / output by linear predictor

n One 2 3 4 5 6 7 8 9 X_in 25 30 35 40 35 30 25 20 15 Cod_X 10 10 10 10 0 -7 -10 -10 -10 P_in 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 37 35 30 25

TABLE 2

Decoder Input / Output by Nonlinear Predictor

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 0 -7 -10 -10 -10 Y_out 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 37 35 30 25

TABLE 3

Encoder Output by Nonlinear Predictor

n One 2 3 4 5 6 7 8 9 X_in 25 30 35 40 35 30 25 20 15 Cod_X 10 10 10 10 0 -5 -10 -10 -10 P_in 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 35 35 30 25

TABLE 4

Decoder Input and Output by Nonlinear Predictor

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 0 -5 -10 -10 -10 Y_out 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 35 35 30 25

Considering the case where the noise occurs in the channel in the transmission state as follows. When n is 5, the original signal is 0, but the output value difference by the decoder using the linear predictor and the nonlinear predictor is shown for the case where the original signal is changed to 100 by the error signal.

TABLE 5

Decoder input and output by linear predictor when noise occurs in channel

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 100 -7 -10 -10 -10 Y_out 20 25 30 35 40 135 63 69 79 P_out 15 20 25 30 35 70 79 89 70

TABLE 6

Decoder input and output by nonlinear predictor in case of noise in channel

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 100 -5 -10 -10 -10 Y_out 20 25 30 35 40 135 35 30 25 P_out 15 20 25 30 35 40 40 35 30

In the case of the nonlinear predictor, the effect is isolated when 0 is changed to 100, but in the linear predictor, the effect is not isolated but propagates and reflects the effect.

By detecting an error signal that actually occurs, the quantizer can be used more effectively during audio encoding, and buffer control can be performed. This is possible by using masked thresholds caused by human psychoacoustics. This threshold represents the power of a signal that is slow even when humans enter it. If the sum of the quantization noises generated when quantizing the signals in the band is less than this, it means that no more detailed quantizer is needed and no more bits are needed. This property is used to control the number of bits used.

As shown in FIG. 6, the amount of error signal generated while decoding the ADPCM coded signals is calculated. It compares the sum of the errors and the threshold constant determined by the psychoacoustic model and checks whether it exceeds or exceeds the limit and uses it for quantization step adjustment of quantizer, delta adjustment of ADPCM, and buffer control of frame. If the limit is exceeded, a new quantizer is used to reduce the resulting value so that it can be adjusted according to trade-offs in sound quality and bit usage.

Three-dimensional sound effects are caused by humans listening to sound collected with two ears. The three-dimensional sound effect can be provided by controlling the signals to be reproduced according to the position of the fixed reproduction speaker when the reproduction by the stereo signal. Studies of human speech perception can be largely divided into one study with only one of the right or left ears and one study considering both ears. Research on one ear can model the process of sensing the presence or absence of sound and its characteristics so that humans can perceive the absolute threshold value of a signal or mask each other when multiple signals come in. ), And the results are used for effective representation of data, that is, compression. The study of both ears is a study of the mutual influence on the input signals coming from both ears, that is, the difference in the magnitude of the sound signals felt by the right and left ears or the phase of the sound coming into the right and left ears caused by the difference in sound propagation time. I have done things about the difference.

As a result of research on both ears, the recognition characteristics of the sound source that exist at a point in space are modeled and this characteristic is called the head related transfer function (HRTF). The HRTF functions are expressed as an impulse response or transfer function at the middle ear for a characteristic when a signal is transmitted to both ears when sound is present at some point in space. By applying the HRTF, the place where sound exists is moved to an arbitrary position in the three-dimensional space to enable more realistic reproduction.

Using information from an arbitrary point A in three-dimensional space, the effect that sounds are reproduced at that point can be easily achieved when listening with headphones. If the sound produced at a specific point A in space is X _A , the signals E_r and E_l coming into the right and left ears are expressed as follows. Where H_ar and H_al are the deformation characteristics of the signal that is felt when the sound from point A is heard by the right and left ears. In matrix,

[Equation 3]

Same as

이라 할 때, 오른쪽 왼쪽 스피커를 통해 귀에 들어오는 신호

들은Use the H_ar, H_al to make the mono input signal feel as if it is coming from point A. When this effect is achieved with the front right / left speakers, the cross-talk effect of the sound generated by the outputs of both speakers must be compensated for. The signals from the right and left speakers

When it comes to, the signal coming into the ear through the right left speaker

Heard

[Equation 4]

It can be represented as here Is the transfer function.

If the values according to the equations (3) and (4) are the same, it is felt that the signal is located at the point A. Loosen it,

[Equation 5]

Becomes

값들이

값이 각각

에 의해 변형된 신호라고 가정해 주면,In order to solve Equation 5, the output values of the right and left speakers should be adjusted. Speaker output value

Values

Each value is

Suppose that the signal is transformed by

[Equation 6]

Is the same as

[Equation 7]

들을 구하면 다음과 같다.Becomes Transformed by inverse transform

If you get them as follows.

[Equation 8]

는 스피커의 위치가 고정되면 결정되는 값들이고,

은 음원의 위치가 정해지면, 그 위치에 따라 정해지는 알려진 값들이기 때문에

을 구해줄 수가 있다.here

Are values determined when the speaker position is fixed.

Are the known values that are determined by the position of the sound source,

Can save

After obtaining these values, Equation 6 can be used to reproduce the signal existing at the position A in the three-dimensional space at another arbitrary position and reproduced using the speaker as if the sound is generated at the position A.

There is a problem with the feeling of the signal heard during speaker reproduction and headphone reproduction due to the lack of proper processing conversion depending on the presence or absence of crosstalk. Therefore, when reproduced with headphones, it is possible to increase the processing efficiency by using Equation 3. In order to consider this difference in the process, the present invention receives the speaker / headphone output adjustment signal from the control unit as shown in FIG. If it is "ON", it is recognized as a speaker and processes to compensate for speaker output values.

According to the present invention, it is possible to implement an encoder and a decoder capable of processing multiple contents and scaling in a single structure. The quantization error actually generated in ADPCM is used for processing to select the quantizer stage and to control the buffer. Content manipulation is possible. That is, the specific content can be turned on and off, and the propagation of errors can be reduced by using a nonlinear predictor.

In addition, it is possible to move the position of the sound source in accordance with the positional movement of the specific content. By performing different processes for the reproduction using the speaker and the reproduction using the headphone, a more suitable processing considering the reproduction environment is possible, and the quantization stage of the ADPCM device is determined in consideration of the human psychoacoustic characteristics.

In addition, when changing the position of the speaker used for reproduction, knowing the changing position, it is possible to process to adjust to the more appropriate reproduction using the information of the newly changed position. Even if the position of the sound source changes due to the movement of the content, the transfer function by the sound source at the specific positions of human beings is used for processing, so that the realistic reproduction can be performed.

Claims (11)

A first filter dividing the input audio signal into a low frequency band signal and a high frequency band signal; A second filter dividing the low frequency band signal separated by the first filter into a finer frequency band; A T / F converter for converting a high frequency band signal separated by the first filter from a time domain to a frequency domain; An ADPCM encoder for encoding the output signal of the second filter into a digital signal by the ADPCM method; A bit allocation and quantization unit for allocating and quantizing the output signal of the T / F converter; An acoustic psychometric unit which processes the low frequency and high frequency signals separated by the first filter according to a predetermined acoustic psychological model and provides information on the step difference value of the quantizer used in the ADPCM encoder; Controlling whether the input audio signal passes through the first filter and the second filter, controlling the processing frequency band of the T / F converter, and indicating information indicating a multi-content processing mode or a scale adjustable mode and position information of the content. Providing a control unit; And a primary bitstream forming unit configured to form a bitstream by using the signal encoded by the ADPCM encoder, the bits quantized by the bit allocation and quantization unit, and the position information of the content of the controller. Device. The apparatus of claim 1, further comprising: a prediction unit configured to obtain data correlation between previous frame information and current frame information of the primary bitstream forming unit; And a bitstream forming unit configured to form a bitstream by reducing overlapping data according to the frame correlation of the prediction unit calculated by the prediction unit. The audio encoding apparatus of claim 1 or 2, wherein the ADPCM encoder uses a nonlinear predictor. A bitstream decomposing unit for decomposing the input bitstream; An inverse quantizer for inversely quantizing the bitstream decomposed by the bitstream decomposing unit; An ADPCM decoder for decoding the bitstream decomposed by the bitstream decomposing unit; A first signal synthesizer for synthesizing the signals of the low frequency bands decoded by the ADPCM decoder; An F / T converter for converting a high frequency band signal into a time domain; A second signal synthesizer for synthesizing the low frequency band signal synthesized by the first signal synthesizer and the output signal of the F / T converter; A space control processor configured to extract position information in the space of contents from the signal disassembled by the bitstream decomposing unit and adjust the position of the sound source according to whether the speaker or the headphone is reproduced; Receiving the signal disassembled by the bitstream decomposing unit, it is determined whether the bitstream is a multi-content processing mode or a scalable control mode, and if the determined mode is a multi-content processing mode, an output signal of the F / T converter is output. A control unit controlling the scale adjustment in the space control processing unit according to a user's scale adjustment command; And a buffer output unit which temporarily stores and outputs signals output from the spatial control processing unit and the second signal synthesizing unit. The apparatus of claim 4, further comprising: a prediction unit that determines whether the bitstream decomposed by the bitstream decomposing unit is a bitstream using previous frame information; And a primary bitstream decomposing unit configured to reconstruct the bitstream using previous frame information if the prediction unit determines that the determination is a bitstream using previous frame information. The audio decoding apparatus as claimed in claim 4, wherein the spatial control processing unit obtains a new position coordinate of the image content according to the positional change of the image content having the position information and adjusts the sound in consideration of the positional movement of the sound source. The audio decoding apparatus of claim 6, wherein the position information of the spatial control processor uses the position of the speech engine as a reference position. 5. The audio decoding apparatus of claim 4, wherein the ADPCM decoder uses a nonlinear predictor. The audio decoding apparatus as claimed in claim 4, wherein the information about the enlargement / reduction of the image content is reflected in the reproduction of the audio signal. A frequency band separating step of dividing an input audio signal into a low frequency band signal and a high frequency band signal; A low frequency separation step of dividing the low frequency band signal separated in the frequency band separation step into finer frequency bands; Determining whether to encode multiple contents at the same time or to encode scales; An encoding step of encoding a signal separated in the low frequency separation step into a digital signal by an ADPCM method when encoding multiple content at the same time; In order to encode the scale, T which encodes the signal separated in the low frequency separation step into a digital signal by the ADPCM method and converts the high frequency band signal separated in the frequency band separation step from the time domain to the frequency domain. / F conversion step; A quantization step of bit-allocating and quantizing the signal converted in the T / F conversion step; An acoustic psychology step of processing the low frequency and high frequency signals separated in the frequency band separation step according to a predetermined acoustic psychological model to provide information on the step difference value of the quantizer used in the encoding step; And forming a bitstream using the encoded signal, the quantized bits, and position information of the content. A bitstream decomposing step of decomposing the input bitstream; Determining whether the decoded bitstream in the bitstream decomposing step is a multi-content processing mode or a scalable control mode; An inverse quantization step of inversely quantizing the disassembled bitstream; A decoding step of decoding the bitstream decomposed in the bitstream decomposing step; A first signal synthesis step of synthesizing the decoded low frequency band-specific signals; An F / T conversion step of converting a high frequency band signal into a time domain when the determined mode is not a multiple content processing mode; A second signal synthesis step of synthesizing the low frequency band signal synthesized in the signal synthesis step and the signal converted in the F / T conversion step; Spatial processing step of adjusting the scale according to the user's scale adjustment command and adjusting the position of the sound source according to whether the speaker or headphone is reproduced by extracting the position information in the space of the contents from the signal disassembled in the bitstream decomposing step. ; And buffering and outputting the signal synthesized in the second signal synthesis step and the signal processed in the spatial processing step.

Images