KR19980073078A - Audio encoding / decoding apparatus and method - Google Patents

Abstract

The present invention relates to an audio encoding / decoding apparatus and method capable of scaling by a single structure and capable of simultaneously processing multiple contents, and an audio encoding apparatus includes a first filter for dividing an input audio signal into a low frequency band signal and a high frequency band signal, ; A second filter for dividing the separated low frequency band signal into a finer frequency band; A T / F converter for converting a high frequency band signal separated by the first filter from a time domain to a frequency domain; An ADPCM encoder for encoding an output signal of the second filter into a digital signal by an ADPCM method; A bit allocation quantization unit for bit-allocating and quantizing an output signal of the T / F conversion unit; A psychoacoustic unit processing the separated low frequency and high frequency signals according to a predetermined psychoacoustic model to provide information on quantization error control generated in the ADPCM encoder and the T / F converter; A control unit for controlling whether the input audio signal passes through a first filter and a second filter, controls a processing frequency band of the T / F conversion unit, and provides information indicating the multiple content processing mode or the scalable mode, ; And a primary bitstream forming unit for forming a bitstream using the signal encoded by the ADPCM encoder and the position information of the quantized bits and the content in the bit allocation quantization unit.

According to the present invention, it is possible to implement an encoder and a decoder capable of processing various contents with a single structure and capable of scale adjustment.

Description

Audio encoding / decoding apparatus and method

The present invention relates to an audio encoding / decoding apparatus and method, and more particularly, to an audio encoding / decoding apparatus and method capable of performing scale adjustment with a single structure and simultaneously processing multiple contents.

Recently, various interactive services such as video conferencing and video shopping have been provided. In such an interactive service, meaningful image units (hereinafter referred to as contents) are gathered to form a single screen. Each of the meaningful image units (contents) becomes one processing unit and is moved, enlarged, reduced and deleted individually. A system for processing a plurality of contents individually or simultaneously at the same time is called a multiple concurrent system.

Further, in order to effectively use the data transmission line, it is necessary to control the bit rate used for reproduction according to the importance of the bits of the bits used for the representation of information or the demand of the user. That is, a process that can be scalable to the number of bits used is required.

Generally, when audio data and video data are encoded or decoded, each content is encoded and decoded without being distinguished from each other. Therefore, only a specific audio signal among existing audio signals is extracted and reproduced, It is not easy to extract only a portion and perform processes such as movement, deletion, and transformation. This problem can be solved if independent control of each of the contents becomes possible. In the case of independent control of each of the contents, it is possible to eliminate the content such as a voice of a specific person, and to change the output audio data in consideration of the changed position when the position on the screen of a specific person changes.

However, in this case, since the sound of each person is transmitted to the independent channel, it can be easily performed by transmitting or not transmitting independent channel data in order to eliminate the sound of a specific person. However, There is a problem that the complexity of the system increases. Further, when the system is not used for a specific purpose such as a video conference using multiple contents, there is a problem that it is difficult to effectively utilize the system because many unused parts of system components are used.

The reason why the scaler is required in the encoder and the decoder is as follows. When there are video and audio information, in some cases, only video information is important, and only audio information is important. In this case, if the bit rate fixed for the video information and the audio information is used, the transmission capability of the channel may not be utilized effectively during information transmission. In this case, by adjusting the data transmission bit rate used for processing according to the importance of information, a channel having a limited transmission capability can be used more effectively. It is also important to know which program is being served, such as video channel search or audio channel search. Thus, the information used in the service is scalably reduced, so that even if the sound quality or the image quality is deteriorated, information on a large number of channels is sent at the same time, thereby enabling an effective channel search. However, in implementing an apparatus capable of adjusting the scale, since the conventional method generates a bitstream that can be scaled after obtaining error signals by encoding and decoding the error signals, if the number of steps required for scale control increases, There is an increasing problem.

On the other hand, the reason for multiple concurrent processing is as follows. In the case of a video conferencing or a multi-party call, when processing is possible for each person or each content, it is possible to delete a content such as a voice of a specific person if the person does not want to hear it. When changing, the position of the sound source can be moved and processed by modifying the output audio data in consideration of the changing position. If all these processes do not happen at the same time, the sound and the mouth shape will be different, and it will be unnatural because it does not seem to be in real-time conversation. So to handle multiple content, it must be a multiple concurrent processing system.

At this time, the positional shift of the sound source can be improved by applying the result of studying the human being listening to the sound existing in the three-dimensional space with both ears. In other words, the recognition characteristics of a human being recognizing a sound source in a spatial point were modeled by a study on the size difference of the sound signal to be felt by the right ear and the left ear or the sound transmission time, function. The HRTF functions are represented by an impulse response or transfer function at the middle ear for a characteristic when the signal is transmitted to the ear when sound is present at any point in space. By applying the HRTF, it becomes possible to carry out a process of transferring the sound to an arbitrary position in the three-dimensional space.

However, conventionally, it has been difficult to select only a sound generated from a specific area on the screen, that is, a meaningful part (content) such as a person or an animal. For example, it has been difficult to process video and audio to change or eliminate the position of a particular person on the screen. Therefore, it is not possible to process each independent content, and it is not easy to transform only a part of data transmitted or stored in the decoding step. As a result, in the conventional encoding and decoding system, scale control is possible and there is no consideration for a method capable of processing multiple contents at the same time.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and it is an object of the present invention to provide an apparatus and method for processing a plurality of contents that can be used for scale- And an audio encoding / decoding apparatus and method capable of performing simultaneous processing of multiple contents.

FIG. 1 is a block diagram of a configuration of an audio encoder capable of scale adjustment and multi-content processing according to the present invention.

FIG. 2 is a block diagram of a configuration of an audio decoder capable of scale adjustment and multi-content processing according to the present invention.

3A and 3B are block diagrams of a conventional ADPCM encoder and an ADPCM decoder.

4 illustrates a method of representing content location information of an image screen of an encoder.

FIGS. 5A and 5B are views for explaining a location information presentation method according to content movement on an image screen in a decoder, and show a screen after an original screen and content are moved.

6A and 6B are block diagrams showing configurations of an ADPCM encoder and an ADPCM decoder used in the present invention.

FIG. 7 shows an example of reproducing with a headphone and a speaker.

According to another aspect of the present invention, there is provided an audio encoding apparatus capable of scaling and scalable processing of multiple contents, including a first filter for dividing an input audio signal into a low frequency band signal and a high frequency band signal; A second filter for dividing the low-frequency band signal separated by the first filter into a finer frequency band; A T / F converter for converting a high frequency band signal separated by the first filter from a time domain to a frequency domain; An ADPCM encoder for encoding an output signal of the second filter into a digital signal by an ADPCM method; A bit allocation quantizer for bit allocating and quantizing an output signal of the T / F converter; A psychoacoustic unit processing the low-frequency and high-frequency signals separated by the first filter according to a predetermined acoustic psychological model and providing information on quantization error control generated in the ADPCM encoder and the T / F converter; And controls the processing frequency band of the T / F conversion unit. The information processing apparatus includes information indicating a multi-content processing mode or a scaleable mode, and position information of the content ; And a primary bitstream forming unit for forming a bitstream using the signal encoded by the ADPCM encoder, the quantized bits of the bit allocation quantization unit, and the position information of the content of the control unit.

According to another aspect of the present invention, there is provided an audio decoding apparatus capable of scaling by a single structure and capable of simultaneously processing multiple contents, comprising: a bit stream decomposer for decomposing an input bit stream; An inverse quantizer for inversely quantizing the bitstream deconstructed by the bitstream decomposition unit; An ADPCM decoder for decoding the bitstream deconstructed by the bitstream decomposition unit; A first signal synthesizer for synthesizing low frequency band signals decoded by the ADPCM decoder; An F / T converter for converting a high frequency band signal into a time domain; A second signal synthesizer for synthesizing the low frequency band signal synthesized by the first signal synthesizer and the F / T converter output signal; A space control processor for extracting positional information in a space of contents from the signal demultiplexed by the bitstream demultiplexing unit and adjusting a position of the sound source according to speaker reproduction or headphone reproduction; Wherein the decoding means decodes the bitstream to determine whether the bitstream is in a multi-content processing mode or a scalable mode, and outputs the output signal of the F / T converter when the determined mode is the multi- And controlling the scale adjustment in the space control processing unit according to a scale adjustment command of the user; And a buffer output unit for temporarily storing and outputting signals output from the space control processing unit and the second signal combining unit.

According to another aspect of the present invention, there is provided an audio encoding method capable of scaling a single structure and simultaneously processing multiple contents according to the present invention. The audio encoding method includes: dividing an input audio signal into a low frequency band signal and a high frequency band signal, Separation step; A low frequency separation step of dividing the low frequency band signal separated in the frequency band separation step into a finer frequency band; Determining whether to encode multiple contents to be processed simultaneously or to encode to enable scale adjustment; A coding step of coding a signal separated in the low frequency separation step into a digital signal by an ADPCM method when coding is performed so that multiple contents can be simultaneously processed; Frequency band separating step, the signal separated in the low-frequency demultiplexing step is encoded into a digital signal by the ADPCM method, and the high-frequency band signal separated in the frequency- / F conversion step; A quantization step of bit-allocating and quantizing a signal transformed in the T / F conversion step; An acoustic psychological step of processing the low-frequency and high-frequency signals separated in the frequency band separation step according to a predetermined acoustic psychological model and providing information on a step difference value of a quantizer used in the encoding step; And forming a bit stream using the encoded signal, the quantized bits, and the position information of the content.

According to another aspect of the present invention, there is provided an audio decoding method capable of adjusting a scale by a single structure and simultaneously processing multiple contents, comprising: a bit stream decomposing step of decomposing an input bit stream; Determining whether the decoded bitstream is a multiple content processing mode or a scalable mode in the bitstream decompression step; An inverse quantization step of inversely quantizing the decoded bit stream; A decoding step of decoding the decoded bit stream in the decoding step; A first signal synthesizing step of synthesizing the decoded low frequency band signal; An F / T conversion step of converting a high frequency band signal into a time domain if the determined mode is not a multiple content processing mode; A second signal synthesis step of synthesizing the low-frequency band signal synthesized in the signal synthesis step and the signal converted in the F / T conversion step; A spatial processing step of adjusting a scale according to a scale adjustment command of a user and extracting position information in a space of contents from the decomposed signal in the bit stream decompression step to adjust a position of a sound source according to speaker reproduction or headphone reproduction, ; And buffering the signal synthesized in the second signal synthesis step and the signal processed in the spatial processing step and outputting the signal.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram of a configuration of an audio encoder capable of scale control and multiple content processing according to an embodiment of the present invention. The first filter 100, the second filter 110, the ADPCM encoder 120, the T / A bit allocation quantization unit 140, a primary bitstream formation unit 150, an acoustic psychology unit 160, a control unit 170, a prediction unit 180, and a bitstream formation unit 190 .

The first filter 100 separates an input audio signal into a low frequency band signal and a high frequency band signal. The second filter 110 separates the low frequency band signal separated by the first filter 100 into a finer frequency band. The ADPCM encoder 120 encodes an output signal of the second filter 110 into a digital signal by an ADPCM method.

The T / F converter 130 converts a high frequency band signal separated by the first filter 100 from a time domain to a frequency domain. The bit allocation quantization unit 140 performs bit allocation and quantization on the output signal of the T / F conversion unit 130. The primary bitstream forming unit 150 forms a bitstream using the position information and the processing mode of the signal encoded by the ADPCM encoder 120 and the bits and content quantized by the bit allocation quantization unit 40 do.

The acoustic psychotherapy unit 160 processes low frequency and high frequency signals separated from the first filter 100 according to a psychoacoustic model and outputs a delta value which is a step value of a quantizer used in the ADPCM encoder 120 And provides a measure for determining the number of bits used in the bit allocation quantization unit 140.

The control unit 170 controls whether the input audio signal passes through the first filter 100 and the second filter 110 and controls the processing frequency band of the T / F conversion unit 130, A processing mode indicating a processing mode or a scalable mode, and location information of the content.

The predictor 180 obtains the association between the previous frame information of the primary bitstream forming unit 150 and the current frame information. The bitstream forming unit 190 reduces the redundant data according to the frame relevance calculated by the predictor 180 to form a bitstream.

FIG. 2 is a block diagram illustrating a configuration of an audio decoder capable of scale control and multiple content processing according to the present invention. The decoder includes a bitstream decomposition unit 200, an inverse quantizer 250, an ADPCM decoder 230, 1 signal synthesis unit 240, an F / T conversion unit 260, a second signal synthesis unit 280, a spatial control processing unit 270, a control unit 290, a buffer output unit 295, a prediction unit 210, And a primary bit stream demultiplexing unit 220.

The bitstream decompression unit 200 decomposes the input bitstream. The predictor 210 determines whether the bitstream decomposed in the bitstream decomposer 200 is a bitstream using previous frame information.

The primary bitstream demultiplexing unit 220 reconstructs the bitstream using the previous frame information if the prediction unit 210 determines that the bitstream is the previous frame information.

The dequantizer 250 dequantizes the decoded bitstream in the primary bitstream demultiplexer 220. The ADPCM decoder 230 decodes the decoded bitstream in the primary bitstream demultiplexer 200. The first signal synthesizer 240 synthesizes low frequency band signals decoded by the ADPCM decoder 230.

The F / T converter 260 converts the high frequency band signal into a time domain. The second signal synthesizer 280 synthesizes the low frequency band signal synthesized by the first signal synthesizer 240 and the output signal of the F / T converter 260.

The space control processing unit 270 extracts positional information in the space of contents from the signal decomposed in the bitstream decomposition unit 200 and adjusts the position of the sound source according to the speaker reproduction or the headphone reproduction. The space control processor 270 also obtains the positional coordinates of the video content according to the positional change of the video content having the positional information to adjust the sound considering the positional shift of the sound source. The position information of the space control processor 270 uses the position of the vocal organ as a reference position.

The control unit 290 receives the demultiplexed signal from the bitstream demultiplexing unit 220 to determine whether the bitstream is in the multiple content processing mode or the scale adjustable mode. If the determined mode is the multiple content processing mode, The output signal of the F / T converter 260 is not output, and the scale control in the space control processor 270 is controlled according to the scale adjustment command of the user.

The buffer output unit 295 temporarily stores signals output from the space control processing unit 270 and the second signal combining unit 280 and outputs the signals.

The operation of the present invention will now be described based on the above-described configuration. First, let's look at the encoder. The user selects one of the multiple concurrent processing and the scalable coding in the controller 170. [ If the selected operation mode is multiple concurrent processing, the controller 170 does not operate the T / F converter 130, and the filters 100 and 110 and the ADPCM encoder 120 operate And processes the allocated content. The filters 100 and 110 are anti-aliasing filters for considering the frequency characteristics of the content, and the band-limited signals are encoded by the ADPCM encoder as shown in FIG. 6A. At this time, the step delta of the quantizer (not shown) used in the ADPCM encoder 120 is controlled by the psychoacoustic unit 160 modeling the acoustic psychology. Here, by using four filters and ADPCM in parallel, concurrent processing of the contents is possible when a maximum of four contents are present.

On the other hand, when the user selects scalable coding in the operation mode in the controller 170, the input signal is for one piece of content, and the input signal can be processed in five frequency bands. If the sampling frequency is Fs, it is processed into five bands of Fs / 4 - Fs / 2, 0 - Fs / 16, Fs / 16 - Fs / 8, Fs / 8 - 3Fs / 16 and 3Fs / 16 - do. Since the sensitivity of the human is low for the high frequency signal, the T / F conversion unit 120 performs the T / F conversion on the high frequency side by widening the frequency band used for the processing and the complexity on the low frequency side is simple And the ADPCM unit 130 is used.

Here, ADPCM using a nonlinear predictor is used in the low frequency band to resilience errors that may occur in data transmission. An example of the error resilience of the ADPCM encoder 120 using the nonlinear predictor will be described later in more detail. The reason for using the ADPCM without using the DPCM is to use a quantizer suitable for the signal and also to calculate the power of the error signal generated by the ADPCM result as shown in FIG. The buffer control can be made taking into consideration whether or not it falls within the limits determined by the psychoacoustic model.

The result of the processing of the data is transmitted to the primary bitstream forming unit 150, and after being transmitted, the prediction unit 180 compares the bitstream with the previously configured bitstream to obtain different points. At this time, if the association between the previous frame and the subsequent frame becomes equal to or greater than a predetermined threshold value, a bit stream is formed and transmitted through the bit stream forming unit 190 with prediction on, The prediction off is performed to form a bit stream.

On the other hand, the decoder is as follows. First, a bitstream is decomposed through the bitstream hetec unit 200. Then, the prediction unit 210 checks the prediction on / off information on the decoded bit stream to reconstruct the bit stream, knowing whether or not to use the previous frame result for processing. If the prediction is on, the primary bitstream decomposing unit 220 decomposes the bitstream using the previous frame result in the process, and is inversely quantized through the inverse quantizer 250. If the prediction is off, the bitstream demultiplexer 200 uses the bitstream as it is.

Then, the control unit 290 reads information on the bitstream and detects whether the bitstream is performing multiple concurrent processing or a scalable decoder. In case of multiple concurrent processing, the bit stream decoded by the controller 290 is not passed to the F / T converter 260 but is performed by the ADPCM decoder 230. [ Then, in contrast to the case of being encoded by the first signal synthesizing unit 240, the signals are further summed with respect to the low frequency portions finely divided into one low frequency band. If it is used as a scalable encoder and a decoder, signals are reproduced while being passed to the F / T converter 260 under the control of the controller 290.

The low frequency band signal reproduced by the first signal synthesizer 240 and the high frequency band signal converted by the F / T converter 260 are combined through the signal synthesizer 280 and output to the buffer output unit 295 .

The location information of each content on the bit stream is used to perform a more effective processing on the spatial position of each content in decoding. Here, the control unit 290 obtains a moving position by moving the position of a content, and compensates for the movement of the position of the sound source. The reason why the encoder decides not to compensate the position of the sound source in the encoder is because if the encoder performs a transformation according to another movement when transformed by the encoder, the transformed effect is canceled after the transformed effect is removed from the encoder This is because there is a problem that the complexity is doubled. It is possible to prevent the complexity from doubling by considering only the decoder.

The process for the multiple content will be described in more detail as follows. It is detected by the controller 290 whether the bitstream is a multiple content processing input or not. and ADPCM decoding 230 used for processing low frequency signals in the case of multiple contents, respectively, to process independent content. In this case, the signal to be handled is scalable and is different from the frequency bandwidth used when the ADPCM encoder 120 and the T / F converter 130 are used. Since the processing is performed using an ADPCM encoder that is independent of each content at the time of encoding, the sound of the specific content can be completely eliminated by the user's control, and the distribution characteristic in the space possessed by the specific content can be modified.

And if the signal is scalable, the input signal is for one content. 4 - Fs / 2, 0 - Fs / 16, Fs / 16 - Fs / 8, Fs / 8 - 3Fs / 16, and 3Fs / 16 - Fs / 4. Since the sensitivity of the human being to the high frequency signal is lowered, the frequency band used for processing is widened and subjected to T / F conversion in the high frequency side, so that the F / T conversion unit 260 restores it by F / T conversion. In order to simplify the configuration complexity in the low frequency side, the signal is decoded by the ADPCM decoder 230 since it is encoded by the ADPCM encoder. When a fast search is needed, it improves the processing efficiency by reading and decoding only a part of the bit stream.

On the other hand, new location information is obtained and processed using the location information of the content as follows. And the location of the content of the image is included in the bitstream at the time of encoding. FIG. 4 illustrates a content position information presentation method of an image screen of an encoder. The location information transmitted by a bitstream is information on x and y coordinate values in an image screen as shown in FIG. 4, As a reference. Since the position of the mouth is the position of the sound source, the position of the mouth is used for processing. In the case of the mouth which does not appear in the image, a point on the image frame is assumed as the position where the mouth exists.

FIG. 4 shows an example based on the reference point. At this time, the screen used for the processing is divided into the background, the content, and the border lines of the respective contents, and the combined images are reproduced to extract the content using the border line information when the content of the screen is moved by the decoder Move back to the new position.

At the time of decoding, the video information is a single content, and the user can move the content up / down / left / right or adjust the size by zoom in / out. When decoded, considering the processing of coordinate values which change according to the movement in the up, down, left, and right direction, the processing is performed as if the sound comes out from the position seen on the screen upon restoration. 5A and 5B, if the user changes the position of the person to the original position (FIG. 5A) as shown in FIG. 5B, the user reproduces the position using the changed position information value (x, y) The sound that is changed is processed in consideration of the position of the changed video content. In addition, when the video content zoom in / out, the information is used as the (z) information and the new (x, y, z) . As a result, the up / down / left / right movement of the image content can be processed for the back and forth movement with water. The method of spatial movement of sound sources will be described in more detail later.

The reason why the ADPCM encoder and decoder 120 and 230 used in the encoder and the decoder in the low frequency band and the multiple concurrent processing use the nonlinear predictor will be described. The effect of the error signal varies depending on whether the predictor constituting the DPCM unit or the ADPCM unit is a linear predictor or a nonlinear predictor. The linear predictor has the effect that the error signal is accumulated and continues to be transmitted to the surrounding signal, whereas the nonlinear predictor has the effect that the influence of the error signal is not propagated to the surrounding signal because the error is isolated. For example, consider the case where a linear predictor and a nonlinear predictor are used in the predictor portion of the ADPCM encoder / decoder of FIGS. 3A and 3B.

[Equation 1]

P_out [n] = mean {P_in [n], P_in [n-1], P_in [n-2]

= integer [(P_in [n] + P_in [n-1] + P_in [n-2]) / 3.0]

The linear predictor adds three values of P_in [n], P_in [n-1] and P_in [n-2] as shown in Equation 1 and divides the value by 3 into an integer value to obtain a value of P_out [n] .

On the other hand, as the nonlinear predictor, a median predictor such as Equation (2) is used.

&Quot; (2) "

P_out = median {P_in [n], P_in [n-1], P_in [n-2]

That is, the nonlinear predictor arranges the three samples P_in [n], P_in [n-1], and P_in [n-2] in the order of magnitude as described above, To the value of P_out [n].

The input / output values of the encoder and the input / output values of the decoder for the linear predictor / nonlinear predictor for X_in, Cod_X, Cod_Y and Y_out are as follows.

[Table 1]

Encoder input / output by linear predictor

n One 2 3 4 5 6 7 8 9 X_in 25 30 35 40 35 30 25 20 15 Cod_X 10 10 10 10 0 -7 -10 -10 -10 P_in 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 37 35 30 25

[Table 2]

Decoder input / output by nonlinear predictor

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 0 -7 -10 -10 -10 Y_out 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 37 35 30 25

[Table 3]

Encoder output by nonlinear predictor

n One 2 3 4 5 6 7 8 9 X_in 25 30 35 40 35 30 25 20 15 Cod_X 10 10 10 10 0 -5 -10 -10 -10 P_in 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 35 35 30 25

[Table 4]

Decoder input and output by nonlinear predictor

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 0 -5 -10 -10 -10 Y_out 20 25 30 35 40 35 30 25 20 P_out 15 20 25 30 35 35 35 30 25

Consider a case where noise is generated in the channel in the transmitted state as follows. In the case where n is 5, the output value of the decoder using the linear predictor and the nonlinear predictor is different when the original signal is 0 but is changed to 100 by the error signal.

[Table 5]

Decoder input and output by linear predictor when noise occurs in channel

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 100 -7 -10 -10 -10 Y_out 20 25 30 35 40 135 63 69 79 P_out 15 20 25 30 35 70 79 89 70

[Table 6]

Decoder input and output by nonlinear predictor when noise occurs in channel

n One 2 3 4 5 6 7 8 9 Cod_Y 10 10 10 10 100 -5 -10 -10 -10 Y_out 20 25 30 35 40 135 35 30 25 P_out 15 20 25 30 35 40 40 35 30

In the case of the nonlinear predictor, the effect is isolated when the value of 0 is changed to 100, but it can be seen that the effect is propagated without being isolated in the linear predictor.

It is possible to detect the error signal actually generated and use the quantizer more effectively in audio coding, and to perform buffer control. This is possible by using a masked threshold generated by human acoustic psychology. This threshold represents the power of a signal that a human can not even hear. If the sum of the quantization noise generated when the signals of the corresponding band are quantized becomes smaller, it means that no more fine quantizer is needed and no more bits are needed. This property is used to control the number of bits used.

The amount of error signal generated while decoding the ADPCM-coded signals is calculated as shown in FIG. The quantization step of the quantizer, the delta adjustment of the ADPCM, and the buffer control of the frame are examined by examining whether the sum of the errors and the threshold constant determined by the psychoacoustic model exceed the limit. If the limit is exceeded, a new quantizer is used to reduce the result so that it can be adjusted according to the trade-off between sound quality and bit usage.

The three-dimensional sound effect is an effect that occurs because humans collect and listen to sound as two ears. Such a three-dimensional sound effect can be provided by controlling signals reproduced according to the positions of the fixed reproduction speakers when the stereo signal is reproduced. Studies on human speech recognition can be broadly divided into one study with one ear of the right or left ear and one study with both ears. Studies on one ear can be modeled on the process of feeling the existence of a sound and its characteristics, so that the absolute threshold value of a human perceptible signal, or the masking ), And the results are used for effective expression of data, that is, compression. The study of both ears is based on a study of the interactions between input signals in both ears, ie, the difference in the magnitude of the sound signal between the right ear and the left ear, or the phase of the sound coming into the right ear and left ear, I have done things about difference.

The results of these studies on both ears have modeled the cognitive characteristics of human beings that perceive sound sources at one point in space, and these characteristics are called HRTF (head related transfer function). The HRTF functions are represented by an impulse response or transfer function at the middle ear for a characteristic when the signal is transmitted in a positive direction when sound is present at any point in space. By applying the HRTF, the place where the sound exists can be moved to an arbitrary position in the three-dimensional space, so that a more realistic reproduction is possible.

Using the information of an arbitrary point A in the three-dimensional space, the effect of reproducing the sound at that point can be easily obtained by listening to the headphones. If the sound at a specific point A in the space is X _A , the signals E_r and E_l coming into the right ear and left ear are expressed as follows. Here, H_ar and H_al are the deformation characteristics of the signal that is heard when the sound from point A is heard on the right and left ears. Expressed as a matrix,

&Quot; (3) "

Respectively.

Using the H_ar, H_al, the mono input signal is made to feel as though it is heard at point A When these effects are applied to the front right / left speakers, it is necessary to compensate for the cross-talk effect caused by the output of both speakers. The signals from the right speaker and the left speaker are , The signal coming into the ear through the right and left speakers The

&Quot; (4) "

. here Is a transfer function.

If the values according to the equations (3) and (4) are the same, the signal is located at the point A. If you release,

&Quot; (5) "

.

In order to obtain the solution of Equation (5), the values output from the right speaker and the left speaker should be adjusted. Output value of speaker The values The values are Assuming that the signal is a signal transformed by < RTI ID = 0.0 >

&Quot; (6) "

(5), " (5) "

&Quot; (7) "

. The values that are transformed by the inverse transform The following are obtained.

&Quot; (8) "

here Are values determined when the position of the speaker is fixed, Is a known value that is determined by the position of the sound source when it is determined Can be saved.

After obtaining these values, the signal existing at position A in the three-dimensional space can be reproduced at a different arbitrary position by using Equation (6), and reproduced by using a speaker as in the case of sound at A position.

There is a problem that the speaker reproduction and the feeling on the sound signal when the headphone is reproduced are different from each other due to the lack of proper processing conversion depending on the presence or absence of the crosstalk. Therefore, when reproducing with a headphone, the processing efficiency can be obtained by using Equation (3). In order to consider such a difference in processing, the present invention receives a speaker / headphone output adjustment signal from the control unit as shown in FIG. 7, and if the value is OFF, recognizes only the headphone and processes it so that the speaker output compensation process is not performed. And recognizes it as a speaker and performs processing to compensate for speaker output values.

According to the present invention, it is possible to implement an encoder and a decoder capable of processing various contents with a single structure and capable of scale adjustment. The quantization error that is actually generated in the ADPCM is used for processing to select a quantizer step and perform buffer control. Content manipulation is possible. That is, it is possible to turn on / off specific contents, and it is possible to reduce error propagation by using a nonlinear predictor.

In addition, it is possible to move the position of the sound source according to the movement of the specific content. By performing different processes for the reproduction using the speaker and the reproduction using the headphone, a more suitable processing considering the reproduction environment is made possible, and the quantization step of the ADPCM is determined in consideration of human acoustic psychological characteristics.

Also, when changing the position of a speaker used for reproduction, knowing the changing position can be used to adjust the reproduction to be more appropriate by using the information of the newly changed position. Even if the position of the sound source changes due to the movement of the content, the transfer function by the sound source at the specific positions of the human being in the processing is used for the reproduction.

Claims (11)

A first filter for dividing an input audio signal into a low frequency band signal and a high frequency band signal; A second filter for dividing the low-frequency band signal separated by the first filter into a finer frequency band; A T / F converter for converting a high frequency band signal separated by the first filter from a time domain to a frequency domain; An ADPCM encoder for encoding an output signal of the second filter into a digital signal by an ADPCM method; A bit allocation quantizer for bit allocating and quantizing an output signal of the T / F converter; A psychoacoustic unit processing the low-frequency and high-frequency signals separated by the first filter according to a predetermined psychoacoustic model to provide information on a step difference value of a quantizer used in the ADPCM encoder; And controls the processing frequency band of the T / F conversion unit. The information processing apparatus includes information indicating a multi-content processing mode or a scaleable mode, and position information of the content ; And a primary bitstream formation unit for forming a bitstream using the signal encoded by the ADPCM encoder, the quantized bits of the bit allocation quantization unit, and the position information of the content of the control unit. . The apparatus of claim 1, further comprising: a prediction unit for obtaining data association between previous frame information and current frame information of the primary bitstream forming unit; And a bitstream forming unit for forming a bitstream by reducing redundant data according to a frame association of the predictor calculated by the predictor. 3. The audio encoding apparatus according to claim 1 or 2, wherein the ADPCM encoder uses a nonlinear predictor. A bit stream decomposition unit for decomposing an input bit stream; An inverse quantizer for inversely quantizing the bitstream deconstructed by the bitstream decomposition unit; An ADPCM decoder for decoding the bitstream deconstructed by the bitstream decomposition unit; A first signal synthesizer for synthesizing low frequency band signals decoded by the ADPCM decoder; An F / T converter for converting a high frequency band signal into a time domain; A second signal synthesizer for synthesizing the low frequency band signal synthesized by the first signal synthesizer and the F / T converter output signal; A space control processor for extracting positional information in a space of contents from the signal demultiplexed by the bitstream demultiplexing unit and adjusting a position of the sound source according to speaker reproduction or headphone reproduction; Wherein the decoding means decodes the bitstream to determine whether the bitstream is in a multi-content processing mode or a scalable mode, and outputs the output signal of the F / T converter when the determined mode is the multi- And controlling the scale adjustment in the space control processing unit according to a scale adjustment command of the user; And a buffer output unit for temporarily storing and outputting signals output from the space control processing unit and the second signal combining unit. 5. The apparatus of claim 4, further comprising: a predictor for determining whether the bitstream demultiplexed by the bitstream demultiplexer is a bitstream using previous frame information; And a primary bitstream demultiplexer for reconstructing a bitstream using previous frame information if it is determined that the prediction in the predictor is a bitstream using previous frame information. 5. The audio decoding apparatus of claim 4, wherein the spatial control processor adjusts the position of the image content according to the positional change of the image content having the positional information to adjust the sound considering the positional shift of the sound source. 7. The audio decoding apparatus of claim 6, wherein the position information of the space control processing unit uses a position of a vocal organ as a reference position. 5. The audio decoding apparatus of claim 4, wherein the ADPCM decoder uses a nonlinear predictor. 5. The audio decoding apparatus of claim 4, wherein the information about the enlargement / reduction of the video content is reflected in reproduction of the audio signal. A frequency band separation step of dividing the input audio signal into a low frequency band signal and a high frequency band signal; A low frequency separation step of dividing the low frequency band signal separated in the frequency band separation step into a finer frequency band; Determining whether to encode multiple contents to be processed simultaneously or to encode to enable scale adjustment; A coding step of coding a signal separated in the low frequency separation step into a digital signal by an ADPCM method when coding is performed so that multiple contents can be simultaneously processed; Frequency band separating step, the signal separated in the low-frequency demultiplexing step is encoded into a digital signal by the ADPCM method, and the high-frequency band signal separated in the frequency- / F conversion step; A quantization step of bit-allocating and quantizing a signal transformed in the T / F conversion step; An acoustic psychological step of processing the low-frequency and high-frequency signals separated in the frequency band separation step according to a predetermined acoustic psychological model and providing information on a step difference value of a quantizer used in the encoding step; And forming a bitstream using the encoded signal, the quantized bits, and the location information of the content. A bit stream decomposing step of decomposing an input bit stream; Determining whether the decoded bitstream is a multiple content processing mode or a scalable mode in the bitstream decompression step; An inverse quantization step of inversely quantizing the decoded bit stream; A decoding step of decoding the decoded bit stream in the decoding step; A first signal synthesizing step of synthesizing the decoded low frequency band signal; An F / T conversion step of converting a high frequency band signal into a time domain if the determined mode is not a multiple content processing mode; A second signal synthesis step of synthesizing the low-frequency band signal synthesized in the signal synthesis step and the signal converted in the F / T conversion step; A spatial processing step of adjusting a scale according to a scale adjustment command of a user and extracting position information in a space of contents from the decomposed signal in the bit stream decompression step to adjust a position of a sound source according to speaker reproduction or headphone reproduction, ; And buffering the signal synthesized in the second signal synthesis step and the signal processed in the spatial processing step and outputting the buffered signal.