JP2000151531A - Device and method for correcting audio data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device and a method for correcting audio data that corrects missing audio data in order to encode and multiplex audio data and video data synchronously with the video data. SOLUTION: A data quantity measurement means 11 measures a data quantity 1i per unit tie of received audio data. A data quantity comparison means 12 compares a data quantity 1t per preset unit time with the data quantity 1i. A correction data insert means 13 outputs the received audio data when the comparison result is equal, distributes uniformly correction data by a difference between the data quantity 1t and the data quantity 1i to the input audio data after the judgement of the smaller data quantity it and provides an output of the result when the comparison result indicates that the data quantity 1i is less than the data quantity 1t.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio data correcting apparatus and method for correcting audio data having a loss, and more particularly, to a method for encoding and multiplexing audio data and video data in synchronization with video data. TECHNICAL FIELD The present invention relates to an audio data correction apparatus and method for correcting audio data with noise.

[0002]

2. Description of the Related Art In recent years, personal computers called multimedia-compatible personal computers have appeared on the market at low prices, and it has become possible to easily reproduce audio and video on the personal computer. In addition, distribution of audio and video over the Internet has also been performed. Since the amount of audio and video data is very large, it is common to use coded data to reduce the amount of data, rather than using these data as the original data. Data including both audio data and video data is multiplexed and used so that the audio data and the video data can be synchronously reproduced (decoded). Examples of such coded multiplexing include:
MPEG is well known. Further, since the amount of processing required for reproduction (decoding) is relatively small, the use of software has become the mainstream at present.

Conventionally, in this field, when encoding and multiplexing audio data and video data, the amount of processing required for encoding is enormous. Using an expansion board or a peripheral device external to a personal computer) in real time, or using the software to take audio and video data in several times the real time. Was common. When using hardware equipped with a dedicated LSI, the acquisition of audio data
There was no loss in the audio data because it was performed by SI. Further, when software is used, the audio data may be prepared in advance, and there is no loss in the audio data. Therefore, there is no need to use the audio data with the loss, so that there is no need for an audio data correction apparatus and method for correcting the audio data with the loss.

[0004]

In such a situation, it is becoming possible to encode and multiplex audio data and video data by software in real time by improving the performance of the CPU. Real-time software is a high-performance CPU
In the world of PCs equipped with, it is expected that the mainstream of playback (decryption) will become mainstream in the future, just as software has become the mainstream. The real-time software in this case also includes software for performing processing by sharing a part of the encoding and multiplexing processing with hardware.

When encoding and multiplexing in real time by software, audio data is taken into a personal computer using an expansion board called a capture board that converts an analog audio signal into digital audio data. Usually, this board has a function of converting digital audio data into an analog audio signal and outputting the signal to a speaker, and is mounted on a personal computer as an audio board for inputting and outputting audio data. Unlike hardware with conventional dedicated LSI, it is used for other than the acquisition of audio data for encoding and multiplexing. Digitally converted audio data is used by software to share memory or processing inside the PC. Then, the data is transferred to hardware.

When the audio data is fetched in this way, there is a case where the fetching process by the software cannot be performed in real time, and the audio data may be missed. For example, A (1), A (2), A (3),.
If the user tries to capture the audio data of (N-1) and A (N) and fails to capture the audio data of A (2), A
A (2), which is (1), A (3),..., A (N-1), and A (N), is lost.

[0007] Since the audio data does not have time information indicating at what time the individual audio data was captured, A (1), A (3), ..., A (N-1), A (N )
Does not know if there is any loss or not. Further, it is not known which audio data is missing. When multiplexing is performed with video data using the audio data, a synchronization error occurs between the audio and the video after the time when the audio data is lost. This synchronization deviation is hardly perceived by a human when the loss is minute, but can be clearly perceived when the loss is accumulated. Immediately before the end of the video, there is no sound.

In view of the above-mentioned conventional problems, the present invention provides an audio data correction apparatus and method for correcting defective audio data in order to encode and multiplex audio data and video data in synchronization with video data. The purpose is to provide.

[0009]

According to the present invention, there is provided an audio data correction apparatus comprising: means for measuring a first data amount per unit time of input audio data; and a second data amount per unit time set in advance. Means for comparing the first data amount with the first data amount; outputting the input audio data when the first data amount is equal to the second data amount; and outputting the input audio data when the first data amount is smaller than the second data amount. Means for uniformly dispersing the correction data by the difference between the second data amount and the first data amount and inserting the corrected data into the input audio data after the point in time when it is determined that the data amount is small, and outputting the data.

The audio data correction apparatus of the present invention further comprises means for measuring a first data amount per unit time of the input audio data, and a second data amount per unit time which is set in advance. Means for comparing the amount of data with the data, means for temporarily storing the input voice data, and when the first data amount is equal to the second data amount, outputting the input voice data stored in the storing means; When the first data amount is smaller than the second data amount, the first data amount is evenly distributed and inserted into the input audio data stored in the correction data storing means by the difference between the second data amount and the first data amount. Output means.

Further, instead of inserting the correction data, the audio data correction apparatus of the present invention upsamples the input audio data and re-interpolates the input audio data to obtain a data amount corresponding to the difference between the second data amount and the first data amount. Is provided.

[0012]

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) FIG. 1 is a block diagram showing an example of the configuration of an audio data correction apparatus according to Embodiment 1 of the present invention. In FIG. 1, data amount measuring means 11
Is a data amount comparing means 1 for measuring a data amount of input voice data per unit time (hereinafter referred to as a data amount 1i).
Reference numeral 2 denotes a correction data insertion unit 13 that compares a preset data amount per unit time (hereinafter referred to as a data amount 1t) with a data amount 1i measured by the data amount measurement unit 11.
Outputs the input voice data when the comparison result of the data amount comparing means 12 is equal, and outputs the correction data by the difference between the data amount 1t and the data amount 1i when the comparison result of the data amount comparing means 12 is small. The audio data encoding device 14 that uniformly scatters and inserts and outputs the input audio data after the point in time when it is determined that the data amount 1i is small, encodes the corrected audio data. Encodes the input video data. The multiplexing means 16 multiplexes the coded audio data and the coded video data in synchronization with each other and outputs coded audio / video data.

The operation of the audio data correction device will be described in more detail with reference to the example shown in FIG. In FIG. 3, reference numeral 31 denotes original voice data, reference numeral 32 denotes defective data, reference numeral 33 denotes input voice data input to the voice data correction device, and reference numeral 34 denotes corrected voice data output from the voice data correction device. This figure shows a part of the audio data that is continuously input.

If there is no loss, audio data is input at a constant speed data amount, and this data amount is known in advance.
The data amount per unit time measured by the data amount measuring means 11 is always constant unless there is any loss in the audio data.
Assuming that this data amount is m, the data amount comparison unit 12 compares the data amount measured by the data amount measurement unit 11 with m. If they are equal, there is no loss, so the input voice data is output as it is.

If the data amount measured by the data amount measuring means 11 is smaller, it can be determined that there is a loss. Assuming that the data amount measured at this time is m ', data for mm' has been lost, and it is necessary to increase the data by mm '. Since it is not possible to determine which data has been lost, correction data of mm ′ is inserted in the input audio data next to the unit of measurement in which the data has been determined to be lost, evenly distributed and output. Inserting evenly distributed means that changes to the input audio data are distributed to several places and changes per place are kept to a minimum, making it difficult for a human to perceive that the correction data is inserted. That's why.

In this way, by correcting input audio data having a loss, encoded data in which video data and audio data are synchronized can be created. Further, since it is not necessary to perform a process of delaying the input voice data itself, it is possible to reduce a delay time due to the correction of the input voice data. The small delay time is effective for applications where it is desired to minimize the delay time, such as in the case of a TV phone or a surveillance camera.

In the first embodiment, the correction data is inserted into the input audio data next to the measurement unit in which the data is determined to be lost. If it exists, it may be anywhere and may extend to a plurality of measurement units. Further, the positions at which the correction data are inserted are uniformly distributed. However, the positions at which the correction data are inserted are not strictly uniform, but need not be extremely biased.

(Embodiment 2) FIG. 2 is a block diagram showing an example of the configuration of an audio data correction apparatus according to Embodiment 2 of the present invention. In FIG. 2, the data amount measuring means 21
Is a data amount comparing means 2 for measuring a data amount of input voice data per unit time (hereinafter referred to as a data amount 2i).
2 compares the preset data amount per unit time (hereinafter referred to as data amount 2t) with the data amount 2i measured by the data amount measuring means 21. The buffer 27 temporarily stores the input audio data. Correction data insertion means 23
Outputs the input audio data stored in the buffer 27 when the comparison result of the data amount comparing unit 22 is equal, and outputs the data amount 2t and the data amount 2i when the comparison result of the data amount comparing unit 22 is small. The audio data encoding device 24 encodes this corrected audio data by inserting and outputting the corrected data evenly dispersed in the input audio data stored in the buffer 27 and outputting the corrected data. Encodes the input video data. The multiplexing means 26 multiplexes the coded audio data and the coded video data in synchronization with each other and outputs coded audio / video data.

The operation of the audio data correction device will be described in more detail with reference to the example of FIG. In FIG. 4, reference numeral 41 denotes original voice data, reference numeral 42 denotes missing data, reference numeral 43 denotes input voice data input to the voice data correction device, and reference numeral 44 denotes corrected voice data output from the voice data correction device. This figure shows a part of the audio data that is continuously input.

The voice data is input at a constant speed data amount if there is no loss, and this data amount is known in advance.
The data amount per unit time measured by the data amount measuring means 21 is always constant unless there is any loss in the audio data.
Assuming that this data amount is m, the data amount comparison unit 22 compares the data amount measured by the data amount measurement unit 21 with m. If they are equal, there is no loss, so the input audio data stored in the buffer 27 is output as it is.

If the data amount measured by the data amount measuring means 21 is smaller, it can be determined that there is a loss. Assuming that the data amount measured at this time is m ', data for mm' has been lost, and it is necessary to increase the data by mm '. Since it is not possible to determine which data has been lost, the input audio data stored in the buffer 27 including the unit of measurement determined to have lost the data includes MM ′
The correction data is inserted evenly dispersed and output. Inserting evenly distributed means that changes to the input audio data are distributed to several places and changes per place are kept to a minimum, making it difficult for a human to perceive that the correction data is inserted. That's why.

As described above, by correcting input audio data having a loss, encoded data in which video data and audio data are synchronized can be created. Further, since the correction data is dispersed over a wide range, it is possible to make it more difficult to perceive that the correction data has been inserted. In addition, if the buffer storage unit is set to be the same as the data amount measurement unit, the correction data is inserted only in the measurement unit in which the loss has occurred, so that the influence of the correction data can be suppressed to a minimum range. The input audio data of the measurement units other than the above can be completely synchronized, and then encoded data synchronized as a whole can be created.

In the second embodiment, the correction data is inserted over the entire input audio data stored in the buffer 27 including the measurement unit whose data is determined to be lost. Good. Further, the positions at which the correction data are inserted are uniformly distributed. However, the positions at which the correction data are inserted are not strictly uniform, but need not be extremely biased.

(Embodiment 3) Embodiment 3 of the present invention
The correction data inserted by the correction data insertion means 13 of FIG. 1 described in the first embodiment and the correction data insertion means 23 of FIG. 2 described in the second embodiment is the sound data, the immediately preceding audio data, or It is either the immediately following audio data or interpolation data of the immediately preceding audio data and the immediately following audio data.

Since it is not known what the lost input voice data was, it is necessary to insert some data instead of it as correction data. Since the correction data is originally incorrect data, there is a possibility that the correction data is perceived as noise or abnormal noise. Silent data is hard to perceive because it is a momentary break in speech. Since the data immediately before or immediately after is the instantaneous continuation of the same sound, it is more difficult to perceive. The interpolation data of the immediately preceding audio data and the immediately succeeding audio data is the audio estimated from the preceding and following audio data, and is therefore more difficult to perceive.

(Embodiment 4) Embodiment 4 of the present invention
Is that the correction data insertion means 13 of FIG. 1 described in the first embodiment and the correction data insertion means 23 of FIG. 2 described in the second embodiment up-sample input audio data instead of inserting correction data. In this way, the data amount is increased by the lost MM ′ to correct the audio data.

Since the input voice data is up-sampled, the corrected voice data is obtained by expanding the input voice data temporally by the amount of the missing data, and it is difficult to perceive the correction.

(Embodiment 5) FIG. 5 is a flowchart showing an example of the configuration of the audio data correction method according to Embodiment 5 of the present invention. In FIG. 5, a data amount measuring step 52 measures a data amount of input audio data per unit time (hereinafter referred to as a data amount 5i). A data amount comparing step 53 sets a preset data amount per unit time. (Hereinafter referred to as data amount 5t) and data amount 5i measured in data amount measurement step 52. If the comparison result in data amount comparison step 53 is equal (5i = 5t), input voice is output in data output step 56. If the comparison result of the data amount comparison step 53 that outputs data is that the data amount 5i is small (5i <5t), in the correction data insertion step 55, the correction data is evenly distributed by the difference between the data amount 5t and the data amount 5i. Is inserted into the input audio data after the time when it is determined that the data amount 5i is small, and the data is output in a data output step 56. . The above steps are repeated until the input voice data ends.

The operation of the audio data correction method will be described in more detail with reference to the example of FIG. If there is no loss, the voice data is input at a constant speed data amount, and this data amount is known in advance. The data amount per unit time measured in the data amount measurement step 52 is always constant unless there is any loss in the audio data. Assuming that this data amount is m, a data amount comparison step 53 compares the data amount measured in the data amount measurement step 52 with m. If they are equal, there is no loss, so the input voice data is output as it is.

If the data amount measured in the data amount measuring step 52 is smaller, it can be determined that there is a loss. Assuming that the data amount measured at this time is m ', data for mm' has been lost, and it is necessary to increase the data by mm '. Since it is not possible to determine which data has been lost, correction data of mm ′ is inserted in the input audio data next to the unit of measurement in which the data has been determined to be lost, evenly distributed and output. Inserting evenly distributed means that changes to the input audio data are distributed to several places and changes per place are kept to a minimum, making it difficult for a human to perceive that the correction data is inserted. That's why.

In this way, by correcting the input audio data having a loss, encoded data in which the video data and the audio data are synchronized can be created. Further, since it is not necessary to perform a process of delaying the input voice data itself, it is possible to reduce a delay time due to the correction of the input voice data. The small delay time is effective for applications where it is desired to minimize the delay time, such as in the case of a TV phone or a surveillance camera.

In the fifth embodiment, the correction data is inserted into the input audio data next to the measurement unit in which the data is determined to be lost. If it exists, it may be anywhere and may extend to a plurality of measurement units. Further, the positions at which the correction data are inserted are uniformly distributed. However, the positions at which the correction data are inserted are not strictly uniform, but need not be extremely biased.

(Embodiment 6) FIG. 6 is a flowchart showing an example of the configuration of the audio data correction method according to Embodiment 6 of the present invention. In FIG. 6, in a data temporary storage step 67, the input audio data is temporarily stored in a buffer. In a data amount measurement step 62, a data amount of the input audio data per unit time (hereinafter referred to as a data amount 6i).
In the data amount comparison step 63, the data amount per unit time set in advance (hereinafter referred to as data amount 6t) and the data amount 6 measured in the data amount measurement step 62
When the comparison result of the data amount comparison step 63 for comparing i is equal (6i = 6t), the data output step 66
Then, the comparison result of the data amount comparison step 63 for outputting the input audio data indicates that the data amount 6i is small (6i <6t).
In this case, in the correction data insertion step 65, the correction data is evenly dispersed and inserted into the input audio data stored in the data temporary storage step 67 by the difference between the data amount 6t and the data amount 6i, and in the data output step 66, Output data. The above steps are repeated until the input voice data ends.

The operation of the audio data correction method will be described in more detail with reference to the example of FIG. If there is no loss, the voice data is input at a constant speed data amount, and this data amount is known in advance. The data amount per unit time measured in the data amount measurement step 62 is always constant unless there is any loss in the audio data. Assuming that this data amount is m, the data amount measured in the data amount measurement step 62 is compared with m in the data amount comparison step 63. If they are equal, there is no loss, and the input voice data stored in the data temporary storage step 67 is output as it is.

If the data amount measured in the data amount measuring step 62 is smaller, it can be determined that there is a loss. Assuming that the data amount measured at this time is m ', data for mm' has been lost, and it is necessary to increase the data by mm '. Since it is not possible to determine which data has been lost, the correction data of only m−m ′ is evenly distributed in the input voice data stored in the data temporary storage step 67 including the unit of measurement determined to be lost. Insert and output. Inserting evenly distributed is
This is because it is difficult for a human to perceive that the correction data is inserted by dispersing the change to the input voice data in several places and minimizing the change per place.

As described above, by correcting input audio data having a loss, encoded data in which video data and audio data are synchronized can be created. Further, since the correction data is dispersed over a wide range, it is possible to make it more difficult to perceive that the correction data has been inserted. In addition, if the buffer storage unit is set to be the same as the data amount measurement unit, the correction data is inserted only in the measurement unit in which the loss has occurred, so that the influence of the correction data can be suppressed to a minimum range. The input audio data of the measurement units other than the above can be completely synchronized, and then encoded data synchronized as a whole can be created.

In the sixth embodiment, the correction data is inserted over the entire input voice data stored in the data temporary storing step 67 including the unit of measurement determined to have lost data. Only the unit may be used.
Further, the positions at which the correction data are inserted are uniformly distributed. However, the positions at which the correction data are inserted are not strictly uniform, but need not be extremely biased.

(Embodiment 7) Embodiment 7 of the present invention
The correction data inserted in the correction data insertion step 55 shown in FIG. 5 described in the fifth embodiment and the correction data insertion step 65 shown in FIG. It is either the immediately following audio data or interpolation data of the immediately preceding audio data and the immediately following audio data.

Since it is not known what the lost input voice data was, it is necessary to insert some data instead of it as correction data. Since the correction data is originally incorrect data, there is a possibility that the correction data is perceived as noise or abnormal noise. Silent data is hard to perceive because it is a momentary break in speech. Since the data immediately before or immediately after is the instantaneous continuation of the same sound, it is more difficult to perceive. The interpolation data of the immediately preceding audio data and the immediately succeeding audio data is the audio estimated from the preceding and following audio data, and is therefore more difficult to perceive.

Embodiment 8 Embodiment 8 of the present invention
In the correction data insertion step 55 of FIG. 5 described in the fifth embodiment and the correction data insertion step 65 of FIG. 6 described in the sixth embodiment, input audio data is up-sampled instead of inserting correction data. In this way, the data amount is increased by the lost MM ′ to correct the audio data.

Since the input voice data is up-sampled, the corrected voice data is obtained by expanding the input voice data temporally by the amount of the missing data, and it is difficult to perceive the correction.

In all of the above-described first to eighth embodiments, only the case where the measured data amount is equal and the case where the measured data amount is small have been described. There may be many cases. In this case, the extra data amount may be evenly distributed and deleted, or down-sampling may be performed so that the extra data amount is reduced.

[0044]

As described above, according to the present invention, there is provided means for measuring a first data amount of input voice data per unit time,
The second data amount per unit time set in advance and the first
Means for comparing with the data amount of the input audio data and means for inserting and outputting the correction data into the input audio data, when the first data amount is equal to the second data amount, output the input audio data;
If the first data amount is smaller than the second data amount, the second
Correction of the input audio data with loss by dispersing the correction data evenly by the difference between the data amount of the first audio data and the first data amount and inserting and outputting the input audio data after the time point when the data amount is determined to be small. Thus, encoded data in which the video data and the audio data are synchronized can be created.

The present invention also relates to a means for measuring a first data amount per unit time of input audio data, and comparing the second data amount and the first data amount per unit time set in advance. Means for temporarily storing input audio data, and means for inserting correction data into the input audio data and outputting the same when the first data amount is equal to the second data amount. Outputting the stored input voice data, and when the first data amount is smaller than the second data amount, the input stored in the means for storing the correction data by the difference between the second data amount and the first data amount By inserting and outputting the audio data evenly dispersed in the audio data, it is possible to correct the deficient input audio data and create encoded data in which the video data and the audio data are synchronized.

Also, according to the present invention, the means for inserting and outputting correction data into the input audio data is such that instead of inserting the correction data, the input audio data is up-sampled and re-interpolated, and the second data amount and the second By increasing the data amount by the difference of the data amount of 1, it becomes possible to correct the deficient input audio data and create encoded data in which the video data and the audio data are synchronized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an example of a configuration according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of a configuration according to a second embodiment of the present invention;

FIG. 3 is an explanatory diagram illustrating an example of an operation according to the first embodiment of the present invention;

FIG. 4 is an explanatory diagram illustrating an example of an operation according to the second embodiment of the present invention.

FIG. 5 is a flowchart illustrating an example of a configuration according to a fifth embodiment of the present invention;

FIG. 6 is a flowchart illustrating an example of a configuration according to the sixth embodiment of the present invention;

[Explanation of symbols]

 11, 21 Data amount measurement means 12, 22 Data amount comparison means 13, 23 Correction data insertion means 27 Buffer 52, 62 Data amount measurement step 53, 63 Data amount comparison step 55, 65 Correction data insertion step 56, 66 Data output step 67 Temporary Data Storage Step

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Aki Yoneda 1006 Kazuma Kadoma, Kadoma, Osaka Prefecture F-term in Matsushita Electric Industrial Co., Ltd. 5C063 AB05 AC05 BA08 CA14 CA20 CA40 5K028 EE03 EE08 KK32 SS03 SS24

Claims (8)

[Claims] A means for measuring a first data amount per unit time of input audio data; and a means for comparing the first data amount with a second data amount per unit time set in advance. Outputting the input audio data when the first data amount is equal to the second data amount; and outputting the input audio data when the first data amount is smaller than the second data amount. Means for dispersing the correction data evenly by the difference in the first data amount and inserting and outputting the corrected data in the input audio data after the point in time when it is determined that the data amount is small. apparatus. 2. A means for measuring a first data amount per unit time of input voice data, and a means for comparing a second data amount per unit time and the first data amount set in advance. Means for temporarily storing the input sound data, and, when the first data amount is equal to the second data amount, outputting the input sound data stored in the storing means; If the data amount is smaller than the second data amount, the correction data is evenly distributed to the input audio data stored in the storage unit by the difference between the second data amount and the first data amount. And a means for inserting and outputting the data. 3. The method according to claim 1, wherein the correction data is any of silent data, immediately preceding audio data, immediately following audio data, or interpolation data of immediately preceding and succeeding audio data. The audio data correction device according to claim 1 or 2. 4. The method according to claim 1, wherein instead of inserting said correction data, said input audio data is upsampled and re-interpolated to increase the data amount by the difference between said second data amount and said first data amount. Claim 1 or Claim 2
An audio data correction device according to claim 1. 5. A step of measuring a first data amount per unit time of input audio data, and a step of comparing the second data amount per unit time and the first data amount set in advance. Outputting the input audio data when the first data amount is equal to the second data amount; and outputting the input audio data when the first data amount is smaller than the second data amount. A step of dispersing the correction data evenly by the difference in the first data amount and inserting the correction data into the input audio data after the point in time when the data amount is determined to be small, and outputting the input data. Method. 6. A step of measuring a first data amount of input audio data per unit time, and a step of comparing the first data amount with a preset second data amount per unit time. Temporarily storing the input audio data; and, if the first data amount is equal to the second data amount, outputting the input audio data stored in the storing step; Is smaller than the second data amount, the correction data is evenly distributed to the input audio data stored in the step of storing the correction data by the difference between the second data amount and the first data amount. And inserting and outputting the audio data. 7. The method according to claim 1, wherein the correction data is any of silent data, immediately preceding audio data, immediately following audio data, or interpolation data of immediately preceding and succeeding audio data. The audio data correction method according to claim 5 or 6. 8. Instead of inserting the correction data, up-sampling and re-interpolating the input audio data to increase the data amount by the difference between the second data amount and the first data amount. Claim 5 or Claim 6
The described audio data correction method.