JP2011172153A - Media encoding and transmitting apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent band utilization efficiency from being deteriorated, while maintaining media reproduction quality, and to prevent a transmission error even when using a transmission line limited in band of the line. <P>SOLUTION: A media encoding and transmitting apparatus includes an importance level discriminating section 13, an encoding rate determining section 14, and the like. The importance level discriminating section 13 discriminates an importance level of media data to be encoded. With the importance level discriminated by the importance level discriminating section 13 as a criterion, the encoding rate determining section 14 determines an encoding rate E<SB>rate</SB>when compression-encoding the media data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a media encoding / transmission apparatus that compresses and encodes media data and transmits media encoded data that is media data after compression encoding in real time via a network.

For example, media encoding and transmission devices have been developed that compress and encode media data such as video data and audio data, and transmit media encoded data that is media data after compression encoding in real time via a network.
However, it occurs in the middle of transmission because the media encoded data sent from the media encoding transmission device passes through various repeaters constituting the network and circuits with different characteristics until it reaches the receiving side. The encoded media data may be damaged due to the influence of electrical noise or the like, or the encoded media may be discarded by the repeater due to congestion of the transmitted data, resulting in a transmission error in the encoded media data.

As a countermeasure when a transmission error occurs, when the receiving side detects the occurrence of a transmission error, it requests the transmitting side to retransmit the media encoded data, and the media encoded data retransmitted from the transmitting side is used to cause damage. In general, a retransmission process is performed to repair existing media encoded data.
However, in the case of such retransmission processing, since the transmission side starts retransmission of the media encoded data from the time when the transmission side receives the retransmission request from the reception side, there is a delay until the repair of the damaged media encoded data is completed. This is not suitable for real-time transmission of media encoded data.

Forward error correction in which error correction data capable of correcting transmission errors is added to media encoded data in advance and transmitted on the receiving side as a method for error correction of damaged media encoded data without delay (Forward Error Correction: FEC) is available (for example, see Non-Patent Document 1).
When the forward error correction method is used, when transmitting media data transmission packets constituting media encoded data, one FEC packet is set at regular intervals (for example, every 10 packets) of the media data transmission packets. Insert and send.
The content of the FEC packet is an XOR (eXclusive OR) of, for example, 10 media data transmission packets immediately before transmitting the FEC packet.

When the receiving side detects a missing packet by referring to the number continuously attached to the media data transmission packet, the remaining nine media data transmission packets and the FEC packet that are not missing are combined. Further, by taking XOR of a total of 10 packets, the missing media data transmission packet can be recovered.
However, when the forward error correction method is used, it is necessary to always transmit a certain amount of FEC packets regardless of whether or not a transmission error occurs, so that the band for transmitting original media encoded data is reduced. As a result, transmission efficiency is reduced.

In view of this, a media encoding and transmission apparatus has been developed that prevents a reduction in bandwidth utilization efficiency by controlling the data amount of error correction data according to the importance of media data (see, for example, Patent Document 1). .
In such a media encoding / transmission apparatus, the data length of the error correction data is changed according to the importance of the media data (for example, MPEG picture type).
The longer the data length of the error correction data, the higher the error correction capability. Therefore, the data length of the error correction data added to important media data is lengthened, and the error correction data to be added to other media data By shortening the data length, the bandwidth utilization efficiency is prevented from decreasing while maintaining the media playback quality.

JP 2008-92214 A (paragraph number [0016])

IETF RFC 2733 “An RTP Payload Format for Generic Forward Error Correction”

  Since the conventional media encoding / transmission apparatus is configured as described above, it controls the data length of the error correction data according to the importance of the media data, thereby preventing a reduction in bandwidth utilization efficiency. . However, even if the amount of media data is always constant, the data length of the error correction data changes according to the importance of the media data, so the total data length of the media data and the error correction data combined. Fluctuates from hour to hour. For this reason, for example, when using a transmission line with limited line bandwidth, such as satellite communication or wireless communication, when the entire data length increases and the transmission amount increases, the line bandwidth is limited. There is a problem that a transmission error may occur due to overrun.

  The present invention has been made to solve the above-described problems, and while maintaining the reproduction quality of media, it is possible to prevent a decrease in bandwidth utilization efficiency and to provide a transmission line with a limited bandwidth of the line. An object of the present invention is to obtain a media encoding transmission apparatus that can prevent transmission errors even when used.

  A media encoding transmission apparatus according to the present invention includes an importance level determining unit that determines the level of importance of media data to be encoded, and compressing the media data based on the level of importance determined by the level of importance determining unit. A rate determining means for determining an additional rate of an error correction code to be added to the media data after compression encoding, and a media rate at the encoding rate determined by the rate determining means. Media data encoding means for compressing and outputting media encoded data, which is media data after compression encoding, and media encoded data output from the media data encoding means determined by the rate determining means Error correction code is added at the specified additional rate and media encoded data with error correction code is output. Code adding means and data sending means for sending the media encoded data with error correction code output from the error correcting code adding means to the network are provided, and media encoding with error correcting code sent from the data sending means is provided When the data transmission rate deviates from the allowable range, the rate correction means instructs the rate determination means to correct the encoding rate and the additional rate so that the transmission rate falls within the allowable range. It is.

  According to the present invention, the importance determining means for determining the importance of the media data to be encoded, and the code for compressing and encoding the media data on the basis of the importance determined by the importance determining means A rate determination means for determining an addition rate of an error correction code to be added to the media data after compression encoding, and compression encoding the media data at the encoding rate determined by the rate determination means, Media data encoding means for outputting media encoded data that is media data after compression encoding, and the media encoded data output from the media data encoding means at an additional rate determined by the rate determining means Error correction code adding means for adding error correction code and outputting media encoded data with error correction code, and error Data transmission means for sending the media encoded data with error correction code output from the positive code adding means to the network, and the transmission rate of the media encoded data with error correction code sent from the data transmission means is allowed When the rate deviates from the range, the rate correction means instructs the rate determination means to correct the encoding rate and the additional rate so that the transmission rate falls within the allowable range. While maintaining the above, it is possible to prevent a decrease in bandwidth utilization efficiency and to prevent the occurrence of transmission errors even when a transmission line with a limited bandwidth is used.

It is a block diagram which shows the media coding transmission apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content of the media coding transmission apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the media data compression-encoded with the encoding rate E _rate , and the error correction code added with the addition rate A _rate . It is a block diagram which shows the media coding transmission apparatus by Embodiment 2 of this invention. It is a block diagram which shows the media coding transmission apparatus by Embodiment 3 of this invention. Video encoded data and audio encoded data multiplexed by the video / audio multiplexing unit 22, video encoded data and audio encoded data rearranged by the video / audio aligning unit 23, and an error correction code are added. It is explanatory drawing which shows the video coding data and audio | voice coding data which were read. It is a block diagram which shows the media coding transmission apparatus by Embodiment 9 of this invention.

Embodiment 1 FIG.
1 is a block diagram showing a media encoding / transmission apparatus according to Embodiment 1 of the present invention.
In FIG. 1, a media encoding / transmission apparatus 1 receives media data such as video data and audio data from a media source 2, compresses and encodes the media data, and encodes media data that is media data after compression encoding. This is a device for sending data to the network 3.
The media source 2 is a device that can output media data such as video data and audio data, and corresponds to, for example, a video camera or a VTR.
The network 3 corresponds to, for example, a wireless LAN or a satellite communication line.

  The media compression encoding processing unit 11 includes a feature amount extraction processing unit 12 and an encoding processing unit 16. When media data is input from the media source 2, the feature amount of the media data is extracted and the feature amount is extracted. Is output to the importance determination unit 13, and the media data is compressed and encoded, and the media encoded data that is the media data after the compression encoding is output to the error correction code adding unit 17.

The feature amount extraction processing unit 12 includes an interface (for example, a video input terminal or an audio input terminal) for inputting media data to be encoded from the media source 2, and features (for example, feature amounts of the input media data to be encoded) , Motion vectors, DCT (Discrete Cosine Transform) coefficients, presence / absence of voice data, etc.) are extracted, and their feature values are output to the importance level discrimination unit 13 and the encoding processing unit 16, and the media data Is output to the encoding processing unit 16.
The importance determination unit 13 refers to the feature amount output from the feature amount extraction processing unit 12 and performs a process of determining the importance of the media data to be encoded.
The feature amount extraction processing unit 12 and the importance level determination unit 13 constitute importance level determination means.

The encoding rate determination unit 14 performs a process of determining an encoding rate when the media data input by the feature amount extraction processing unit 12 is compression-encoded based on the importance determined by the importance determination unit 13. carry out.
The error correction code addition rate determination unit 15 determines the addition rate of the error correction code added to the media data compressed and encoded by the encoding processing unit 16 based on the importance determined by the importance determination unit 13. Implement the process.
The coding rate determination unit 14 and the error correction code addition rate determination unit 15 constitute a rate determination unit.

The encoding processing unit 16 compresses and encodes the media data according to the feature amount output from the feature amount extraction processing unit 12 at the encoding rate determined by the encoding rate determination unit 14, and the media data after the compression encoding The process which outputs the media coding data which is is implemented. Note that the encoding processing unit 16 constitutes media data encoding means.
The error correction code adding unit 17 adds an error correction code to the media encoded data output from the encoding processing unit 16 at the additional rate determined by the error correction code additional rate determining unit 15 and includes the error correction code. The process of outputting the encoded media data is executed. The error correction code adding unit 17 constitutes an error correction code adding means.

The data transmission unit 18 includes an interface (for example, a wireless LAN adapter, a wireless communication device, etc.) to the network 3, and transmits the media encoded data with the error correction code output from the error correction code addition unit 17 to the network 3. Perform the process. The data sending unit 18 constitutes data sending means.
When the transmission rate of the media encoded data with the error correction code sent from the data sending unit 18 deviates from the preset allowable range, the transmission rate correcting unit 19 and the coding rate determining unit 14 and the error The correction code addition rate determination unit 15 is instructed to correct the encoding rate and the addition rate, and performs processing for keeping the transmission rate within an allowable range. The transmission rate correction unit 19 constitutes a rate correction unit.

In the example of FIG. 1, the feature amount extraction processing unit 12, the importance level determination unit 13, the encoding rate determination unit 14, the error correction code addition rate determination unit 15, and the encoding processing unit 16 that are components of the media encoding transmission apparatus. Each of the error correction code adding unit 17, the data sending unit 18, and the transmission rate correcting unit 19 is configured by dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, or a one-chip microcomputer). However, when the media encoding / transmission apparatus is configured by a computer, the feature amount extraction processing unit 12, the importance level determination unit 13, the encoding rate determination unit 14, the error correction code addition rate determination unit 15, A program describing the processing contents of the encoding processing unit 16, the error correction code adding unit 17, the data sending unit 18 and the transmission rate correcting unit 19 is stored in a computer program. Stored in the re, CPU of the computer may execute a program stored in the memory.
FIG. 2 is a flowchart showing the processing contents of the media encoding / transmission apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
When the feature data extraction processing unit 12 of the media compression coding processing unit 11 inputs media data (video data, audio data) to be coded from the media source 2 (step ST1), the feature value extraction processing unit 12 extracts the feature value of the media data. (Step ST2).
Examples of the feature amount of the media data include, for example, the picture type of the media data, the motion vector used when the video data is compressed and encoded, the DCT coefficient calculated when the video data is compressed and encoded, and the media data The presence or absence of audio data is considered.

As specific procedures for extracting feature quantities, for example, “ISO / IEC MPEG (Moving Picture Experts Group) -1/2/4” and “ITU-T Recommendation H.261 / 262/263/264” are specified. Therefore, detailed description thereof will be omitted here.
When the feature amount extraction processing unit 12 extracts the feature amount of the media data, the feature amount extraction processing unit 12 outputs the feature amount to the importance determination unit 13 and the encoding processing unit 16 and outputs the media data to the encoding processing unit 16.

When receiving the feature amount of the media data from the feature amount extraction processing unit 12, the importance level determination unit 13 refers to the feature amount to determine the importance level of the media data to be encoded (step ST3).
The importance of the media data is determined based on the feature amount (the specific determination process of the importance will be described in Embodiments 4 to 8), but the importance of the media data is “the medium to be encoded” This is an index indicating the “degree of influence of data loss on the quality of media playback”. The greater the degree of influence, the higher the importance.

When the importance determination unit 13 determines the importance of the media data, the encoding rate determination unit 14 compresses and encodes the media data input by the feature amount extraction processing unit 12 based on the importance. The encoding rate E _rate is determined, and the encoding rate E _rate is output to the encoding processing unit 16 (step ST4).
Specific determination processing of the encoding rate E _rate will be described later.

When the encoding processing unit 16 receives the encoding rate E _rate from the encoding rate determining unit 14, the encoding processing unit 16 compresses the media data at the encoding rate E _rate in accordance with the feature amount output from the feature amount extraction processing unit 12. The encoded encoded media data, which is media data after compression encoding, is output to the error correction code adding unit 17 (step ST5).
Specific procedures for compressing and encoding media data include, for example, “ISO / IEC MPEG (Moving Picture Experts Group) -1/2/4” and “ITU-T Recommendation H.261 / 262/263/264”. Therefore, detailed description thereof is omitted here.

When the importance level determination unit 13 determines the importance level of the media data, the error correction code addition rate determination unit 15 adds an error to the media data compressed and encoded by the encoding processing unit 16 based on the importance level. The correction code addition rate A _rate is determined, and the error correction code addition rate A _rate is output to the error correction code addition unit 17 (step ST6).
Specific processing for determining the additional rate A _rate will be described later.

When the error correction code adding unit 17 receives the media encoded data from the encoding processing unit 16 and the media encoded data is damaged during transmission, the error correcting code adding unit 17 is used to restore the damaged media encoded data on the receiving side. An error correction code to be generated is generated.
That is, the error correction code adding unit 17 generates an error correction code having a data length corresponding to the additional rate A _rate output from the encoding processing unit 16, and adds the error correction code to the media encoded data (step). ST7).
For specific procedures for adding an error correction code, for example, the Reed-Solomon code procedure or the FEC packet insertion procedure defined in “IETF RFC2733“ An RTP Payload Format for Generic Forward Error Correction ”” is used. Detailed description will be omitted here.

Here, FIG. 3 is an explanatory diagram showing the media data compressed and encoded at the encoding rate E _rate and the error correction code added at the additional rate A _rate .
Hereinafter, with reference to FIG. 3, processing contents for determining the encoding rate E _rate and the additional rate A _rate according to the importance determined by the importance determination unit 13 will be specifically described.
In FIG. 3, the vertical axis represents the transmission rate of media encoded data with error correction code (media encoded data + error correction code).
The horizontal axis represents the transmission time of media encoded data with an error correction code, and the area represents the amount of media encoded data and error correction code information that can be transmitted.

Further, the example of FIG. 3 shows a case where the importance of media data is three stages of “high / medium / low”.
The upper diagram in FIG. 3 shows the distribution of error correction codes and media encoded data when the importance of the media data is not taken into account. The error correction code and the media code are always allocated regardless of the importance of the media data. The rate distribution of data is constant.
The lower diagram in FIG. 3 shows the distribution of error correction codes and media encoded data when the importance of the media data is taken into consideration.

If media data with high importance is lost due to a transmission error, it has a great influence on the reproduction quality of the media. Therefore, even if it is lost, it is necessary to improve the error correction capability so that it can be accurately corrected on the receiving side.
Therefore, the error correction code adding unit 17 increases the error correction code addition rate A _rate when the importance of the media data is higher than when the importance is low (see the lower diagram in FIG. 3). .
However, in order to maintain the entire transmission rate constant, it is necessary to decrease the encoding rate E _rate of the media data as the additional rate A _rate of the error correction code increases. The encoding rate E _rate of the media data is lowered.

However, in order to make it possible to transmit the same amount of information before and after increasing the additional rate A _rate of the error correction code, the transmission time allocated to the encoded data of the media data having a high importance is increased in the entire transmission time. The amount of information that can be transmitted is secured.
Maintaining the overall transmission time by shortening the transmission time allocated to encoded data of medium / low importance media data by increasing the transmission time allocated to encoded data of media data with high importance is doing.
If the amount of information that can be transmitted with medium / low importance media data is insufficient due to a reduction in transmission time, an error correction code is added to the encoded data of media data with low importance. The amount of information is secured by decreasing the rate A _rate .

Upon receiving the media encoded data with the error correction code from the error correction code adding unit 17, the data transmission unit 18 transmits the media encoded data with the error correction code to the network 3 (step ST8).
As described above, since the encoding rate E _rate and the additional rate A _rate are determined according to the importance of the media data, the transmission rate of the media encoded data with the error correction code sent from the data sending unit 18 Is controlled to be constant.
However, actually, an error may occur in the transmission rate depending on the contents of the media data.

Therefore, the transmission rate correction unit 19 monitors the transmission rate of the media encoded data with the error correction code sent from the data sending unit 18, and the transmission rate deviates from the preset allowable range. (Step ST9), the encoding rate determination unit 14 and the error correction code additional rate determination unit 15 are instructed to correct the encoding rate E _rate and the additional rate A _rate , and the transmission rate falls within the allowable range. (Step ST10).
Specifically, it is as follows.

The transmission rate correction unit 19 monitors the transmission rate of the media encoded data with the error correction code sent from the data sending unit 18, and the overall transmission rate of the media encoded data and the media encoded data Analyze the transmission rate by importance.
When the overall transmission rate of the media encoded data fluctuates beyond an allowable range, the transmission rate correction unit 19 performs media encoded data (for example, important data with an error correction code having a specific importance). Whether high-level media data occurs only in encoded data and error-correcting codes that have been compression-encoded or fluctuated as a whole regardless of the specific importance (errors related to all importance) Check if the media encoded data with the correction code has changed).

The transmission rate correction unit 19 encodes a correction instruction for the encoding rate E _rate corresponding to the corresponding importance when the fluctuation occurs only in the media encoded data with the error correction code related to the specific importance. The output is output to the rate determination unit 14, and the correction of the additional rate A _rate corresponding to the corresponding importance is output to the error correction code additional rate determination unit 15.
As a result, the encoding rate determination unit 14 performs correction to lower the encoding rate E _rate corresponding to the corresponding importance, and the error correction code addition rate determination unit 15 _sets the additional rate A _rate corresponding to the corresponding importance. Implement a correction to lower.

The transmission rate correction unit 19 outputs, to the encoding rate determination unit 14, a correction instruction for the encoding rate E _rate corresponding to all the importance levels when the overall fluctuation occurs regardless of the specific importance level. The correction instruction of the additional rate A _rate corresponding to all the importance levels is output to the error correction code additional rate determining unit 15.
As a result, the encoding rate determination unit 14 performs correction to lower the encoding rate E _rate corresponding to all the importance levels, and the error correction code additional rate determination unit 15 _sets the additional rate A _rate corresponding to all the importance levels. Implement a correction to lower.

As is apparent from the above, according to the first embodiment, the importance determination unit 13 that determines the importance of the media data to be encoded, and the importance determined by the importance determination unit 13 are used as a reference. The media data after compression encoding based on the importance determined by the encoding rate determination unit 14 and the importance determination unit 13 for determining the encoding rate E _rate when the media data is compressed and encoded. An error correction code addition rate determination unit 15 that determines an addition rate A _rate of the error correction code to be added to the media, and compression-encodes the media data at the encoding rate E _rate determined by the encoding rate determination unit 14 Encoding processing unit 16 that outputs media encoded data that is media data after conversion, and media encoded data output from encoding processing unit 16 , Adds an error correcting code added rate A _rate determined by the error correction code addition rate determining section 15 is provided with an error correction code addition unit 17 for outputting the medium coded data with the error correction code, the data delivery unit 18 is configured to send the media encoded data with the error correction code output from the error correction code adding unit 17 to the network 3, thereby preventing reduction in bandwidth utilization efficiency while maintaining the reproduction quality of the media. There is an effect that can be.

When the transmission rate of the media encoded data with the error correction code transmitted from the data transmission unit 18 deviates from the allowable range, the transmission rate correction unit 19 adds the encoding rate determination unit 14 and the error correction code addition. Since the rate determining unit 15 is instructed to correct the encoding rate E _rate and the additional rate A _rate and the transmission rate falls within the allowable range, a transmission path with a limited line bandwidth is used. Even in this case, there is an effect that it is possible to prevent occurrence of transmission errors.

Embodiment 2. FIG.
4 is a block diagram showing a media encoding / transmission apparatus according to Embodiment 2 of the present invention. In the figure, the same reference numerals as those in FIG.
The video compression coding processing unit 11a is composed of a video feature amount extraction processing unit 12a and a video coding processing unit 16a. When video data as media data is input from the media source 2, the feature amount of the video data is extracted. Then, the feature amount is output to the video importance determination unit 13a, and the video data is compressed and encoded, and the video encoded data which is the video data after the compression encoding is output to the video error correction code adding unit 17a. Perform the process.
The voice compression coding processing unit 11b is composed of a voice feature amount extraction processing unit 12b and a voice coding processing unit 16b. When voice data as media data is input from the media source 2, the feature amount of the voice data is extracted. Then, the feature amount is output to the speech importance determination unit 13b, the speech data is compressed and encoded, and the speech encoded data that is the speech data after compression encoding is output to the speech error correction code adding unit 17b. Perform the process.

The video feature quantity extraction processing unit 12a includes an interface (for example, a video input terminal) for inputting video data to be encoded from the media source 2, and features (for example, motion vectors) of the input video data to be encoded. , DCT coefficients, etc.) are extracted, and the feature amount is output to the video importance determination unit 13a and the video encoding processing unit 16a, and the video data is output to the video encoding processing unit 16a.
The audio feature amount extraction processing unit 12b includes an interface (for example, an audio input terminal) that inputs audio data to be encoded from the media source 2, and extracts the feature amount of the input audio data to be encoded, The feature amount is output to the speech importance determination unit 13b and the speech encoding processing unit 16b, and the process of outputting the speech data to the speech encoding processing unit 16b is performed.

The video importance level determination unit 13a refers to the feature amount output from the video feature amount extraction processing unit 12a and performs a process of determining the importance level of video data to be encoded.
The speech importance determination unit 13b refers to the feature amount output from the speech feature amount extraction processing unit 12b, and performs a process of determining the importance of the speech data to be encoded.
The video feature amount extraction processing unit 12a, the audio feature amount extraction processing unit 12b, the video importance level determination unit 13a, and the audio importance level determination unit 13b constitute importance level determination means.

The video coding rate determination unit 14a determines a coding rate for compressing and coding the video data input by the video feature amount extraction processing unit 12a based on the importance determined by the video importance determination unit 13a. Perform the process.
The speech coding rate determination unit 14b determines a coding rate when the speech data input by the speech feature amount extraction processing unit 12b is compression-encoded based on the importance determined by the speech importance determination unit 13b. Perform the process.

The video error correction code addition rate determination unit 15a uses the importance determined by the video importance determination unit 13a as a reference, and the error correction code addition rate added to the video data compressed and encoded by the video encoding processing unit 16a. The process of determining is performed.
The voice error correction code addition rate determination unit 15b uses the importance determined by the voice importance determination unit 13b as a reference, and the addition rate of the error correction code added to the voice data compressed and encoded by the voice encoding processing unit 16b. The process of determining is performed.
The video coding rate determination unit 14a, the audio coding rate determination unit 14b, the video error correction code addition rate determination unit 15a, and the audio error correction code addition rate determination unit 15b constitute a rate determination unit.

The video encoding processing unit 16a compresses and encodes video data according to the feature amount output from the video feature amount extraction processing unit 12a at the encoding rate determined by the video encoding rate determination unit 14a. The process which outputs the video coding data which is the video data of this is implemented.
The audio encoding processing unit 16b compresses and encodes audio data according to the feature amount output from the audio feature amount extraction processing unit 12b at the encoding rate determined by the audio encoding rate determination unit 14b. The process which outputs the audio | voice coding data which are audio | voice data of this is implemented.
The video encoding processing unit 16a and the audio encoding processing unit 16b constitute media data encoding means.

The video error correction code addition unit 17a adds an error correction code to the video encoded data output from the video encoding processing unit 16a at the additional rate determined by the video error correction code addition rate determination unit 15a, and generates an error. A process of outputting video encoded data with a correction code is performed.
The voice error correction code addition unit 17b adds an error correction code to the voice encoded data output from the voice coding processing unit 16b at the additional rate determined by the voice error correction code addition rate determination unit 15b. A process of outputting speech encoded data with a correction code is performed.
The video error correcting code adding unit 17a and the audio error correcting code adding unit 17b constitute error correcting code adding means.

The data transmission unit 20 includes an interface (for example, a wireless LAN adapter, a wireless communication device, etc.) to the network 3, and the video encoded data with error correction code and audio error correction output from the video error correction code adding unit 17a. A process of transmitting the voice encoded data with the error correction code output from the code adding unit 17b to the network 3 is performed. The data sending unit 20 constitutes data sending means.
The transmission rate correction unit 21 sets in advance the transmission rate of media encoded data with error correction code (video encoded data with error correction code + audio encoded data with error correction code) transmitted from the data transmission unit 20. If it deviates from the allowable range, the video coding rate determination unit 14a and the audio coding rate determination unit 14b are instructed to correct the coding rate, and the video error correction code addition rate determination unit 15a. And the correction of the additional rate is instructed to the voice error correcting code additional rate determining unit 15b, and the process of keeping the transmission rate within the allowable range is performed. The transmission rate correction unit 21 constitutes a rate correction unit.

  In the example of FIG. 4, the video feature amount extraction processing unit 12a, the audio feature amount extraction processing unit 12b, the video importance level determination unit 13a, the audio importance level determination unit 13b, and the video encoding rate, which are components of the media encoding transmission apparatus. Determination unit 14a, audio encoding rate determination unit 14b, video error correction code addition rate determination unit 15a, audio error correction code addition rate determination unit 15b, video encoding processing unit 16a, audio encoding processing unit 16b, video error correction code Each of the adding unit 17a, the voice error correcting code adding unit 17b, the data sending unit 20, and the transmission rate correcting unit 21 is a dedicated hardware (for example, a semiconductor integrated circuit mounted with a CPU or a one-chip microcomputer). If the media encoding transmission device is configured by a computer, a video feature amount extraction processing unit is assumed. 2a, audio feature amount extraction processing unit 12b, video importance level determination unit 13a, audio importance level determination unit 13b, video encoding rate determination unit 14a, audio encoding rate determination unit 14b, video error correction code addition rate determination unit 15a, Audio error correction code addition rate determination unit 15b, video encoding processing unit 16a, audio encoding processing unit 16b, video error correction code addition unit 17a, audio error correction code addition unit 17b, data transmission unit 20 and transmission rate correction unit 21 It is also possible to store a program describing the processing contents of the above in a memory of a computer and execute the program stored in the memory by the CPU of the computer.

In the first embodiment, media data is compressed and encoded and transmitted. However, in many cases, media data is often superimposed with a plurality of types of data such as video data and audio data.
However, a scene having a high importance level from the viewpoint of video data does not necessarily have a high importance level in audio data. On the other hand, a scene having a high importance level from the viewpoint of audio data does not necessarily have a high importance level in video data.
Therefore, in the second embodiment, the importance levels of the video data and the audio data are individually determined, and the encoding rate IE _rate and additional rate IA _rate related to the video data, and the encoding rate VE _rate related to the audio data, and The additional rate VA _rate is determined separately.

Next, the operation will be described.
When the video feature amount extraction processing unit 12a of the video compression encoding processing unit 11a receives the video data to be encoded from the media source 2, the feature amount of the video data is the same as the feature amount extraction processing unit 12 of FIG. To extract.
When the video feature amount extraction processing unit 12a extracts the feature amount of the video data, the video feature amount extraction processing unit 12a outputs the feature amount to the video importance level determination unit 13a and the video encoding processing unit 16a, and also outputs the video data to the video encoding processing unit 16a. Output to.

When the speech feature amount extraction processing unit 12b of the speech compression encoding processing unit 11b receives the speech data to be encoded from the media source 2, the feature amount of the speech data is the same as the feature amount extraction processing unit 12 of FIG. To extract.
When the speech feature amount extraction processing unit 12b extracts the feature amount of the speech data, the speech feature amount extraction processing unit 12b outputs the feature amount to the speech importance level determination unit 13b and the speech encoding processing unit 16b, and the speech data processing unit 16b outputs the speech data. Output to.

When receiving the feature quantity of the video data from the video feature quantity extraction processing section 12a, the video importance level judging section 13a refers to the feature quantity, as with the importance level judgment section 13 in FIG. Determine the importance of the data.
When receiving the feature amount of the voice data from the voice feature amount extraction processing unit 12b, the voice importance level discrimination unit 13b refers to the feature amount and, like the importance level discrimination unit 13 in FIG. Determine the importance of the data.

When the video importance level determination unit 13a determines the importance level of the video data, the video coding rate determination unit 14a extracts the video feature amount based on the importance level as in the coding rate determination unit 14 of FIG. An encoding rate IE _rate for compressing and encoding the video data input by the processing unit 12a is determined, and the encoding rate IE _rate is output to the video encoding processing unit 16a.
When the speech importance level determination unit 13b determines the importance level of the speech data, the speech coding rate determination unit 14b extracts speech feature values based on the importance level, as in the coding rate determination unit 14 of FIG. A coding rate VE _rate for compressing and coding the speech data input by the processing unit 12b is determined, and the coding rate VE _rate is output to the speech coding processing unit 16b.

When receiving the encoding rate IE _rate from the video encoding rate determining unit 14a, the video encoding processing unit 16a performs the video feature amount extraction process at the encoding rate IE _rate , as in the encoding processing unit 16 of FIG. The video data is compression-encoded according to the feature amount output from the unit 12a, and the video encoded data which is the video data after the compression encoding is output to the video error correction code adding unit 17a.
When the speech encoding processing unit 16b receives the encoding rate VE _rate from the speech encoding rate determining unit 14b, the speech encoding amount extraction process is performed at the encoding rate VE _rate as in the encoding processing unit 16 of FIG. The audio data is compression-encoded according to the feature amount output from the unit 12b, and the audio encoded data that is the audio data after compression encoding is output to the audio error correction code adding unit 17b.

When the video importance determination unit 13a determines the importance of the video data, the video error correction code addition rate determination unit 15a uses the importance as a reference in the same manner as the error correction code addition rate determination unit 15 in FIG. The addition rate IA _rate of the error correction code added to the video data compressed and encoded by the video encoding processing unit 16a is determined, and the addition rate IA _rate of the error correction code is output to the video error correction code addition unit 17a.
The voice error correction code addition rate determination unit 15b, when the voice importance determination unit 13b determines the importance of the voice data, like the error correction code addition rate determination unit 15 in FIG. The addition rate VA _rate of the error correction code to be added to the audio data compressed and encoded by the audio encoding processing unit 16b is determined, and the addition rate VA _rate of the error correction code is output to the audio error correction code addition unit 17b.

When the video error correction code adding unit 17a receives the video encoded data from the video encoding processing unit 16a, when the video encoded data is damaged during transmission, the video error correcting code adding unit 17a restores the damaged video encoded data on the receiving side. An error correction code used in the above is generated.
That is, the video error correction code adding unit 17a generates an error correction code having a data length corresponding to the additional rate IA _rate output from the video encoding processing unit 16a, and adds the error correction code to the video encoded data. .

When the speech error correction code adding unit 17b receives the speech encoded data from the speech encoding processing unit 16b, when the speech encoded data is damaged during transmission, the speech error correcting code adding unit 17b restores the damaged speech encoded data on the receiving side. An error correction code used in the above is generated.
That is, the voice error correction code adding unit 17b generates an error correction code having a data length corresponding to the additional rate VA _rate output from the voice encoding processing unit 16b, and adds the error correction code to the voice encoded data. .

When the data sending unit 20 receives the encoded video data with the error correcting code from the video error correcting code adding unit 17a and receives the encoded audio data with the error correcting code from the audio error correcting code adding unit 17b, the error correcting code Attached media encoded data (video encoded data with error correction code + audio encoded data with error correction code) is sent to the network 3.
As described above, since the encoding rates IE _rate and VE _rate and the additional rates IA _rate and VA _rate are determined according to the importance of the video data / audio data, the error correction code sent from the data sending unit 20 The transmission rate of the attached media encoded data is controlled to be constant.
However, actually, an error may occur in the transmission rate depending on the contents of the media data.

Therefore, the transmission rate correction unit 21 monitors the transmission rate of the media encoded data with the error correction code sent from the data sending unit 20, and the transmission rate deviates from a preset allowable range. For example, the video encoding rate determining unit 14a and the audio encoding rate determining unit 14b are instructed to correct the encoding rates IE _rate and VE _rate , and the video error correcting code adding rate determining unit 15a and the audio error correcting code are added. The rate determination unit 15b is instructed to correct the additional rates IA _rate and VA _rate so that the transmission rate falls within the allowable range.
Specifically, it is as follows.

The transmission rate correction unit 21 monitors the transmission rate of the media encoded data with the error correction code sent from the data sending unit 20, and determines the overall transmission rate of the media encoded data and the media encoded data. Analyze the transmission rate by importance.
When the overall transmission rate of the media encoded data fluctuates beyond an allowable range, the transmission rate correction unit 21 media encoded data with an error correction code having a specific importance (for example, important Whether video data / audio data with a high degree is generated only in encoded data and error correction code), it fluctuates as a whole regardless of the specific importance (all importance levels) Whether or not there is a change in the media encoded data with the error correction code.

When the fluctuation occurs only in the video encoded data with the error correction code related to the specific importance, the transmission rate correction unit 21 gives an instruction to correct the encoding rate IE _rate corresponding to the corresponding importance. And the correction of the additional rate IA _rate corresponding to the corresponding importance is output to the video error correction code additional rate determining unit 15a.
As a result, the video coding rate determination unit 14a performs correction to lower the coding rate IE _rate corresponding to the corresponding importance, and the video error correction code additional rate determination unit 15a performs the additional rate IA corresponding to the corresponding importance. Perform correction to lower the _rate .

When the fluctuation occurs only in the voice encoded data with the error correction code related to the specific importance, the transmission rate correction unit 21 gives a correction instruction of the coding rate VE _rate corresponding to the corresponding importance to the voice code. And the correction of the additional rate VA _rate corresponding to the corresponding importance is output to the voice error correction code additional rate determining unit 15b.
As a result, the speech coding rate determination unit 14b performs correction to lower the coding rate VE _rate corresponding to the corresponding importance, and the speech error correction code addition rate determination unit 15b performs the additional rate VA corresponding to the corresponding importance. Perform correction to lower the _rate .

When the transmission rate correction unit 21 changes as a whole regardless of the specific importance level, the transmission rate correction unit 21 sends the encoding rate IE _rate and VE _rate correction instructions corresponding to all the importance levels to the video encoding rate determination unit 14a and It outputs to the audio coding rate determination unit 14b, and gives instructions for correcting the additional rates IA _rate and VA _rate corresponding to all the importance levels. Output to.
As a result, the video coding rate determination unit 14a and the audio coding rate determination unit 14b perform correction to lower the coding rates IE _rate and VE _rate corresponding to all the importance levels, and the video error correction code addition rate determination unit 15a. The voice error correction code addition rate determination unit 15b performs correction to lower the addition rates IA _rate and VA _rate corresponding to all the importance levels.

As is apparent from the above, according to the second embodiment, the importance levels of the video data and the audio data are individually determined, and the encoding rate IE _rate and the additional rate IA _rate related to the video data are added to the audio data. Since the encoding rate VE _rate and the additional rate VA _rate are determined separately, even when the importance of the video data and audio data of the same scene is different, the band usage is maintained while maintaining the reproduction quality of the media. It is possible to prevent a reduction in efficiency and to prevent the occurrence of transmission errors even when using a transmission line with a limited line bandwidth.

Embodiment 3 FIG.
In the second embodiment, the importance levels of the video data and the audio data are individually determined, and the encoding rate IE _rate and additional rate IA _rate related to the video data, and the encoding rate VE _rate and additional rate related to the audio data are determined. Although the VA _rate is determined separately, for example, the video data and the audio code are encoded after the video data and the audio data are encoded and then the error correction code is added. In the case where multiplexed data is multiplexed, the second embodiment cannot be applied.
The third embodiment discloses an applicable embodiment even when video encoded data and audio encoded data are multiplexed after video data and audio data are encoded and before an error correction code is added. To do.

5 is a block diagram showing a media encoding / transmission apparatus according to Embodiment 3 of the present invention. In the figure, the same reference numerals as those in FIG.
The video / audio multiplexing unit 22 multiplexes the video encoded data output from the video encoding processing unit 16a and the audio encoded data output from the audio encoding processing unit 16b, and media encoded data (video encoded data + audio A process of outputting the encoded data) to the video / audio sorting unit 23 is performed. The video / audio multiplexing unit 22 constitutes data multiplexing means.
The video / audio sorting unit 23 performs a rearrangement process of the video encoded data and the audio encoded data in the media encoded data output from the video / audio multiplexing unit 22. The video / audio sorting unit 23 constitutes data sorting means.

The video error correction code adding unit 24a applies to the video encoded data in the media encoded data rearranged by the video / audio aligning unit 23 at the additional rate determined by the video error correcting code additional rate determining unit 15a. A process of adding an error correction code is performed.
The audio error correction code adding unit 24b applies to the audio encoded data in the media encoded data rearranged by the video / audio aligning unit 23 at the additional rate determined by the audio error correcting code additional rate determining unit 15b. A process of adding an error correction code is performed.
The video error correction code addition unit 24a and the audio error correction code addition unit 24b constitute error correction code addition means.

  In the example of FIG. 5, the video feature amount extraction processing unit 12a, the audio feature amount extraction processing unit 12b, the video importance level determination unit 13a, the audio importance level determination unit 13b, and the video encoding rate, which are components of the media encoding transmission device. Determination unit 14a, audio encoding rate determination unit 14b, video error correction code addition rate determination unit 15a, audio error correction code addition rate determination unit 15b, video encoding processing unit 16a, audio encoding processing unit 16b, video / audio multiplexing unit 22, each of the video / audio alignment unit 23, the video error correction code addition unit 24a, the audio error correction code addition unit 24b, the data transmission unit 20, and the transmission rate correction unit 21 is mounted with dedicated hardware (for example, a CPU). It is assumed that it is composed of a semiconductor integrated circuit or a one-chip microcomputer. In this case, the video feature extraction processing unit 12a, the audio feature extraction processing unit 12b, the video importance determination unit 13a, the audio importance determination unit 13b, the video encoding rate determination unit 14a, and the audio encoding rate determination unit 14b, video error correction code addition rate determination unit 15a, audio error correction code addition rate determination unit 15b, video encoding processing unit 16a, audio encoding processing unit 16b, video / audio multiplexing unit 22, video / audio alignment unit 23, video error A program describing the processing contents of the correction code adding unit 24a, the voice error correcting code adding unit 24b, the data sending unit 20, and the transmission rate correcting unit 21 is stored in a computer memory, and the CPU of the computer stores the program in the memory. You may make it run the program currently performed.

Next, the operation will be described.
When the video encoding processing unit 16a compresses and encodes video data and outputs video encoded data, and the audio encoding processing unit 16b compresses and encodes audio data and outputs audio encoded data, the video and audio multiplexing unit 22 The video encoded data and the audio encoded data are multiplexed, and the media encoded data (video encoded data + audio encoded data) is output to the video / audio aligning unit 23.
Here, FIG. 6 shows the video encoded data and audio encoded data multiplexed by the video / audio multiplexing unit 22, the video encoded data and audio encoded data rearranged by the video / audio aligning unit 23, It is explanatory drawing which shows the video coding data and audio | voice coding data to which the error correction code was added.

Upon receiving the media encoded data (video encoded data + audio encoded data) as shown in FIG. 6 from the video / audio multiplexing unit 22, the video / audio aligning unit 23 receives the video encoded data in the media encoded data, A rearrangement process of speech encoded data is performed.
Specifically, the rearrangement process as shown below is performed.

For example, when it is specified that one error correction code is added every 5 packets for encoded video data, one error is generated for encoded video data packets [1] to [5]. A correction code is added.
In the example of FIG. 6, two video encoded data packets [6] and [7] are input after the video encoded data packets [1] to [5], but five packets are input. Since the audio encoded data packet [8] has been input before, one error correction code cannot be added to the 5 video encoded data packets.
Therefore, a rearrangement process is performed in which the video encoded data packets [9] to [11] input next to the audio encoded data packet [8] are replaced before the audio encoded data packet [8]. Thus, one error correction code can be added to the five packets [6] [7] [9] [10] [11] of the encoded video data.

However, when an MPEG system is used for transmission, media encoded data includes a signal (PCR: Program Clock Reference) for synchronizing the time on the transmission side and the reception side.
If the transmission order of packets containing this PCR is rearranged, the time synchronization will be shifted on the receiving side. Therefore, if the PCR is included in the packets to be rearranged, The time synchronization on the receiving side is protected by correcting the PCR so that the time synchronization is not shifted.

When the video / audio sorting unit 23 rearranges the video encoded data and the audio encoded data in the media encoded data, the video error correction code adding unit 24a performs the video in the media encoded data subjected to the rearrangement. An error correction code is added to the encoded data at the addition rate IA _rate determined by the video error correction code addition rate determination unit 15a.
In FIG. 6, for example, one error correction code is added to video encoded data packets [1] to [5], and video encoded data packets [6] [7] [9] [9] [ 10] [11], one error correction code is added.

When the video / audio sorting unit 23 rearranges the video encoded data and the audio encoded data in the media encoded data, the audio error correcting code adding unit 24b performs the audio in the media encoded data subjected to the rearrangement. An error correction code is added to the encoded data at the addition rate VA _rate determined by the voice error correction code addition rate determination unit 15b.
In the example of FIG. 6, one error correction code is added to the voice encoded data packet [8].
Since the processing contents of the data sending unit 20 and the transmission rate correction unit 21 are the same as those in the second embodiment, description thereof is omitted.

  As apparent from the above, according to the third embodiment, the video / audio sorting unit 23 rearranges the video encoded data and the audio encoded data in the media encoded data output from the video / audio multiplexing unit 22. Since it is configured to be applied, it is applicable even when video encoded data and audio encoded data are multiplexed after encoding video data and audio data and before adding an error correction code as in an MPEG system, etc. can do.

Embodiment 4 FIG.
In the fourth embodiment, the feature quantity extraction unit 12 in FIG. 1 (including the video feature quantity extraction unit 12a and the audio feature quantity extraction unit 12b in FIGS. 4 and 5) is used as the feature quantity of the media data to be encoded. It is confirmed whether the media data is information constituting the header part or information constituting the data part, and the importance level discrimination unit 13 in FIG. 1 (video importance level discrimination in FIGS. 4 and 5). Section 13a and voice importance level determination section 13b), if the media data to be encoded is information constituting the header section, it is determined that the importance level of the media data is “high”. If the media data to be converted is information constituting a data part, it is determined that the importance of the media data is “medium” or “low”.

Specifically, it is as follows.
For example, information to be actually transmitted such as video data and audio data is stored in the data portion of the media data, and the type and data configuration of the information stored in the data portion are stored in the header portion.
As the type of header portion, for example, in the MPEG system, a GOP (Group of Pictures) in which a sequence header in which the number of vertical and horizontal pixels of a video, a frame rate, and the like are stored, a time from the beginning of the sequence, and the like are stored. There are headers.
If the header part is missing, the type and configuration of the information stored in the data part will not be known, and even if the data part is received normally, it will not be possible to decode the media data. High importance.

Therefore, the feature quantity extraction unit 12 confirms whether the media data is information constituting the header part or information constituting the data part as the feature quantity of the media data to be encoded. .
The importance determination unit 13 determines that the importance of the media data is “high” if the media data to be encoded is information constituting the header portion, and the media data to be encoded is data. If the information constitutes a section, it is determined that the importance of the media data is “medium” or “low”.

  As is apparent from the above, according to the fourth embodiment, when determining the importance of the media data to be encoded, whether the media data is information constituting the header portion or the data portion is configured. To determine whether the media data is the information constituting the header part, and to set the importance higher than the importance of the information constituting the data part. Thus, there is an effect that the error correction capability of the header part can be increased and the quality of the decoding media can be improved.

Embodiment 5 FIG.
In the fifth embodiment, when the media data to be encoded is video data, the feature quantity extraction unit 12 in FIG. 1 (including the video feature quantity extraction unit 12a in FIGS. 4 and 5) is the video to be encoded. As the data feature amount, the picture type of the video data is confirmed, and the importance level determination unit 13 in FIG. 1 (including the video importance level determination unit 13a in FIGS. 4 and 5) uses the picture type as a reference, The importance of the video data is determined.

Specifically, it is as follows.
For example, when MPEG-2 is used for compressing and encoding video data, if an I picture is lost, decoding of a P picture or a B picture that refers to the I picture is also affected. It can be said that the data is more important than the picture and the B picture.
In addition, if a P picture is missing, it also affects the decoding of a B picture that refers to the P picture, and therefore it can be said that the P picture is data that is more important than the B picture.

Therefore, the feature amount extraction unit 12 confirms the picture type of the video data as the feature amount of the video data to be encoded.
When the feature amount extraction unit 12 confirms the picture type of the video data, the importance level determination unit 13 determines the importance level of the video data on the basis of the picture type.
For example, if the picture type of the video data is I picture, it is determined that the importance of the video data is “high”. If the picture type of the video data is P picture, the importance of the video data is “ If the picture type of the video data is a B picture, it is determined that the importance of the video data is “low”.

  As apparent from the above, according to the fifth embodiment, when the media data to be encoded is video data, the picture type of the video data is confirmed, and the video data is based on the picture type. Therefore, the image quality of the decoded video can be improved.

Embodiment 6 FIG.
In the sixth embodiment, when the media data to be encoded is video data, the feature quantity extraction unit 12 in FIG. 1 (including the video feature quantity extraction unit 12a in FIGS. 4 and 5) is the video to be encoded. As a data feature amount, a motion vector used when the video data is compression-encoded is confirmed, and the importance determination unit 13 in FIG. 1 (including the video importance determination unit 13a in FIGS. 4 and 5). The importance of the video data is determined based on the motion vector.

Specifically, it is as follows.
For example, in a video compression encoding method such as MPEG, motion compensation interframe prediction is performed as one of the compression encoding methods.
When motion compensation inter-frame prediction is performed, a picture having a large movement amount (motion vector) means that the change from the reference picture is large. Therefore, if this picture is missing, the picture that refers to this picture This will affect the decoded video.
For this reason, it can be said that a picture with a large motion vector is more important than a picture without a large motion vector.
Thus, for example, by increasing the importance of a picture having a large average value of motion vector sizes, a picture having a high importance and a picture having a low importance are determined among P pictures and B pictures. Thus, the encoding rate and the error correction code addition rate can be controlled more finely.

Therefore, the feature quantity extraction unit 12 confirms a motion vector used when the video data is compression-coded as the feature quantity of the video data to be encoded. When confirming a motion vector, the feature quantity extraction unit 12 may implement a motion vector search unit that searches for a motion vector, or the encoding processing unit 16 of FIG. 1 (video encoding of FIGS. 4 and 5). The motion vector used when the compression encoding is performed may be acquired from the processing unit 16a).
The importance level determination unit 13 determines the importance level of the video data based on the motion vector confirmed by the feature amount extraction unit 12.
For example, the importance of the picture is set higher as the picture (video data) has a larger average value of the motion vectors.

  As is apparent from the above, according to the sixth embodiment, when the media data to be encoded is video data, the motion data used when the video data is compressed and encoded is used as a reference. Since the configuration is such that the importance of the video data is discriminated, the encoding rate and the additional rate can be finely controlled, and as a result, the image quality of the decoded video can be improved.

  In the sixth embodiment, the higher the importance of the picture is set as the picture (video data) having a larger average value of the motion vectors is shown. However, the present invention is not limited to this. For example, the importance of the video data may be determined according to the number of motion vectors having a magnitude equal to or greater than a certain threshold, the number of motion vectors pointing in a certain direction, and the like.

Embodiment 7 FIG.
In the seventh embodiment, when the media data to be encoded is video data, the feature quantity extraction unit 12 in FIG. 1 (including the video feature quantity extraction unit 12a in FIGS. 4 and 5) is the video to be encoded. The DCT coefficient (discrete cosine transform coefficient) calculated when the video data is compression-encoded is confirmed as the data feature amount, and the importance level discrimination unit 13 in FIG. 1 (video importance level in FIGS. 4 and 5) is confirmed. (Including the discriminator 13a) discriminates the importance of the video data on the basis of the DCT coefficient.

Specifically, it is as follows.
For example, in a video compression encoding system such as MPEG, in order to reduce redundancy in the spatial direction, video data to be encoded is DCT-converted and the DCT coefficient that is the conversion result is quantized.
At this time, a picture having a large value (especially a large value for the high frequency component) in the AC component of the DCT coefficient means that the image in the picture is complicated. Therefore, the distribution of the DCT coefficient is calculated, If the importance of a picture having many values is set high, the error correction capability of a complex image can be enhanced.

Therefore, the feature amount extraction unit 12 confirms the DCT coefficient calculated when the video data is compression-encoded as the feature amount of the video data to be encoded. When confirming the DCT coefficient, the feature amount extraction unit 12 is mounted with, for example, a DCT device that calculates the DCT coefficient by DCT conversion of difference data between the video data to be encoded and the video data of the reference image. Alternatively, the DCT coefficient calculated when the compression encoding is performed from the encoding processing unit 16 of FIG. 1 (including the video encoding processing unit 16a of FIGS. 4 and 5) may be acquired.
The importance level determination unit 13 determines the importance level of the video data based on the DCT coefficient confirmed by the feature amount extraction unit 12.
For example, the importance of the picture is set higher as the picture (video data) has a larger value in the AC component of the DCT coefficient.

  As is apparent from the above, according to the seventh embodiment, when the media data to be encoded is video data, it is calculated as the feature amount of the video data when the video data is compressed and encoded. The DCT coefficient is determined and the importance of the video data is determined based on the DCT coefficient, so that the encoding rate and the additional rate can be finely controlled. There is an effect that the image quality of the decoded video can be improved.

Embodiment 8 FIG.
In the eighth embodiment, when determining the importance of the media data to be encoded, the feature quantity extraction unit 12 in FIG. 1 confirms whether or not the media data includes audio data. The importance determination unit 13 determines that the importance of the media data when the audio data is included is higher than the importance of the media data when the audio data is not included.

Specifically, it is as follows.
Video data is often used together with audio data, and video data is often transmitted with audio data superimposed thereon.
Since the amount of audio data per unit time is very small compared to video data, if it is video data, even if it is audio data even if there is a lack of data amount that degrades the image quality of a part of the decoded video, There is a possibility of missing such that the voice cannot be decoded.
That is, the influence of one data loss on decoded audio is larger than the influence of one data loss on decoded video.
Therefore, if the importance of media data including audio data is determined to be high, the transmission error correction capability of audio data can be increased and the quality of decoded audio can be maintained.

Therefore, the feature quantity extraction unit 12 checks whether the media data includes audio data as the feature quantity of the media data to be encoded.
The importance determination unit 13 sets the importance of the media data when the media data to be encoded includes audio data higher than the importance of the media data when it does not include audio data.

  As is apparent from the above, according to the eighth embodiment, when determining the importance of the media data to be encoded, it is confirmed whether or not the media data includes audio data. Since the importance of media data is set higher than the importance of media data when audio data is not included, the transmission error correction capability of audio data is increased and the quality of decoded audio is increased. There is an effect that can be maintained.

Embodiment 9 FIG.
In the above first to eighth embodiments, an error correction code is added to enable error correction on the receiving side. However, the error correction code is redundant data and a transmission error occurs. Otherwise, media data with higher image quality can be transmitted if the band used by the error correction code is allocated to the media encoded data.
Therefore, if the transmission condition is good, it is desirable to increase the transmission efficiency by reducing the error correction code addition rate, and increase the error correction capability by increasing the error correction code addition rate if the transmission condition is degraded. .

FIG. 7 is a block diagram showing a media encoding / transmission apparatus according to Embodiment 9 of the present invention. In the figure, the same reference numerals as those in FIG.
The line status acquisition unit 25 performs a process of collecting the line status (network communication status) from other devices in the network 3 such as a receiving device. The line status acquisition unit 25 constitutes a communication status collection unit.

The transmission rate correction unit 26 performs the same processing as the transmission rate correction unit 21 of FIG. 4 and determines an instruction to determine the encoding rate according to the line status collected by the line status acquisition unit 25. 14a and the audio coding rate determining unit 14b, and processing for instructing the video error correcting code additional rate determining unit 15a and the audio error correcting code additional rate determining unit 15b to determine an additional rate according to the line status. carry out. The transmission rate correction unit 26 constitutes a rate correction unit.
7 shows an example in which the line status acquisition unit 25 and the transmission rate correction unit 26 are mounted on the media encoding transmission apparatus of FIG. 4, the line status acquisition unit 25 and the transmission rate correction unit 26 are shown in FIG. Or you may make it mount in the media encoding transmission apparatus of FIG.

  In the example of FIG. 7, the video feature amount extraction processing unit 12a, the audio feature amount extraction processing unit 12b, the video importance level determination unit 13a, the audio importance level determination unit 13b, and the video encoding rate, which are components of the media encoding transmission apparatus. Determination unit 14a, audio encoding rate determination unit 14b, video error correction code addition rate determination unit 15a, audio error correction code addition rate determination unit 15b, video encoding processing unit 16a, audio encoding processing unit 16b, video error correction code Each of the addition unit 17a, the voice error correction code addition unit 17b, the data transmission unit 20, the line status acquisition unit 25, and the transmission rate correction unit 26 has dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, or One-chip microcomputer, etc.) is assumed, but when the media encoding transmission device is configured with a computer, Image feature extraction processing unit 12a, audio feature extraction processing unit 12b, video importance determination unit 13a, audio importance determination unit 13b, video encoding rate determination unit 14a, audio encoding rate determination unit 14b, video error correction code Additional rate determination unit 15a, audio error correction code addition rate determination unit 15b, video encoding processing unit 16a, audio encoding processing unit 16b, video error correction code addition unit 17a, audio error correction code addition unit 17b, data transmission unit 20 The program describing the processing contents of the line status acquisition unit 25 and the transmission rate correction unit 26 may be stored in a computer memory, and the CPU of the computer may execute the program stored in the memory. .

Next, the operation will be described.
For example, the line status acquisition unit 25 collects the line status from other devices in the network 3 such as a receiving side device and notifies the transmission rate correction unit 26 of the line status.
Specifically, for example, when the transmission path is an IP network, RTCP (Real-Time Transfer Control Protocol) defined in “IETF RFC3550“ RTP: A Tranport Protocol for Real-Time Applications ”” is used. The line status can be collected.
That is, when RTCP is used, the line status acquisition unit 25 receives a report (RR: Receiver Report) from the receiving side apparatus with respect to a report (SR: Sender Report) of the media encoding transmission apparatus 1 that is the transmitting side apparatus. Thus, as the line status of the network 3, the packet loss rate, the cumulative number of lost packets, the packet interval jitter, etc. in the network 3 can be collected.

The transmission rate correction unit 26 performs the same processing as the transmission rate correction unit 21 in FIG. 4 in order to maintain the transmission rate constant, but the encoding rate and the additional rate are set according to the line status of the network 3. Control.
For example, if the line status of the network 3 notified from the line status acquisition unit 25 indicates that there is no packet loss, the transmission rate correction unit 26 decreases the error correction code addition rates IA _rate and VA _rate . The correction instruction is output to the video error correction code addition rate determination unit 15a and the audio error correction code addition rate determination unit 15b so that the transmission efficiency is increased.
On the other hand, if the line status of the network 3 notified from the line status acquisition unit 25 indicates that a packet is missing, a correction instruction for increasing the additional rates IA _rate and VA _rate of the error correction code is displayed. The error correction code addition rate determination unit 15a and the voice error correction code addition rate determination unit 15b are output to improve the error correction capability.

As is apparent from the above, according to the ninth embodiment, the line status acquisition unit 25 that collects the line status of the network 3 is provided, and the transmission rate correction unit 26 collects the line status collected by the line status acquisition unit 25. Since the video error correction code addition rate determination unit 15a and the voice error correction code addition rate determination unit 15b are instructed to determine the addition rates IA _rate and VA _rate according to the network, the network status of the network 3 is good. If there is, the transmission efficiency can be increased. On the other hand, if the line status of the network 3 is deteriorated, the error correction capability can be increased.

Embodiment 10 FIG.
In the ninth embodiment, an example in which the addition rates IA _rate and VA _rate of the error correction code are increased or decreased according to the line status of the network 3 has been described. Since the code amount of the error correction code is increased, the code amount of the media encoded data is decreased, and the media quality is uniformly reduced regardless of the importance of the media data.

Therefore, in the tenth embodiment, when the line status of the network 3 is deteriorated, the transmission rate correction unit 26 re-discriminates the importance of the media data by the video importance determining unit 13a and the audio importance determining unit 13b. I'm trying to tell you.
As a result, the video importance level determination unit 13a redetermines the importance level of the video data, and the audio importance level discrimination unit 13b redetermines the importance level of the audio data.

As a result, when the line condition is deteriorated, the transmission time of media data with high importance is set longer by the amount corresponding to the increase of the additional rates IA _rate and VA _rate of the error correction code, and the medium with low importance is set. By shortening the data transmission time, it is possible to realize an optimum rate distribution capable of more reliably transmitting media data having a high degree of importance.

  DESCRIPTION OF SYMBOLS 1 Media encoding transmission apparatus, 2 Media source, 3 Network, 11 Media compression encoding process part, 11a Video compression encoding process part, 11b Voice compression encoding process part, 12 Feature-value extraction process part (importance discrimination means) , 12a Video feature amount extraction processing unit (importance level determination unit), 12b Audio feature amount extraction processing unit (importance level determination unit), 13 Importance level determination unit (importance level determination unit), 13a Video importance level determination unit (importance level) Discriminating means), 13b audio importance discriminating section (importance discriminating means), 14 encoding rate determining section (rate determining means), 14a video encoding rate determining section (rate determining means), 14b audio encoding rate determining section ( Rate determination means), 15 error correction code addition rate determination section (rate determination means), 15a video error correction code addition rate determination section (rate determination means) , 15b Audio error correction code addition rate determining unit (rate determining unit), 16 encoding processing unit (media data encoding unit), 16a video encoding processing unit (media data encoding unit), 16b audio encoding processing unit ( Media data encoding means), 17 error correction code addition section (error correction code addition means), 17a, 24a Video error correction code addition section (error correction code addition means), 17b, 24b Audio error correction code addition section (error correction) Sign adding means), 18, 20 data sending section (data sending means), 19, 21, 26 transmission rate correcting section (rate correcting means), 22 video / audio multiplexing section (data multiplexing means), 23 video / audio aligning section ( Data alignment means), 25 line status acquisition unit (communication status collection means).

Claims (10)

  An importance level discriminating unit that discriminates the degree of importance of media data to be encoded, and an encoding rate for compressing and encoding the media data based on the importance level discriminated by the importance level discriminating unit. And a rate determining means for determining an additional rate of an error correction code to be added to the media data after compression encoding, and compression encoding by encoding the media data at the encoding rate determined by the rate determining means. An error at an additional rate determined by the rate determining means with respect to the media encoded data output from the media data encoding means; Error correction code adding means for adding a correction code and outputting media encoded data with an error correction code; Data transmission means for sending media encoded data with error correction code output from error correction code adding means to the network, and transmission rate of media encoded data with error correction code sent from the data transmission means are allowed A media encoding / transmission apparatus comprising: rate correction means for instructing the rate determination means to correct the encoding rate and the additional rate when deviating from the range, and to keep the transmission rate within an allowable range .   The importance discriminating means for discriminating the importance of the video data which is the media data to be encoded, while determining the importance of the audio data which is the media data to be encoded, and the video discriminated by the importance discriminating means. Based on the importance of the data, the coding rate for compressing and encoding the video data is determined, and the addition rate of the error correction code to be added to the video data after compression encoding is determined. Determine the encoding rate when compressing and encoding the audio data based on the importance of the audio data determined by the degree determination means, and add an error correction code to be added to the audio data after compression encoding A rate determining means for determining a rate, and a video after compression encoding the video data at the encoding rate determined by the rate determining means; Video encoded data, which is data, and at the same time, the audio data is compressed and encoded at the encoding rate determined by the rate determining means, and audio encoded data which is audio data after compression encoding is output. An error correction code is added to the media data encoding means and the encoded video data output from the media data encoding means at an additional rate determined by the rate determination means, and a video code with an error correction code is added. On the other hand, an error correction code is added to the audio encoded data output from the media data encoding means at an additional rate determined by the rate determining means, and an audio code with an error correction code is output. Error correction code adding means for outputting encoded data, and video coding data with an error correction code output from the error correction code adding means. Data transmission means for transmitting voice encoded data with a data and error correction code to a network, and transmission of video encoded data with error correction code and voice encoded data with error correction code transmitted from the data transmission means If the rate deviates from the allowable range, the media code is provided with rate correction means for instructing the rate determination means to correct the encoding rate and the additional rate, and to keep the transmission rate within the allowable range. Transmission equipment.   The importance discriminating means for discriminating the importance of the video data which is the media data to be encoded, while determining the importance of the audio data which is the media data to be encoded, and the video discriminated by the importance discriminating means. Based on the importance of the data, the coding rate for compressing and encoding the video data is determined, and the addition rate of the error correction code to be added to the video data after compression encoding is determined. Determine the encoding rate when compressing and encoding the audio data based on the importance of the audio data determined by the degree determination means, and add an error correction code to be added to the audio data after compression encoding A rate determining means for determining a rate, and a video after compression encoding the video data at the encoding rate determined by the rate determining means; Video encoded data, which is data, and at the same time, the audio data is compressed and encoded at the encoding rate determined by the rate determining means, and audio encoded data which is audio data after compression encoding is output. Media data encoding means, and media encoding in which video encoded data and audio encoded data output from the media data encoding means are multiplexed, and the video encoded data and audio encoded data are multiplexed Data multiplexing means for outputting data, data alignment means for rearranging video encoded data and audio encoded data in the media encoded data output from the data multiplexing means, and alignment by the data alignment means The rate determining means applies the video encoded data and the audio encoded data in the media encoded data subjected to the replacement. An error correction code adding means for adding an error correction code at the determined addition rate; a data sending means for sending media encoded data to which an error correction code has been added by the error correction code adding means to the network; and the data When the transmission rate of the media encoded data with the error correction code sent from the sending means is out of the allowable range, the rate determining means is instructed to correct the coding rate and the additional rate, and A media encoding transmission apparatus comprising rate correction means for keeping a transmission rate within an allowable range.   The importance determining means, when determining the importance of the media data to be encoded, confirms whether the media data is information constituting the header part or information constituting the data part. The importance level when the media data is information constituting the header portion is set higher than the importance level when the media data is information constituting the data portion. 4. The media encoding transmission apparatus according to claim 1.   The importance level determination means is characterized in that, when the media data to be encoded is video data, the picture type of the video data is confirmed, and the importance level of the video data is determined based on the picture type. The media encoding transmission apparatus according to any one of claims 1 to 3.   The importance determining means determines the importance of the video data based on a motion vector used when the video data is compression encoded when the media data to be encoded is video data. The media encoding transmission apparatus according to any one of claims 1 to 3, wherein the media encoding transmission apparatus is any one of claims 1 to 3.   The importance determining means determines the importance of the video data based on a discrete cosine transform coefficient calculated when the video data is compressed and encoded when the media data to be encoded is video data. The media encoding transmission apparatus according to any one of claims 1 to 3, characterized in that:   When determining the importance of the media data to be encoded, the importance determination means checks whether the media data includes audio data, and determines the importance of the media data when the audio data includes the audio data. 2. The media encoding transmission apparatus according to claim 1, wherein the media encoding transmission device is set to be higher than the importance of media data when no audio data is included.   A communication status collection unit that collects the communication status of the network is provided, and the rate determination unit instructs the rate determination unit to determine an additional rate according to the communication status of the network collected by the communication status collection unit. The media encoding transmission apparatus according to any one of claims 1 to 8.   The rate correction unit instructs the importance level determination unit to redeterminate the importance of the media data when the network communication status collected by the communication status collection unit is deteriorated from the allowable status. 9. The media encoding transmission apparatus according to 9.

Images