JP6708882B2 - Encoding device and encoding method - Google Patents

Description

The present invention relates to an encoding device and an encoding method.

When transferring moving image content over a network, compression techniques are used that reduce the amount of communication on the network. In the compression technique, a transmitting device compresses and encodes a frame forming a content and transmits the frame to a network, and a receiving device receives and decodes the compression-encoded data to restore the content.

There is a differential encoding method that improves the compression rate in the compression technology. In the differential encoding method, a part of the frame included in the content is intraframe encoded as a key frame. In addition, the difference from the key frame is encoded, that is, inter-frame encoding, with the remaining part of the frame included in the content excluding the part as the difference frame.

Japanese Patent Application Laid-Open No. 2004-242242 discloses a compression encoding technique that suppresses a rapid increase in the encoding processing amount of a key frame, and a difference that appears during reproduction between an intra-frame encoded frame and an inter-frame encoded frame. A technique for reducing the above is disclosed.

JP, 2002-171527, A

The technique disclosed in Patent Document 1 introduces a concept of a key GOB (block area) in addition to the key frame, so that an effect of suppressing a change in the encoding processing amount is expected, while the encoded data of the encoded data is expected. It is indifferent to the data size. For example, an inter-frame coded frame having a data amount corresponding to the difference from the key frame immediately before the target frame to be coded, even when the video does not move (or almost does not move) or when the video moves rapidly. Is generated and sent to the network.

Therefore, there is a problem that the transmitted frame exerts pressure on the communication amount of other communication performed in the network, or adversely affects other communication due to communication loss that may occur in the network.

The present invention has been made to solve the above problem, and an object of the present invention is to provide an encoding device and the like that reduce the data amount of encoded data transmitted to a network.

In order to solve the above problems, an encoding device according to an aspect of the present invention provides an encoding unit that performs intra-frame encoding or inter-frame encoding of frames included in a series of frames, and the encoding unit performs encoding. A control unit for controlling whether or not to output the converted frame, wherein the control unit includes a first frame of the series of frames and a second frame that is a frame next to the first frame. and when the encoding unit both have been inter-frame coding, when (a) encoded data size of the second frame, not belonging to a predetermined range including the data size of the first frame which is encoded To output the encoded second frame, and (b) when the data size of the encoded second frame belongs to the predetermined range, the encoded second frame is output. Control not to output.

According to this, the encoding device determines that if two consecutive frames which are the same or substantially the same image in a series of frames are inter-frame encoded, two of the two frames are The second frame does not send the encoded data encoded in the network to the network. Since these two frames have the same or substantially the same image, even if the encoded data of the second frame is not transmitted, the influence on the reproduced content is relatively small. Thereby, the encoding device can reduce the data amount of the encoded data transmitted to the network.

Further, in (a), the control unit is configured to code the second frame when the data size of the coded second frame is different from the data size of the coded first frame. controls to output, in the (b), the encoded data size of the second frame, when matching the encoded data size of the first frame, the second coded Control not to output the frame.

According to this, the encoding device encodes the second frame of the two frames when the two consecutive frames, which are the same image, are inter-frame encoded. The encoded data that has been encoded is not transmitted to the network. Thereby, the encoding device can reduce the data amount of the encoded data transmitted to the network.

In addition, the control unit, when the third frame of the series of frames is intra-coded by the encoding unit, a fourth frame which is a frame next to the third frame of the series of frames. The frame is controlled to be interframe encoded.

According to this, the encoding device can reduce the size of the encoded data by performing inter-frame encoding on the frame after the intra-frame encoded frame.

Further, the control unit, when the data size of a predetermined number of consecutively interframe-encoded frames of the series of frames is equal to or smaller than a threshold value, sets the frame next to the predetermined number of frames as an intraframe. Control to encode.

According to this, even when the encoder continuously controls the transmission of the inter-frame coded frame to be transmitted, the intra-frame coded frame is generated for each frame of the predetermined value. Outputs encoded data including. As a result, even if the encoded data is lost during communication on the network, the reproduction device can continue to reproduce the content by the intra-frame encoded frame to be transmitted next. ..

Further, when controlling not to output the second frame, the control unit controls to output control information indicating that there is no change in the frame.

According to this, when the decoding device which receives the encoded data transmitted by the encoding device does not receive the encoded data for a certain period of time, it is possible to prevent the reproduction device from stopping the reproduction of the content. This is because, if the reproduction device does not receive the encoded data for a certain period of time, it may determine that the reproduction of the content cannot be continued due to a problem in the transmission device or the network and may stop the reproduction.

Further, a coding method according to an aspect of the present invention outputs a coding step of performing intra-frame coding or inter-frame coding of frames included in a series of frames, and a frame coded in the coding step. And a step of controlling whether or not the first frame of the series of frames, and a second frame that is a frame next to the first frame, the encoding step. When the inter-frame coding is performed with (a), when the encoded data size of the second frame does not belong to a predetermined range including the encoded data size of the first frame, the encoded data size is encoded. The second frame is controlled to be output, and (b) the encoded second frame is controlled not to be output when the encoded data size of the second frame falls within the predetermined range.

As a result, the same effect as that of the above-described encoding device can be obtained.

The encoding device and the like according to the present invention can reduce the data amount of encoded data transmitted to the network.

FIG. 1 is a configuration diagram showing a configuration of a content reproduction system according to an embodiment. FIG. 2 is a block diagram showing a hardware configuration of the encoding device according to the embodiment. FIG. 3 is a block diagram showing a configuration of an encoding device according to a related technique. FIG. 4 is a flowchart showing the processing of the encoding device according to the related art. FIG. 5 is a block diagram showing a configuration of a decoding device according to a related technique. FIG. 6 is a flowchart showing the processing of the decoding device according to the related art. FIG. 7 is a block diagram showing the configuration of the encoding device according to the embodiment. FIG. 8 is an explanatory diagram of an example of slide show contents including still images. FIG. 9 is a flowchart showing the processing of the encoding device according to the embodiment. FIG. 10 is a flowchart showing the encoding process of the encoding device according to the embodiment. FIG. 11 is a flowchart showing the coding control process of the coding device according to the embodiment. FIG. 12 is a first explanatory diagram for explaining the effect of still image detection by the encoding device according to the embodiment. FIG. 13 is a second explanatory diagram for explaining the effect of still image detection by the encoding device according to the embodiment.

Hereinafter, embodiments will be specifically described with reference to the drawings.

Each of the embodiments described below shows a preferred specific example of the present invention. Numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of constituent elements, steps, order of steps, and the like shown in the following embodiments are examples, and are not intended to limit the present invention. Further, among the constituent elements in the following embodiments, the constituent elements that are not described in the independent claims showing the highest concept of the present invention will be described as arbitrary constituent elements that constitute a more preferable form. In addition, the same components may be denoted by the same reference numerals and the description thereof may be omitted.

(Embodiment)
In the present embodiment, an encoding device and the like for reducing the data amount of encoded data transmitted to the network will be described.

FIG. 1 is a configuration diagram showing a configuration of a content reproduction system S according to this embodiment.

As shown in FIG. 1, the content reproduction system S includes a transmission device 1 and a reproduction device 2. The transmission device 1 and the reproduction device 2 are communicably connected via a network 3.

The content reproduction system S is a content reproduction system that reproduces moving image content (hereinafter, also simply referred to as content) held by the transmission device 1 by one or more reproduction devices 2.

The transmission device 1 is a transmission device that holds content and transmits the content to the playback device 2 through the network 3. The transmission device 1 includes an encoding device 10 that compresses and encodes content (hereinafter, also simply referred to as encoding). The encoding device 10 generates encoded data in which the content data amount is reduced by encoding, and transmits the generated encoded data to the network 3.

The reproduction device 2 is a reproduction device that receives content from the transmission device 1 through the network 3 and reproduces the received content. The playback device 2 receives, via the network 3, the encoded data generated by the encoding device 10 based on the content held by the transmission device 1. The playback device 2 includes a decoding device 20 that decodes encoded data. The decoding device 20 restores the content by decoding the encoded data received from the network 3, and the playback device 2 reproduces the restored content.

The network 3 is a network that connects the transmitting device 1 and the reproducing device 2 in a communicable manner. The network 3 is specifically a wireless LAN (Local Area Network) conforming to the IEEE 802.11a, b, g, n standards or the like, or a wired LAN conforming to the IEEE 802.3 standard or the like.

Note that the content reproduction system S may include two or more reproduction devices 2. In that case, it is necessary that each of two or more playback devices 2 is connected to the network 3 and can receive the encoded data transmitted by the transmission device 1. When there are two or more playback devices 2, the transmission device 1 may perform unicast transmission to each of the two or more playback devices 2, or perform broadcast transmission so that the two or more playback devices 2 can receive. May be.

FIG. 2 is a block diagram showing the hardware configuration of encoding apparatus 10 according to the present embodiment. The decoding device 20 also has the same hardware configuration as the encoding device 10.

As shown in FIG. 2, the encoding device 10 includes a CPU 11, a main memory 12, and a storage 13.

The CPU 11 is a processor that executes a control program stored in the main memory 12 or the storage 13. The processing executed by the encoding device 10 can be realized by the CPU 11 executing the control program.

The main memory 12 is a volatile storage area used as a work area used when the CPU 11 executes a control program.

The storage 13 is a non-volatile storage device that stores data of frames that form content. The storage 13 is, for example, a HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and may be built in the encoding device 10 or may be a USB (Universal Serial Bus) standard or the like. It may be externally attached.

Hereinafter, configurations of the coding apparatus and the decoding apparatus according to the present embodiment will be described. Regarding the encoding device, the encoding device according to the present embodiment will be described after first describing the encoding device according to the related art. Here, the related technique is an example of a conventional technique in which the communication amount of other communication performed in the network may be squeezed or the communication loss that may occur in the network may adversely affect the other communication. .. Regarding the decoding device, there is no difference between the decoding device according to the related art and the decoding device of the present embodiment.

FIG. 3 is a block diagram showing the configuration of the encoding device 50 according to the related art. Encoding device 50 is arranged at the position of encoding device 10 in the present embodiment.

As illustrated in FIG. 3, the encoding device 50 includes a frame holding unit 100, an acquisition unit 101, a blocking filter 102, a conversion unit 103, a difference unit 104, a quantization unit 105, and an encoding unit 106. And an inverse quantization unit 107 and a reference holding unit 108.

The frame holding unit 100 is a storage unit that holds the data of frames that form the content to be encoded. The frame holding unit 100 is realized by, for example, the main memory 12 or the like.

The acquisition unit 101 is a processing unit that acquires an encoding target frame. The acquisition unit 101 acquires a frame to be encoded among the frames included in the content stored in the frame holding unit 100 by reading. As the encoding target frame, for example, among a plurality of frames forming the content, a frame associated with the current time information managed by the encoding device 50 is sequentially allocated.

The blocking filter 102 is a processing unit that performs a blocking filter process on a block boundary that is a processing unit included in the encoding target frame acquired by the acquisition unit 101. The blocking filter process is a process to be performed before the conversion process in order to suppress a discontinuity that may occur at a block boundary that is a processing unit of the conversion process (described later).

The conversion unit 103 is a processing unit that performs conversion processing for each block on the frame after the blocking filter 102 has performed the blocking filter processing. An example of the conversion processing is frequency conversion processing. In the frequency conversion, the encoding target frame is converted into a coefficient (frequency coefficient) when it is expressed as an addition of a plurality of trigonometric functions that are bases. In addition, as the conversion processing, order conversion (for example, Hadamard conversion) processing or the like can be adopted.

The difference unit 104 calculates the difference between the encoding target frame (frequency coefficient) after the conversion unit 103 has performed the conversion process and the reference frame held by the reference holding unit 108 to obtain the difference frame. Is a processing unit that generates The difference unit 104 reads the reference frame from the reference holding unit 108 before calculating the difference. The difference unit 104 calculates the difference when the encoding target frame is not the key frame, and does not calculate the difference when the encoding target frame is the key frame. Note that whether the encoding target frame is a key frame or not a key frame may be determined by the time when the difference unit 104 acquires the encoding target frame from the conversion unit 103, or some control may be performed. The difference unit 104 may make the determination based on this. This control includes, for example, control by the control unit 111 described later.

The timing at which the difference unit 104 reads out the reference frame comes after the timing at which the acquisition unit 101 reads out the encoding target frame from the frame holding unit 100. This is because the acquisition unit 101 reads out the coding target frame from the frame holding unit 100, the processes of the blocking filter 102 and the conversion unit 103 are executed, and then the difference unit 104 reads out the reference frame. In other words, it can be said that the blocking filter 102 and the conversion unit 103 function as a delay unit that delays the reading of the reference frame by the difference unit 104 with respect to the reading of the encoding target frame by the acquisition unit 101.

The quantization unit 105 quantizes the encoding target frame after the conversion process by the conversion unit 103 or the difference frame generated by the difference unit 104 (that is, converts the frame into a quantized coefficient). It is a department.

The encoding unit 106 is a processing unit that generates encoded data by encoding the frame quantized by the quantization unit 105. The encoded data generated by the encoding unit 106 corresponds to the content data transmitted by the transmission device 1.

The inverse quantization unit 107 is a processing unit that inversely quantizes the frame quantized by the quantization unit 105. The inverse quantization unit 107 performs inverse quantization when the frame quantized by the quantization unit 105 is a key frame. The dequantization unit 107 stores the dequantized frame in the reference holding unit 108 as a reference frame.

The reference holding unit 108 is a storage unit that holds the key frame dequantized by the dequantization unit 107 as a reference frame. The key frame held by the reference holding unit 108 is referred to by the difference unit 104 and used in the process of calculating the difference. The reference holding unit 108 is realized by, for example, the main memory 12.

The processing of the encoding device 50 configured as above will be described.

FIG. 4 is a flowchart showing the processing of the encoding device 50 according to the related technique.

In step S101, the acquisition unit 101 acquires the encoding target frame by reading it from the frame holding unit 100.

In step S102, the blocking filter 102 performs a blocking filter process on the block boundary in the encoding target frame acquired by the acquisition unit 101.

In step S103, the conversion unit 103 performs conversion processing for each block on the frame after the blocking filter processing by the blocking filter 102.

In step S104, the encoding device 50 branches the process depending on whether the encoding target frame is a key frame. If the frame to be encoded is a key frame (Yes in step S104), the process proceeds to step S105, and if not (No in step S104), the process proceeds to step S111.

In step S105, the quantization unit 105 performs quantization processing on the encoding target frame after the conversion processing by the conversion unit 103.

In step S106, the inverse quantization unit 107 performs inverse quantization processing on the key frame quantized by the quantization unit 105.

In step S107, the dequantization unit 107 stores the frame dequantized in step S106 in the reference holding unit 108.

In step S111, the difference unit 104 generates, as a difference frame, a difference between the encoding target frame after the conversion processing by the conversion unit 103 and the reference frame read from the reference holding unit 108.

In step S112, the quantization unit 105 quantizes the difference frame that has been subjected to the conversion processing by the difference unit 104.

In step S108, the encoding unit 106 encodes the key frame quantized by the quantization unit 105 to generate encoded data. The generated encoded data is transmitted to the reproducing device 2 by the transmitting device 1.

FIG. 5 is a block diagram showing the configuration of decoding apparatus 20 according to the present embodiment.

As shown in FIG. 5, the decoding device 20 includes a decoding unit 201, an addition unit 202, an inverse quantization unit 203, a reference holding unit 204, an inverse transformation unit 205, a deblocking filter 206, and a generation unit. And section 207.

The decoding unit 201 is a processing unit that receives and decodes encoded data, which is content data transmitted by the transmission device 1, as a decoding target frame. The frame generated by decoding by the decoding unit 201 is a coding target frame (quantization coefficient) that is the original data that is the target of coding by the coding unit 106.

The addition unit 202 is a processing unit that generates an addition frame (quantization coefficient) by adding the decoding target frame decoded by the decoding unit 201 and the reference frame held by the reference holding unit 204. .. The addition unit 202 reads the reference frame from the reference holding unit 204 before performing the above-mentioned addition processing. The addition frame generated by the addition unit 202 is a frequency coefficient generated by the conversion unit 103 based on the encoding target frame.

The inverse quantization unit 203 is a processing unit that calculates a decoding target frame (frequency coefficient) by inversely quantizing the addition frame generated by the addition unit 202 by the addition process. The process of the inverse quantization unit 203 is the same as the inverse quantization unit 107 of the encoding device 50.

The reference holding unit 204 is a storage unit that holds the key frame of the decoding target frame generated by the dequantization unit 203 as a reference frame. The key frame held by the reference holding unit 204 is referred to by the adding unit 202 and is used in the adding process. The reference holding unit 204 is realized by, for example, the main memory 12 or the like.

The inverse transform unit 205 is a processing unit that performs inverse transform processing on the decoding target frame (frequency coefficient) generated by the inverse quantization unit 203 by inverse quantization. The inverse conversion process is the reverse of the conversion performed by the conversion unit 103 of the encoding device 50. Specifically, the inverse transform process is an inverse frequency transform process when the transform unit 103 performs frequency transform, and is an inverse order transform process when the transform unit 103 performs order transform.

The deblocking filter 206 is a processing unit that performs a deblocking filter process on a block boundary in the decoding target frame after the inverse transform unit 205 performs the inverse transform process.

The generation unit 207 is a processing unit that acquires an addition frame that has been subjected to deblocking filter processing by the deblocking filter 206, generates a frame by adding attribute information and the like to the acquired addition frame, and passes the frame to subsequent processing. is there.

The processing of the decoding device 20 configured as above will be described.

FIG. 6 is a flowchart showing the processing of the decoding device 20 according to the related technique.

In step S201, the decoding unit 201 acquires a decoding target frame by receiving and decoding the encoded data.

In step S202, the decryption device 20 branches the process depending on whether or not the decryption target frame is a key frame. If the decoding target frame is a key frame (Yes in step S202), the process proceeds to step S203, and if not (No in step S202), the process proceeds to step S211.

In step S203, the dequantization unit 203 dequantizes the key frame that is the decoding target frame.

In step S204, the dequantization unit 203 stores the key frame dequantized in step S203 in the reference holding unit 204.

In step S205, the inverse transformation unit 205 performs an inverse transformation process on the addition frame dequantized by the dequantization unit 203.

In step S206, the deblocking filter 206 performs the deblocking filter process on the block boundary in the addition frame after the inverse transform unit 205 performs the inverse transform process.

In step S207, the generation unit 207 generates a frame and passes it to the subsequent processing.

In step S211, the addition unit 202 adds the frame generated by the decoding of the decoding unit 201 and the reference frame read from the reference holding unit 204 to generate an addition frame.

In step S212, the inverse quantization unit 203 inversely quantizes the addition frame generated by the addition unit 202 by the addition process.

In the encoding device according to the related art described above, even when there is no motion in the video or when the video is moving violently, an interframe having a data amount corresponding to the difference from the key frame immediately before the encoding target frame Encoded frames are generated and transmitted to the network. Therefore, the communication amount of other communication performed in the network may be squeezed, or the communication loss that may occur in the network may adversely affect the other communication.

Hereinafter, a technique for reducing the size of encoded data in the encoding device according to the present embodiment will be described in more detail. More specifically, the coding apparatus according to the present embodiment controls whether or not to output a coded frame in accordance with a coding processing result, thereby coding data to be transmitted to a network. Reduce the amount of data in.

FIG. 7 is a block diagram showing the configuration of the coding apparatus according to this embodiment. FIG. 8 is an explanatory diagram of an example of slide show contents including still images.

As shown in FIG. 7, the encoding device 10 includes a frame holding unit 100, an acquisition unit 101, a blocking filter 102, a conversion unit 103, a difference unit 104A, a quantization unit 105, and an encoding unit 106. The inverse quantization unit 107, the reference holding unit 108, the control unit 111, and the switching unit 112.

Here, the frame holding unit 100, the acquisition unit 101, the blocking filter 102, the conversion unit 103, the quantization unit 105, the encoding unit 106, the dequantization unit 107, and the reference holding unit 108. Has the same function as that of the component having the same name of the encoding device 50, and detailed description thereof will be omitted.

Similar to the difference unit 104 in the encoding device 50, the difference unit 104A is the difference between the encoding target frame after the conversion unit 103 has performed the conversion process and the reference frame held by the reference holding unit 108. Is a processing unit that generates as a difference frame. At the time when the difference unit 104A acquires the frame from the conversion unit 103, it is not determined whether this frame is a key frame or a difference frame, and the control unit 111 in the subsequent stage controls the state of the switching unit 112. By doing so, it is determined whether this frame becomes a key frame or a difference frame. When the frame input to the difference unit 104A is inter-frame coded (becomes a difference frame), the frame is provided to the quantization unit 105 through the switching unit 112 in the subsequent stage. On the other hand, when the frame input to the difference unit 104A is intraframe-encoded (becomes a key frame), the frame input to the difference unit 104A is provided to the quantization unit 105 and generated by the difference unit 104A. The difference frame is not used in the subsequent processing.

The control unit 111 makes a determination based on the encoded data generated by the encoding unit 106 by encoding, controls whether or not to output the encoded data according to the determination result, and sets a new encoding target. It is a processing unit that controls the encoding of the frame.

Specifically, when the encoding unit 106 performs inter-frame encoding on both the first frame of the series of frames included in the content and the second frame subsequent to the first frame, the control unit 111 determines that , (A) when the frame size of the encoded second frame does not belong to a predetermined range including the frame size of the encoded first frame, control is performed to output the encoded second frame, (B) When the frame size of the encoded second frame belongs to a predetermined range including the frame size of the encoded first frame, control is performed not to output the encoded second frame. Here, the predetermined range is a range defined as a range in which the first frame and the second frame are considered to be substantially the same, and is, for example, 1/100 or 1/1000 of the data size of the immediately preceding encoded key frame. It can be a degree. The term "frame size of frame" means "data size of frame". The same applies hereafter.

The control unit 111 acquires the encoded data generated by the encoding unit 106 by encoding, information indicating whether the frame included in the encoded data is a key frame, and the data size of the encoded data. To get Then, the control unit 111 makes a determination based on the information and the data size, and intraframe-encodes a new encoding target frame to be encoded next by the encoding device 10 according to the determination result. Alternatively, it is determined whether to perform interframe coding (in other words, which frame is the key frame and which frame is the difference frame). Further, the control unit 111 determines whether to output the encoded data generated by the encoding unit 106 according to the result of the determination, that is, whether to transmit the encoded data from the transmission device 1 to the reproduction device 2. Decide whether or not. Then, the control unit 111 controls the state of the switching unit 112 according to the determination result, and controls whether to output the encoded data generated by the encoding unit 106.

The determination by the control unit 111 is mainly based on the still image detection for the frame. Here, the still image detection is a process of detecting that a plurality of consecutive frames included in the content are the same or substantially the same still image. A plurality of consecutive frames become the same still image when, for example, each of a plurality of consecutive frames is an image in which a photograph, characters, or figures are arranged by computer graphics. This is a slide show content in which a still image is generated every few seconds to several minutes and generated by a presentation application. An example of slide show content is shown in FIG. The content shown in FIG. 8 includes three still images that are sequentially switched. Of the three still images, the first still image is included in frames 1 to 6, the second still image is included in frames 7 to 10, and the third still image is included in frames 11 to 16. ing.

Further, a plurality of consecutive frames are not the same but substantially the same still image, for example, each of the plurality of consecutive frames is a frame that constitutes a moving image captured by a moving image camera of a landscape with very little motion. In some cases. In this case, a plurality of consecutive frames may not be exactly the same due to the influence of an extremely small amount of motion included in the landscape or noise included in the sensor value acquired by the image sensor during shooting. It should be noted that it is possible to adjust how much the motion of the still image is considered to be substantially the same by increasing or decreasing the predetermined range.

The control unit 111 outputs the coded second frame when the frame size of the coded second frame is different from the frame size of the coded first frame in (a) above. In (b) above, when the frame size of the encoded second frame matches the frame size of the encoded first frame, control is performed so as not to output the encoded second frame. You may. This corresponds to the case where the size of the predetermined range is zero. That is, this corresponds to the case where the data sizes of the two encoded frames are exactly the same. In the following description, this case will be described as an example.

When the encoding unit 106 intra-frame encodes the third frame of the series of frames, the control unit 111 inter-frame encodes the fourth frame next to the third frame of the series of frames. You may control to do so. As a result, the encoding device 10 can reduce the size of encoded data by performing inter-frame encoding on the frame after intra-frame encoding.

In addition, when the encoding unit 106 intra-codes the third frame, the control unit 111 controls the inter-frame coding of the fourth and subsequent frames of the series of frames in order of reproduction. However, when the data size of the inter-coded fifth frame is equal to or larger than the threshold value at the time of inter-coded the fifth frame of the fourth and subsequent frames, the fifth frame of the series of frames The sixth frame after the above may be controlled to be intra-frame encoded. As a result, the encoding device 10 intra-frame encodes a new frame when the data size of the inter-frame encoded frame is relatively large. When the difference between the inter-frame coded frame and the key frame is relatively large, by intra-coding the new frame as a new key frame, the difference between the subsequent frames and the new key frame can be calculated. It has the effect of making it relatively small.

The switching unit 112 determines whether the frame provided to the subsequent quantization unit 105 is the frame after the conversion process by the conversion unit 103 or the difference frame generated by the difference process by the difference unit 104A. Is a switching unit for switching. A state in which the switching unit 112 provides the quantization unit 105 with the frame after the conversion processing by the conversion unit 103, that is, a state in which the terminals a and c are connected in FIG. 7 is referred to as a state A. Further, the state in which the switching unit 112 provides the difference frame generated by the difference unit 104A by the difference process to the quantization unit 105, that is, the state in which the terminal b and the terminal c are connected in FIG. 7 is referred to as a state B. The control unit 111 controls whether the switching unit 112 takes the state A or the state B.

The operations of the difference unit 104A and the switching unit 112 described above are calculated by the difference unit 104 of the encoding device 50 when the frame to be encoded is not a key frame, and when the frame is a key frame, the difference is calculated. This is similar to the operation of not calculating the difference. That is, the difference unit 104A and the switching unit 112 are more specifically described by dividing the difference unit 104 of the encoding device 50 into two constituent elements.

The processing of the encoding device 10 configured as above will be described using a flow chart.

FIG. 9 is a flowchart showing the processing of the coding device 10 according to the present embodiment.

As illustrated in FIG. 9, the encoding device 10 executes the encoding process S301, and executes the encoding control process S302 according to the result of the executed encoding process S301. Then, the output control process S303 is performed on the encoded data generated by the encoding process S301 and generated by the encoding control process S302. In the output control processing S303, the encoded data generated by the encoding in the encoding processing S301 based on the control in the encoding control processing S302, more specifically, the information (described later) stored in the encoding control processing S302. Is output or is not output (that is, is discarded). Note that when the encoded data is not output, the information indicating that the encoded data is not output or the information indicating that the immediately preceding frame is continuous may be output. This is to avoid stopping the reproduction by determining that the reproduction apparatus 2 (decoding apparatus 20) does not receive the encoded data for a certain period of time and that the reproduction of the content cannot be continued due to a problem in the transmission apparatus 1 or the network 3. is there.

The encoding process S301 and the encoding control process S302 will be described in detail below.

FIG. 10 is a flowchart showing the encoding process S301 of the encoding device 10 according to the present embodiment.

The processing of steps S101 to S108 in FIG. 10 is substantially the same as the processing of steps S101 to S108 shown in FIG. Here, step S104A is a more specific version of the process of step S104.

In step S104A, the control unit 111 switches between intraframe coding and interframe coding of the coding target frame according to the state of the switching unit 112. That is, when the switching unit 112 is in the state A (“intra-frame coding” in step S104A), a series of processes (steps S105 to S107) for intra-coding the frame to be coded is performed. On the other hand, when the switching unit 112 is in the state B (“interframe coding” in step S104A), a series of processes (steps S111 to S112) for interframe coding the coding target frame is performed.

FIG. 11 is a flowchart showing the coding control process S302 of the coding device 10 according to the present embodiment.

As shown in FIG. 11, in step S401, the control unit 111 determines whether or not the frame encoded by the encoding unit 106 is a key frame, in other words, the encoding unit 106 intra-frame encodes the frame. It is determined whether or not interframe coding is performed. If it is determined that the frame encoded by the encoding unit 106 is a key frame (Yes in step S401), the process proceeds to step S402, and if not (No in step S401), the process proceeds to step S411.

In step S402, the control unit 111 resets the counter. This counter is a counter for counting how many consecutive times the data size of the inter-coded frame is the same as the data size of the inter-frame coded frame immediately before. And is used in step S423 and the like described later. If the encoded frame is a key frame, the above continuity is interrupted and the counter is reset.

In step S403, the control unit 111 controls to output the encoded frame that is the target of the determination in step S401 in the output control processing S303 performed after the main encoding control processing S302. Specifically, the control unit 111 stores information (for example, flag information) indicating that the frame is output.

In step S404, the control unit 111 controls to interframe-encode the next frame in the series of frames. Specifically, the control unit 111 changes the switching unit 112 to the state B. When the process of step S404 is finished, the series of processes shown in this flowchart is finished.

In step S411, the control unit 111 determines whether the data size of the encoded data generated by the encoding unit 106 by encoding is larger than the threshold value. If it is determined that the data size of the encoded data is larger than the threshold value (Yes in step S411), the process proceeds to step S412, and if not (No in step S411), the process proceeds to step S421. Note that the threshold value is, for example, 4/5 of the data size of the key frame that is intra-coded immediately before the frame. This is because when the data size of the inter-frame coded frame is relatively large, the data size of the subsequent inter-frame coded frame is reduced by intra-coding a new frame.

In step S412, the control unit 111 controls to output the encoded frame that is the target of the determination in step S401 in the output control process S303 performed after the main encoding control process S302.

In step S413, the control unit 111 controls so that the next frame in the series of frames is intra-frame encoded. Specifically, the control unit 111 changes the switching unit 112 to the state A. When the process of step S413 is finished, the series of processes shown in this flowchart is finished.

In step S421, the control unit 111 determines whether the data size of the encoded data generated by the inter-frame encoding by the encoding unit 106 matches the data size of the encoded data generated by the previous inter-frame encoding. judge. If it is determined that they match (Yes in step S421), the process proceeds to step S422, and if it is determined that they do not match (No in step S421), the process proceeds to step S431. In addition, when the previous encoding is intra-frame encoding, it is included in the above “when it is determined that they do not match”.

In step S422, the control unit 111 controls so as to prohibit the output of the encoded frame that is the target of the determination in step S401 in the output control processing S303 performed after the main encoding control processing S302.

In step S431, the control unit 111 controls to output the encoded frame that is the target of the determination in step S401 in the output control process S303 performed after the main encoding control process S302.

In step S423, the control unit 111 increments the counter. This counter is a counter for counting the number of consecutive frames in which the data size of the inter-frame coded frame is the same as the data size of the inter-frame coded frame immediately before it.

In step S424, the control unit 111 determines whether the counter is larger than the predetermined value, that is, whether the counter exceeds the predetermined value. If it is determined that the counter is larger than the predetermined value (Yes in step S424), the process proceeds to step S425, and if not (No in step S424), the process proceeds to step S436. The predetermined value is the number of consecutive frames in which the data size of the inter-frame coded frame is the same as the data size of the immediately preceding inter-frame coded frame, and the next frame is set It is a value indicating whether or not to be encoded, and can be set to, for example, about 30 or 150, and may be determined by the reproduction time such as the number of frames included in about 1 second or about 5 seconds. When determined by the reproduction time, it is converted into the number of frames using the frame rate (fps).

In step S425, the control unit 111 controls so that the next frame in the series of frames is intra-frame encoded. Specifically, the control unit 111 changes the switching unit 112 to the state A. When the process of step S425 is finished, the series of processes shown in this flowchart is finished. By the processing of this step, the control unit 111 determines that if the data size of the interframe-coded frames of the predetermined number of consecutive frames in the series of frames is less than or equal to the threshold value, The frame is controlled to be intra-frame encoded. As a result, even when the control for prohibiting the transmission of the inter-frame coded frames (step S422) is continuously performed, the coding including the intra-frame coded frames is performed for each frame of a predetermined value. The data can be received by the playback device 2. As a result, even if the encoded data is lost in the network 3, there is an advantage that the reproduction apparatus 2 can continue to reproduce the content by the intra-frame encoded frame that is transmitted next. is there.

In step S436, the control unit 111 controls to interframe-encode the next frame in the series of frames. Specifically, the control unit 111 changes the switching unit 112 to the state B. When the process of step S436 is finished, the series of processes shown in this flowchart is finished.

Through the series of processes described above, the encoding device 10 can reduce the size of encoded data based on the still image detection.

Hereinafter, the simulation evaluation of the data size reduction effect of the encoded data based on the still image detection performed by the encoding device 10 will be described together with the processing related to the still image detection. In the graphs in the following description, the horizontal axis indicates a frame (frame number) when a series of frames are arranged in the order of reproduction, and the vertical axis indicates the frame size generated by encoding each frame. The straight line connecting the symbols corresponding to the data size and the like on the graph is a straight line for explaining the transition of symbols of the same kind in an easy-to-understand manner. Regarding the data size and the like, only each symbol corresponding to the integer value on the horizontal axis has a meaning, and the portion between the adjacent integer values on the straight line has no meaning.

12 and 13 are explanatory diagrams for explaining the effect of still image detection by the encoding device 10 according to the present embodiment.

First, the intra mode and the inter mode that are compared with the operation of the encoding device 10 will be described with reference to FIG. The intra mode is a mode in which each frame included in the content is intra-frame encoded. The inter mode is a mode in which the frames included in the content are intra-coded for each frame at a constant interval, and the other frames are inter-frame coded.

FIG. 12 shows the data size of the encoded data in each of the intra mode and the inter mode when the content of the slide show (see FIG. 8) in which three still images are sequentially switched is encoded.

In the intra mode, the data size of each of the encoded frames 1 to 6 is about 300 bytes. Since each of the frames 1 to 6 contains the same still image, the data sizes of the encoded frames 1 to 6 are the same. Similarly, the data size of each of the encoded frames 7 to 10 is about 500 bytes, and the data size of each of the encoded frames 11 to 16 is about 200 bytes.

In the inter mode, the first frame, frame 1, is intra-frame coded, and the coded frame 1 has a data size of about 300 bytes. Each of the frames after the frame 2 is inter-frame coded by using the difference from the frame 1 which is the intra-coded frame immediately before the frame. As a result, the data size of each of the encoded frames 2 to 6 becomes about 50 bytes. Since the frames 2 to 6 each include the same still image, the data sizes of the encoded frames 2 to 6 are the same. Here, in each of the frames 2 to 6, although the frame includes the same still image, the data size of the frame is not zero and 50 bytes are generated. This is because, in the inter mode, the frame 1 is first intraframe-encoded, and then the frames after the frame 2 are sequentially arranged in the order of reproduction using the difference from the frame 1 which is the immediately preceding intraframe-encoded frame. This is because each of the frames 2 to 6 has the same amount of difference data because they are inter-coded.

Further, similarly to the above, the data size of each of the encoded frames 7 to 10 is approximately 200 bytes, and the data size of each of the encoded frames 11 to 16 is approximately 100 bytes. ing.

It can be said that the data size of the encoded frame can be suppressed more in the case of the inter mode than in the case of the intra mode. However, in the intra mode, it is necessary to transmit data of the same data size in a portion where the same still image is continuously reproduced. This data communication has a problem that the amount of communication of other communication performed in the network 3 is squeezed and a communication loss that may occur in the network 3 adversely affects other communication.

In addition to the intra mode and the inter mode shown in FIG. 12, FIG. 13 shows the data size of the frame output in the still image determination mode and the threshold value for each frame. The threshold value is set to 4/5 of the data size of the key frame that is intra-coded immediately before the frame.

In the still image determination mode, the encoding device 10 intraframe-encodes the frame 1. The intra-frame encoded frame 1 is controlled to be output in the output control processing S303 (step S403). Further, the encoding device 10 controls so that the next frame 2 is interframe-encoded (step S404).

Next, the encoding device 10 performs interframe encoding of the frame 2. The data size of the inter-frame coded frame 2 is less than or equal to the threshold value (No in step S411), and there is no frame previously inter-frame coded (No in step S421). Therefore, the output control process S303 outputs the data. Control is performed (step S431). In addition, the encoding device 10 controls so that the next frame 3 is inter-frame encoded (step S436). The frame 1 corresponds to the third frame, and the frame 2 corresponds to the fourth frame.

Next, the encoding device 10 encodes the frame 3 between frames. The data size of the inter-frame coded frame 3 is less than or equal to the threshold value (No in step S411), and the data size of the coded frame 3 matches the data size of the frame 2 previously inter-frame coded. Since (Yes in step S421), the encoding apparatus 10 controls so as to prohibit output in the output control process S303 (step S422). After that, the encoding device 10 performs the same processing as that of the frame 3 on each of the frames 4 to 6 and controls so as to prohibit the output of each of the encoded frames 4 to 6. Here, the frame 2 corresponds to the first frame, and the frame 3 corresponds to the second frame.

Next, the encoding device 10 interframe-encodes the frame 7. The data size of the inter-frame coded frame 7 is less than or equal to the threshold value (No in step S411), and the data size of the coded frame 7 is different from the data size of the frame 6 previously inter-frame coded ( Since No in step S421), the output control processing S303 is controlled to output (step S431). After that, the encoding device 10 performs the same processing as that of the frame 3 on each of the frames 8 to 10 and controls so as to prohibit the output of each of the encoded frames 8 to 10. The frame 11 and subsequent frames are the same as the frames 7 to 10.

As described above, in the still image determination mode, for each inter-frame coded frame, output is prohibited when the data size is the same as the previous inter-frame coded frame. In this way, it is possible to prevent the data including the same difference data for the same still image, more specifically, the frames 3 to 6, 8 to 10 and 12 to 16 in FIG. 13 from being transmitted to the network 3. However, the amount of data transmitted to the network 3 can be reduced.

As described above, the encoding device according to the present embodiment is configured so that when two consecutive frames that are the same or substantially the same image in a series of frames are inter-frame encoded, the above two The encoded data in which the second frame of the frames is encoded is not transmitted to the network. Since these two frames have the same or substantially the same image, even if the encoded data of the second frame is not transmitted, the influence on the reproduced content is relatively small. Thereby, the encoding device can reduce the data amount of the encoded data transmitted to the network.

In addition, the encoding device encodes the second frame of the two frames when two consecutive frames, which are the same image, are inter-frame encoded. Stop sending encoded data to the network. Thereby, the encoding device can reduce the data amount of the encoded data transmitted to the network.

In addition, the encoding device can reduce the size of encoded data by performing inter-frame encoding on the frame after the intra-frame encoded frame.

In addition, the coding apparatus, even when continuously performing control to prohibit transmission of inter-frame coded frames, performs coding including intra-frame coded frames for each frame of a predetermined value. Output the data. As a result, even if the encoded data is lost during communication on the network, the reproduction device can continue to reproduce the content by the intra-frame encoded frame to be transmitted next. ..

Further, when the decoding device that receives the encoded data transmitted by the encoding device does not receive the encoded data for a certain period of time, it is possible to avoid the reproduction device from stopping the reproduction of the content. This is because, if the reproduction device does not receive the encoded data for a certain period of time, it may determine that the reproduction of the content cannot be continued due to a problem in the transmission device or the network and may stop the reproduction.

Although the encoding apparatus, the content reproduction system, and the like of the present invention have been described above based on the embodiment, the present invention is not limited to this embodiment. Unless departing from the spirit of the present invention, various modifications that can be thought of by those skilled in the art are made to the present embodiment, and forms constructed by combining components in different embodiments are also included in the scope of the present invention. ..

INDUSTRIAL APPLICABILITY The present invention can be used for an encoding device, a content reproduction system, and the like that reduce the data amount of encoded data transmitted to a network. Specifically, a plurality of the devices can be installed in a public space or a commercial facility and can be used as a reproducing device for reproducing images and the like, a transmitting device for providing contents to the reproducing device, and the like.

1 Transmitter 2 Playback Device 3 Network 10, 50 Encoding Device 11 CPU
12 main memory 13 storage 20 decoding device 100 frame holding unit 101 acquisition unit 102 blocking filter 103 conversion unit 104, 104A difference unit 105 quantization unit 106 encoding unit 107, 203 dequantization unit 108, 204 reference holding unit 111 control Unit 112 switching unit 201 decoding unit 202 addition unit 205 inverse conversion unit 206 deblocking filter 207 generation unit S content reproduction system

Claims (6)

A frame included in the series of frames, an encoding unit for intra-frame encoding or inter-frame encoding,
And a control unit for controlling whether or not the encoding unit outputs an encoded frame,
The control unit is
A first frame of said sequence of frames, both the second frame is the next frame of the first frame, the encoding unit, the difference from the same intraframe coded key frames When inter-frame coding is performed using
(A) Control to output the encoded second frame when the encoded data size of the second frame does not belong to a predetermined range including the encoded data size of the first frame Then
(B) An encoding device that controls so as not to output the encoded second frame when the data size of the encoded second frame belongs to the predetermined range. The control unit is
In (a), when the data size of the encoded second frame is different from the data size of the encoded first frame, control is performed to output the encoded second frame,
In (b), when the data size of the encoded second frame matches the data size of the encoded first frame, control is performed so as not to output the encoded second frame. The encoding device according to claim 1. The control unit is
When the encoding unit intra-codes the third frame of the series of frames, the fourth frame, which is the next frame of the third frame of the series of frames, is inter-frame encoded. The encoding apparatus according to claim 1 or 2, which is controlled as follows. The control unit is
When the data size of a predetermined number of consecutively interframe-encoded frames of the series of frames is equal to or less than a threshold value, control is performed so that the frame next to the predetermined number of frames is intraframe-encoded. The encoding device according to any one of Items 1 to 3. The control unit is
The encoding device according to claim 1, wherein when controlling not to output the second frame, control is performed to output control information indicating that there is no change in the frame. An encoding step of performing intra-frame encoding or inter-frame encoding of the frames included in the series of frames;
And a control step of controlling whether to output the frame encoded in the encoding step,
In the control step,
Both the first frame of the series of frames and the second frame, which is the frame next to the first frame, are subjected to the encoding step to obtain a difference from the same intra-frame encoded key frame. When inter-frame coding is performed using
(A) Control to output the encoded second frame when the encoded data size of the second frame does not belong to a predetermined range including the encoded data size of the first frame Then
(B) A coding method for controlling so as not to output the coded second frame when the data size of the coded second frame belongs to the predetermined range.

Images