JP2011061287A - Communication control device, communication control method, and communication control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication control device, a communication control method, and a communication control program for decreasing a load applied to a band without deteriorating the quality of a moving picture when the band of the network is decreased. <P>SOLUTION: When receiving an I-picture from a transmission source device, an MCU determines whether or not a band of a network with the transmission destination device is decreased (S4). When the band is decreased (S4: YES), the MCU decodes the received I picture once and generates decoded reception data (S8). With data of an image transmitted to the transmission destination device prior to image data of the received I picture as reference data, the MCU newly generates a P-picture from the decoded reception data (S13). The MCU transmits the generated P-picture to the transmission destination device (S14). <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a communication control apparatus, a communication control method, and a communication control program for controlling communication of encoded moving image data.

  Conventionally, a technique for transmitting and receiving encoded moving image data between a plurality of communication devices via a network is known. If the bit rate of the output to the network is larger than the bandwidth of the network between the communication device that transmits data and the communication device that receives the data, the video quality is degraded. For example, in the multipoint video conference system described in Patent Document 1, a conference terminal arranged at each site transmits information on an image encoding rate to an MCU (Multi Point Control Unit) that is a control device. The MCU changes the frame rate of the encoded data to be transmitted to the conference terminal based on the encoding rate information received from the conference terminal. The MCU adjusts the load applied to the network bandwidth by changing the frame rate.

JP-A-8-317366

  The MCU described in Patent Document 1 reduces the frame rate of encoded data to be transmitted when the bandwidth of the network decreases. When the frame rate is reduced, the load applied to the band is reduced, but the quality of the moving image to be reproduced is lowered.

  The present invention provides a communication control device, a communication control method, and a communication control program that can reduce the load applied to the bandwidth without reducing the quality of the moving image when the bandwidth of the network is reduced. Objective.

  A communication control device according to a first aspect of the present invention is connected to a plurality of communication devices via a network, and transmits and receives encoded data, which is encoded video data, between the plurality of communication devices. A communication control device for controlling, wherein the receiving unit receives the encoded data transmitted by a transmission source device that is a communication device that transmits the encoded data to the network among the plurality of communication devices; and the receiving unit A first decoding means for generating decoded reception data, which is image data generated as a result of decoding, by decoding the first encoded data that is the encoded data received by the plurality of communication devices; Among them, a transmission unit that transmits encoded data to a transmission destination device that is a communication device that is a transmission destination of encoded data, and a second code that is the encoded data transmitted by the transmission unit. By decoding the encrypted data, second decoding means for generating decoded transmission data which is image data generated as a result of decoding, and the decoded transmission data generated by the second decoding means are stored in the storage means A code encoded using only intra-frame predictive coding according to a band measured by the measuring unit, a measuring unit for measuring the band of the network between the storage control unit and the transmission destination device The decoded reception data generated by the first decoding means from the I picture (Intra-coded Picture) that is the encoded data is inter-frame encoded with reference to the decoded transmission data stored in the storage means, and the encoded data Generating means for generating a P picture (Predictive-coded Picture), wherein the transmitting means comprises: When the P picture is generated by the generating means, the generated P picture is transmitted as the second encoded data to the transmission destination device, and when the P picture is not generated by the generating means, it is received by the receiving means. The first encoded data is transmitted as the second encoded data to the transmission destination device.

  The communication control apparatus according to the first aspect can newly generate a P picture having a data size smaller than that of the I picture from the received I picture according to the network bandwidth. The generated P picture is transmitted to the transmission destination apparatus. Therefore, the communication control apparatus can reduce the output bit rate without reducing the frame rate. Therefore, it is possible to reduce the load applied to the network bandwidth without reducing the quality of the video.

  The communication control apparatus may further include a bandwidth determination unit that determines whether the bandwidth measured by the measurement unit is lower than a threshold value. The generating unit may generate a P picture when the band determining unit determines that the band is lower than a threshold value. In this case, when the bandwidth is lower than the threshold, the communication control apparatus can reduce the proportion of I pictures included in the entire data to be transmitted. Therefore, the load applied to the band can be easily and appropriately controlled as the band increases or decreases.

  The first decoding unit may generate the decoded reception data from the first encoded data when the band determination unit determines that the band is lower than a threshold value. The generating means may generate a P picture by inter-coding the decoded received data generated by the first decoding means from the first encoded data other than the I picture. In this case, when the bandwidth is lower than the threshold value, all the P pictures transmitted to the transmission destination apparatus are inter-frame encoded by referring to the decoded transmission data of the second encoded data transmitted to the transmission destination apparatus. The Therefore, the referenced data that is referred to when the communication control apparatus generates the P picture and the referenced data that is referred to when the apparatus that has received the P picture decodes the P picture are the same data. Therefore, the communication control apparatus can generate a P picture and transmit it to the transmission destination apparatus without degrading the quality of the decoded image.

  The communication control apparatus determines whether or not the first encoded data received by the receiving unit is the I picture when the band determining unit determines that the band is lower than a threshold value. You may further provide a judgment means. The generating means may generate a P picture from decoded received data obtained by decoding the I picture only when the picture determining means determines that the first encoded data is the I picture. In this case, when the received first encoded data is a P picture, the communication control apparatus transmits the received P picture as it is without performing a process of generating a new P picture. Therefore, the communication control apparatus can efficiently reduce the load applied to the network bandwidth without abruptly increasing the processing load.

  It is desirable that the storage control means always stores at least the latest decoded transmission data in the storage means among the decoded transmission data generated by the second decoding means. As a result, when newly generating a P picture, the communication control apparatus can immediately generate a P picture by referring to the decoded transmission data stored in the storage means. Therefore, the communication control apparatus can reduce the load of the band quickly after the band is lowered, unlike the case where the decoded transmission data is generated after the band is lowered.

  It is desirable that the generation unit generates a P picture with reference to decoded transmission data of the second encoded data transmitted by the transmission unit one frame before the P picture to be generated. As a result, the communication control apparatus can reduce the data size of the P picture as compared to the case of referring to image data of two frames or more before. Therefore, the communication control apparatus can efficiently reduce the load applied to the band.

  The communication control apparatus may further include an input unit that inputs unreceived information that is information indicating that reception of the second encoded data has failed from the transmission destination apparatus. The generation means, when the unreceived information is input by the input means, is transmitted to the transmission destination apparatus before the second encoded data in which the reception failure is indicated by the unreceived information. A P picture may be generated with reference to the decoded transmission data of the encoded data. In this case, in the transmission destination apparatus, it is possible to reduce the possibility that a large number of pictures cannot be decoded in conjunction with the loss of one picture. Therefore, the communication control apparatus can minimize a decrease in image quality.

  The communication control method according to the second aspect of the present invention is performed by a communication control device connected to a plurality of communication devices via a network, and the plurality of communications of encoded data which is encoded moving image data. A communication control method for controlling transmission / reception between devices, wherein reception of the encoded data transmitted by a transmission source device that is a communication device that transmits encoded data to the network among the plurality of communication devices is received. A first decoding step of generating decoded received data that is image data generated as a result of decoding by decoding the first encoded data that is the encoded data received by the receiving step; Of the plurality of communication devices, a transmission step of transmitting the encoded data to a transmission destination device that is a communication device of a transmission destination of the encoded data, and the transmission step. A second decoding step for generating decoded transmission data that is image data generated as a result of decoding by decoding the second encoded data that is the encoded data transmitted by the second decoding step; A storage control step of storing the generated decoded transmission data in a storage means, a measurement step of measuring the bandwidth of the network between the transmission destination devices, and a frame according to the bandwidth measured by the measurement step The decoded received data generated by the first decoding step from the I picture (Intra-coded Picture) that is the encoded data encoded using only the intra prediction encoding is the decoded transmission data stored in the storage means. And P-picture (Predictive-) which is encoded data. a generation step of generating a coded picture), and when the P picture is generated by the generation step, the transmission step transmits the generated P picture as the second encoded data to the transmission destination device. When the P picture is not generated by the generating step, the first encoded data received by the receiving step is transmitted as the second encoded data to the transmission destination device.

  According to the communication control method according to the second aspect, it is possible to newly generate a P picture having a data size smaller than that of an I picture and transmit the generated P picture to the transmission destination apparatus according to the network bandwidth. As a result, the output bit rate decreases without decreasing the frame rate. Therefore, it is possible to reduce the load applied to the network bandwidth without reducing the quality of the video.

  A communication control program according to a third aspect of the present invention is a communication control program for performing control of transmitting and receiving encoded data, which is encoded moving image data, between a plurality of communication devices via a network. A reception step of receiving the encoded data transmitted by a transmission source device, which is a communication device that transmits the encoded data to the network, among the plurality of communication devices; A first decoding step of generating decoded reception data, which is image data generated as a result of decoding, by decoding the first encoded data, which is the encoded data, and encoding among the plurality of communication devices A transmission step of transmitting encoded data to a transmission destination device which is a communication device of a data transmission destination, and the code transmitted by the transmission step A second decoding step for generating decoded transmission data which is image data generated as a result of decoding by decoding the second encoded data which is data; and the decoded transmission data generated by the second decoding step Is stored in the storage means, the measurement step is to measure the bandwidth of the network between the transmission destination devices, and only intra-frame predictive coding is used according to the bandwidth measured by the measurement step. The decoded reception data generated by the first decoding step from the I-picture (Intra-coded Picture) that is the encoded data encoded by the above-described processing is referred to the decoded transmission data stored in the storage means. P-picture (Predictive-coded Picture) that is encoded data Generating a generating step, and in the transmitting step, when a P picture is generated by the generating step, the generated P picture is transmitted to the transmission destination device as the second encoded data, When the P picture is not generated by the generating step, the first encoded data received by the receiving step is transmitted to the transmission destination device as the second encoded data.

  According to the communication control program according to the third aspect, it is possible to newly generate a P picture having a data size smaller than that of an I picture and transmit the generated P picture to the transmission destination apparatus according to the network bandwidth. As a result, the output bit rate decreases without decreasing the frame rate. Therefore, it is possible to reduce the load applied to the network bandwidth without reducing the quality of the video.

1 is a diagram illustrating a system configuration of a video conference system 2. FIG. It is a block diagram which shows the electric constitution of MCU1. It is a flowchart of the main process which MCU1 which concerns on 1st embodiment performs. It is a figure which shows an example (when not received information is not input) of the result of the process by MCU1 of 1st embodiment. It is a figure which shows an example (when unreceived information is input) of the result of the process by MCU1 of 1st embodiment. It is a flowchart of the main process which MCU101 which concerns on 2nd embodiment performs. It is a figure which shows an example of the result of the process by MCU101 of 2nd embodiment.

  Hereinafter, an MCU (Multipoint Control Unit) 1 that embodies a communication control device of the present invention will be described with reference to the drawings. The drawings to be referred to are used for explaining technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like described in the drawings are not intended to be limited to these, but are merely illustrative examples.

  With reference to FIG. 1, the system configuration of the video conference system 2 including the MCU 1 will be described. The video conference system 2 is composed of a plurality of devices connected via a network 5. In FIG. 1, as devices constituting the video conference system 2, an MCU 1, a communication device 3 disposed at the base A, and a personal computer (hereinafter referred to as “PC”) 4 disposed at the base B are illustrated. ing. In the video conference system 2, devices arranged at each site transmit and receive data to each other, thereby allowing users at a plurality of sites to share video and audio. As a result, even when all the users are not at the same base, the users can smoothly execute the conference. Note that the number of devices connected to the MCU 1 via the network 5 is not limited to two, and may be three or more. In the following, in order to simplify the description, as shown in FIG. 1, a case where two devices (communication device 3 and PC 4) are connected to the MCU 1 is taken as an example.

  The MCU 1 receives image data and audio data from each of the devices connected via the network 5. The received data is transmitted (transferred) to another device (transmission destination device) other than the device (transmission source device) that transmitted the data. That is, the MCU 1 controls transmission / reception of image data and audio data between connected devices. As a result, the PC 4 can output the video and audio of the site A at the site B. The communication device 3 can output the video and audio of the site B at the site A. Furthermore, the MCU 1 can reduce the load applied to the network 5 by appropriately controlling transmission / reception of encoded image data without deteriorating the quality of video. In the following, processing of encoded image data performed in the video conference system 2 will be mainly described.

  The communication device 3 and the PC 4 are connected to a microphone 6, a speaker 7, a camera 8, and a display device 9 that exist in the same site. The microphone 6 converts sound into sound data. The speaker 7 outputs sound based on the sound data. The camera 8 captures a moving image. The display device 9 displays an image. The communication device 3 and the PC 4 encode the audio data input from the microphone 6 and the image data input from the camera 8, and output the encoded data to the MCU 1 via the network 5. In addition, the communication device 3 and the PC 4 receive encoded data from other devices via the MCU 1, and output audio and images using the speaker 7 and the display device 9.

  The communication device 3 is a dedicated terminal for executing a video conference. The PC 4 is a general PC that stores a program for executing a video conference. Both the communication device 3 and the PC 4 may have a general configuration for executing the above-described data encoding, transmission / reception of encoded data, and reproduction of sound and images. Specifically, it only needs to have a CPU, RAM, ROM, external communication interface, and the like and be able to execute encoding processing and the like according to a program. Since the configuration and processing of the communication device 3 and the PC 4 are not the main part of the present invention, detailed description is omitted.

  In addition, when the communication apparatus 3 and the PC 4 fail to receive the encoded data of the image transmitted from the MCU 1, the communication apparatus 3 and the PC 4 can transmit unreceived information that is information indicating that the reception has failed to the MCU 1. For example, when the encoded data cannot be received for a certain period of time, when only a part of the encoded data of one picture can be received, it is determined that the reception has failed, and the unreceived information is transmitted to the MCU 1 To do.

  Next, image data transmitted and received between devices of the video conference system 2 will be described. Each device constituting the video conference system 2 is an H.264 standard. It transmits and receives encoded data that has been subjected to image compression encoding based on the H.264 standard. Image compression coding includes intra-frame coding and inter-frame coding.

  Intraframe coding is coding performed by intra prediction within one frame of image data of a plurality of consecutive frames of image data input by a camera. An I picture (Intra-coded Picture) that is encoded data generated by intra-frame encoding can be decoded independently without referring to other pictures.

  In interframe coding, prediction data is calculated by referring to data of a frame different from data of a frame to be encoded among continuous frame data, and the calculated prediction error is encoded. Hereinafter, image data referred to in inter-frame coding is referred to as referenced data. One type of encoded data generated by inter-frame encoding is a P picture (Predictive-coded Picture). In order to decode the P picture, reference data is required. However, the data amount of P pictures is smaller than that of I pictures that can be decoded independently.

  The communication device 3 and the PC 4 perform image compression coding on the image input from the camera 8 in the same base, and transmit the image to the MCU 1 via the network 5. In image compression coding, generally, I pictures are regularly generated between a plurality of P pictures, such as I picture, P picture, P picture, I picture, P picture, and so on. In addition, when generating a P picture, it is desirable to perform encoding using data immediately before the frame to be encoded as referenced data. This is because the prediction error is reduced and the amount of data is reduced by using the immediately preceding data as a referenced picture.

  When the MCU 1 receives the encoded data from the transmission source device, the MCU 1 transmits the reception encoded data to the transmission destination device via the network 5. Here, consider a case where the bandwidth of the network 5 between the MCU 1 and the transmission destination device is reduced (when it is narrowed). If data output at a high bit rate is performed in a state where the bandwidth of the network 5 is lowered, there is a possibility that problems such as occurrence of congestion and packet loss may occur. As a result, the quality of the video displayed on the transmission destination device is degraded.

  In the conventional MCU, the frame rate of the encoded data to be transmitted to the transmission destination apparatus is changed based on the information on the encoding rate of the image received from the transmission destination apparatus. However, when the frame rate is reduced, the load applied to the band is reduced, but the quality of the video displayed on the transmission destination device is reduced. In addition, when the network bandwidth between the MCU and the transmission destination device is reduced, it may be possible to notify the transmission source device to reduce the data size of the encoded data. However, it takes time until the encoded data is transmitted from the transmission source device and received by the transmission destination device. Therefore, there is encoded data that has already been transmitted to the MCU when the transmission destination apparatus grasps the decrease in the bandwidth. Therefore, the load on the network cannot be reduced immediately. When there are a plurality of transmission destination devices, it is difficult for the transmission source device to generate different encoded data for each transmission destination device.

  The MCU 1 according to the present embodiment changes an I picture having a data size larger than the P picture to a P picture when the bandwidth of the network 5 with the transmission destination apparatus is reduced. As a result, it is possible to reduce the load applied to the network 5 without reducing the frame rate. Even when the I picture has already been transmitted from the transmission source device at the time when the bandwidth is reduced, the MCU 1 can newly reduce the data size of the encoded data by generating a new P picture. Since the transmission source device does not need to generate a plurality of encoded data for the same image, the processing load on the transmission source device does not increase. Details of the above processing will be described below.

  The electrical configuration of the MCU 1 will be described with reference to FIG. The MCU 1 includes a CPU 10 that controls the MCU 1. A ROM 11, a RAM 12, an HDD 13, and an input / output interface 16 are connected to the CPU 10 via a bus 15. The ROM 11 stores a program for operating the MCU 1 and initial values. The RAM 12 temporarily stores various information used in the control program. The HDD 13 is a non-volatile storage device that stores various types of information such as programs. Instead of the HDD 13, a storage device such as an EEPROM or a memory card may be used. The input / output interface 16 is connected to an external communication I / F 18 for connecting the MCU 1 to the network 5.

  The RAM 12 will be described in detail. The RAM 12 is provided with a work area 121, a received encoded data storage area 122, a decoded received data storage area 123, and a decoded transmission data storage area 124. The work area 121 stores various data such as flags necessary for processing. The received encoded data storage area 122 stores encoded data (hereinafter referred to as “received encoded data”) received from a transmission source apparatus that is another apparatus in the video conference system 2. The decoded received data storage area 123 stores data generated by decoding received encoded data (hereinafter referred to as “decoded received data”). In the decoded transmission data storage area 124, data (hereinafter referred to as "decoded transmission data") generated by decoding encoded data (hereinafter referred to as "transmission encoded data") transmitted to the transmission destination apparatus. Is memorized. When the MCU 1 newly generates a P picture, the MCU 1 performs inter-frame coding on the data stored in the decoded received data storage area 123 using the data stored in the decoded transmission data storage area 124 as reference data. .

  With reference to FIG. 3, the main process which MCU1 which concerns on 1st embodiment performs is demonstrated. The main process is executed by the CPU 10 in accordance with a program stored in the HDD 13. The main process is performed separately for each of the devices (communication device 3 and PC 4) connected to the MCU1. Specifically, the process for receiving the encoded data from the communication device 3 and transmitting it to the PC 4 and the process for receiving the encoded data from the PC 4 and transmitting it to the communication device 3 are performed separately. Yes. However, the flow of these processes is the same. Therefore, only the main process for receiving the encoded data from the communication device 3 and transmitting it to the PC 4 will be described below.

  When an instruction to execute transmission / reception of encoded data is input to the MCU 1, the CPU 10 starts main processing. First, various data are initialized (S1). Measurement of the communication band of the network 5 with the transmission destination device (PC4) is started (S2). The bandwidth of the network 5 may be measured by any known method.

  Encoded data for one picture is received from the transmission source device (communication device 3) (S3). The received encoded data received is stored in the received encoded data storage area 122 of the RAM 12. Next, it is determined whether or not the bandwidth of the network 5 with the transmission destination device has decreased below a threshold value (S4). If the band value is equal to or greater than the threshold value (S4: NO), the received encoded data is transmitted as it is to the transmission destination device (S5). The transmitted encoded data is decoded to generate decoded transmission data, which is stored in the decoded transmission data storage area 124 of the RAM 12 (S16). The process returns to S3.

  If the measured band value is lower than the threshold value (S4: YES), a process of newly generating a P picture is performed. First, it is determined whether or not the received encoded data is an I picture (S7). If it is an I picture (S7: YES), the received I picture is decoded, decoded received data is generated, and stored in the decoded received data storage area 123 of the RAM 12 (S8). An I picture can be decoded by itself. If the received encoded data is a P picture (S7: NO), the referenced data of the P picture is referred to, the received encoded data is decoded, and decoded received data is generated. The decoded received data is stored in the decoded received data storage area 123 (S9).

  Next, it is determined whether or not unreceived information is input from the transmission destination device (S10). As described above, the unreceived information is input from the transmission destination device when the transmission destination device fails to receive the encoded data transmitted from the MCU 1. If unreceived information has not been input (S10: NO), out of the data stored in the decoded transmission data storage area 124, the encoded data transmitted one frame before the P picture to be generated now The decoded transmission data is determined as the referenced data (S11). In other words, the decoded transmission data immediately before the P picture to be generated is determined as the referenced data that is referred to for inter-frame coding of the P picture.

  On the other hand, if unreceived information is input from the transmission destination device (S10: YES), the past encoded data that has been successfully transmitted is determined. Specifically, encoded data that has not been input with unreceived information from the transmission destination device after transmission is determined as encoded data that has been successfully transmitted in the past. Then, the decoded transmission data of the latest data (encoded data transmitted at the latest time) among the determined encoded data is determined as the referenced data (S12).

  Next, the decoded received data that has been decoded (S8 or S9) is inter-frame encoded with reference to the determined referenced data, and a new P picture is generated (S13). The generated P picture is transmitted to the transmission destination device (S14). The transmitted P picture is decoded, decoded transmission data is generated, and stored in the decoded transmission data storage area 124 (S16). The process returns to S3, and the next picture is processed.

  Next, an example of a result of processing performed by the MCU 1 of the first embodiment will be described with reference to FIGS. 4 and 5. 4 and 5, the time axis is set in the vertical direction. The flow of data transmitted from the transmission source device to the transmission destination device via the MCU 1 is shown in the left-right direction. Between time T1 and T2, the value of the band is lower than the threshold value. “I” indicates an I picture. “P” indicates a P picture. The numbers to the right of “I” and “P” indicate the order of GOP (Group of Pictures). The number on the right end of the P picture indicates the number of the P picture in the GOP. For example, P12 indicates the second P picture in the first GOP. The GOP is a group of pictures for efficiently managing a plurality of pictures. In FIGS. 4 and 5, an I picture is located at the head of each GOP. 4 and 5, the transmission source device transmits the encoded data to MCU 1 in the order of I1, P11, P12, I2, P21, P22, and I3. The transmission source device generates P11 with reference to I1, and generates P12 with reference to P11. Similarly, P21 is generated with reference to I2, and P22 is generated with reference to P21.

  FIG. 4 is a diagram illustrating an example of a result of processing when unreceived information is not input from the transmission destination apparatus to the MCU 1. First, the MCU 1 receives I1 from the transmission source device and stores it in the received encoded data storage area 122. At this time, the value of the bandwidth of the network 5 between the MCU 1 and the transmission destination device is equal to or greater than the threshold value. Therefore, the received I1 is transmitted as it is to the transmission destination device. The transmitted I1 is decoded, and the generated decoded transmission data is stored in the decoded transmission data storage area 124.

  MCU 1 then receives and stores P11. Since the bandwidth has not yet decreased, P11 is transmitted as it is, and P11 is decoded and stored. Note that the transmission destination apparatus receives I1 that is reference data necessary for decoding of P11 before P11, so that P11 can be decoded and an image can be displayed on the display device 9. The MCU 1 transmits P12 to the transmission destination device as it is, and stores the decoded transmission data of P12.

  MCU1 then receives I2 and stores it. At this point, the bandwidth of the network 5 with the transmission destination device has decreased. Therefore, the MCU 1 decodes I2, and stores the generated decoded reception data of I2 in the decoded reception data storage area 123. Using the decoded transmission data generated from P12 that has already been transmitted as the referenced data, the decoded reception data of I2 is interframe-coded to generate P13. P13 is transmitted to the transmission destination device, and the decoded transmission data of P13 is stored. Since the transmission destination apparatus receives P12, which is reference data necessary for decoding of P13, before P13, P13 can be decoded and an image can be displayed.

  Next, MCU1 receives and stores P21. Since the bandwidth is reduced, the MCU 1 decodes P21 with reference to the stored I2 and stores the decoded reception data of P21. Using the decoded transmission data of P13 already transmitted as the referenced data, the decoded reception data of P21 is interframe-coded to generate P14. P14 is transmitted to the transmission destination device, and the decoded transmission data of P14 is stored. Next, the same process as when P21 is received is performed for P22. Thereafter, MCU1 receives I3, but since the band value has recovered to the threshold value or more, I3 is transmitted to the destination device as it is.

  As described above, if the received encoded data is an I picture, the MCU 1 can newly generate and transmit a P picture having a data size smaller than that of the I picture according to the bandwidth of the network 5. Therefore, the MCU 1 can reduce the load applied to the network 5 without reducing the frame rate. The MCU 1 can quickly reduce the bit rate of the encoded data to be transmitted to the transmission destination device even after the transmission source device has already transmitted the I picture. By performing processing by comparing the value of the bandwidth with the threshold value, it is possible to appropriately perform processing as the bandwidth increases or decreases.

  The MCU 1 generates a new P picture using the decoded transmission data as the referenced data even when the band is reduced and the received encoded data is a P picture. In this case, since the received encoded data and newly generated encoded data are both P pictures, there is no large difference in data size. However, P pictures are regenerated by referring to images transmitted in the past to the transmission destination device. Therefore, the data that is referred to at the time of encoding in MCU 1 and the data that is referred to at the time of decoding in the transmission destination device are completely the same data. Therefore, the quality of the video after decoding does not deteriorate.

  The MCU 1 decodes the transmitted encoded data whenever the encoded data is transmitted to the transmission destination device. The decoded transmission data generated by decoding is stored in the decoded transmission data storage area 124. Accordingly, when a new P picture is generated, the MCU 1 can immediately perform inter-frame encoding with reference to the stored decoded transmission data. Therefore, unlike the case where the decoded transmission data is generated after the bandwidth is lowered, the bandwidth load can be quickly reduced after the bandwidth is lowered.

  The MCU 1 can generate a new P picture by referring to the decoded transmission data of the encoded data transmitted one frame before the P picture to be generated. Accordingly, the prediction error is reduced and the data size of the P picture is reduced as compared with the case of referring to data two frames or more before. Therefore, the MCU 1 can efficiently reduce the load applied to the band.

  Next, the example shown in FIG. 5 will be described. FIG. 5 is a diagram illustrating an example of a result of processing when unreceived information is input from the transmission destination apparatus to the MCU 1. First, the MCU 1 receives P12 from the transmission source device and stores it. At this time, the value of the bandwidth of the network 5 with the transmission destination device is equal to or greater than the threshold value. Therefore, the received P12 is transmitted as it is to the transmission destination device. P12 is decoded and the generated decoded transmission data is stored.

  MCU1 then receives I2 and stores it. Since the bandwidth is reduced, MCU1 decodes and stores I2, and generates P13 with reference to the decoded transmission data of P12. P13 is transmitted to the transmission destination device, and the decoded transmission data of P13 is stored. The MCU 1 generates the P 14 by performing the same process as that when the I 2 is received for the P 21 to be received next, and transmits and stores it.

  MCU 1 then receives and stores P22. Since the bandwidth is reduced, a new P picture is generated from P22. However, at this time, the MCU 1 has input unreceived information indicating that the reception of P12 has failed from the transmission destination device. Therefore, the MCU 1 refers to the decoded transmission data of P11 which is the latest data among the encoded data successfully transmitted in the past, and inter-codes the decoded reception data of P22. The generated encoded data of P15 is transmitted to the transmission destination apparatus, and the decoded transmission data of P15 is stored. MCU 1 then receives and stores I3. Using the decoded transmission data of P15 as the referenced data, P16 is generated from the decoded reception data of I3 and transmitted to the transmission destination device.

  As described above, when the transmission destination apparatus fails to receive the encoded data, the MCU 1 uses the decoded transmission data based on the encoded data transmitted to the transmission destination apparatus before the encoded data that has failed to be received as the reference data. A P picture can be generated. As shown in FIG. 5, if the destination apparatus fails to receive one picture (P12 in FIG. 5), it may not be able to decode in conjunction until the next I picture is received. However, the MCU 1 can newly generate a P picture using the picture that has been successfully transmitted as reference data (P15 in FIG. 5). Therefore, in the transmission destination device, decoding of the encoded data after P15 is not impossible in conjunction with the reception failure of P12. Accordingly, the MCU 1 can minimize the deterioration of the video quality even when the transmission destination apparatus fails to receive the encoded data.

  In the first embodiment, the MCU 1 corresponds to the “communication control device” of the present invention. The communication device 3 and the PC 4 correspond to “communication device”. The CPU 10 that receives the encoded data in S3 of FIG. 3 functions as a “reception unit”, and the received encoded data that is the encoded data received in S3 corresponds to “first encoded data”. The CPU 10 that decodes the received encoded data in S8 and S9 in FIG. 3 functions as a “first decoding unit”. The CPU 10 that transmits the encoded data in S5 and S14 of FIG. 3 functions as a “transmission unit”, and the transmission encoded data that is the encoded data transmitted in S5 and S14 corresponds to “second encoded data”. The CPU 10 that stores the decoded transmission data in the RAM 12 in S16 of FIG. 3 functions as a “storage control unit”. The CPU 10 that measures the bandwidth of the network 5 in S2 of FIG. 3 functions as “measuring means”. The CPU 10 that generates the P picture in S4 and S7 to S13 in FIG. 3 functions as a “generating unit”. The CPU 10 that determines whether or not the band value is lower than the threshold value in S4 of FIG. 3 functions as a “band determining unit”. The CPU 10 that inputs unreceived information in S10 of FIG. 3 functions as an “input unit”.

  The process of receiving the encoded data in S3 of FIG. 3 corresponds to the “reception step” of the present invention. The process of decoding the received encoded data in S8 and S9 in FIG. 3 corresponds to the “first decoding step”. The process of transmitting the encoded data in S5 and S14 in FIG. 3 corresponds to a “transmission step”. The process of storing the decoded transmission data in the RAM 12 in S16 of FIG. 3 corresponds to a “storage control step”. The process of measuring the bandwidth of the network 5 in S2 of FIG. 3 corresponds to a “measurement step”. The process of generating a P picture in S4 and S7 to S13 of FIG. 3 corresponds to a “generation step”.

  Next, the MCU 101 according to the second embodiment of the present invention will be described with reference to FIG. 6 and FIG. The MCU 101 according to the second embodiment generates a new P picture only when the received encoded data is an I picture even when the band value is lower than the threshold. Only the MCU 1 according to the embodiment is different. Therefore, the same number is attached | subjected about the same structure and process, and this description is abbreviate | omitted or simplified.

  A main process performed by the MCU 101 will be described with reference to FIG. First, various data are initialized (S1), and bandwidth measurement is started (S2). Encoded data for one picture is received from the transmission source device and stored in the received encoded data storage area 122 (S3). It is determined whether or not the measured bandwidth value has fallen below a threshold value (S104). If the band value is equal to or greater than the threshold value (S104: NO), the received encoded data is transmitted as it is to the transmission destination device (S5). The transmitted encoded data is decoded, decoded transmission data is generated, and stored in the decoded transmission data storage area 124 (S16). The process returns to S3.

  If the measured band value is lower than the threshold value (S104: YES), it is determined whether the received encoded data is an I picture (S107). If it is not an I picture but a P picture (S107: NO), the received P picture is transmitted as it is to the transmission destination device (S5). Only when the received encoded data is an I picture (S107: YES), the received I picture is decoded to generate decoded received data, which is stored in the decoded received data storage area 123 (S108). It is determined whether or not unreceived information is input from the transmission destination device (S110). If unreceived information is not input (S110: NO), the decoded transmission data immediately before the P picture to be generated is determined as the referenced data for inter-frame encoding of the P picture (S111). If unreceived information has been input (S110: YES), the decoded transmission data of the latest data among the encoded data successfully transmitted is determined as the referenced data (S112).

  Next, the decoded received data that has been decoded (S108) is interframe-coded with reference to the determined referenced data, and a new P picture is generated (S113). The generated P picture is transmitted to the transmission destination device (S14). The transmitted P picture is decoded, decoded transmission data is generated, and stored in the decoded transmission data storage area 124 (S16). The process returns to S3, and the next picture is processed.

  With reference to FIG. 7, an example of the result of the process performed by the MCU 101 of the second embodiment will be described. First, the MCU 101 receives I1 from the transmission source device and stores it in the received encoded data storage area 122. At this time, the value of the bandwidth of the network 5 between the MCU 101 and the transmission destination device is equal to or greater than the threshold value. Therefore, the received I1 is transmitted as it is to the transmission destination device. The transmitted I1 is decoded, and the generated decoded transmission data is stored in the decoded transmission data storage area 124. Next, the MCU 101 receives P11 and stores it. Since the bandwidth has not yet decreased, P11 is transmitted as it is, and P11 is decoded and stored. The MCU 101 transmits P12 as it is to the transmission destination apparatus, and stores the decoded transmission data of P12.

  Next, the MCU 101 receives I2 and stores it. At this point, the bandwidth of the network 5 with the transmission destination device has decreased. Further, the received encoded data is an I picture. Therefore, the MCU 1 decodes I2, and stores the generated decoded reception data of I2 in the decoded reception data storage area 123. The decoded transmission data based on P12 that has already been transmitted is used as reference data, and the decoded reception data of I2 is interframe-coded to generate P13. P13 is transmitted to the transmission destination device, and the decoded transmission data of P13 is stored.

  Next, the MCU 101 receives and stores P21. At this point, the bandwidth is reduced, but the received encoded data is a P picture. Accordingly, the MCU 101 transmits P21 as it is to the transmission destination apparatus, and stores the decoded transmission data of P21. The transmission destination device decodes the encoded data of P21 with reference to P13. Since P21 uses I2 as reference data at the time of encoding in the transmission source device, strictly speaking, reference data (I2) at the time of encoding and reference data (P13) at the time of decoding are different. However, both I2 and P13 are encoded data for the same image. Therefore, even if P13 is used as the reference data at the time of decoding, the quality of the displayed video is slightly reduced. The MCU 101 transmits P22 as it is to the transmission destination device, and stores the decoded transmission data of P22. Next, the MCU 101 receives I3, but since the bandwidth has been restored, I3 is transmitted to the destination device as it is.

  As described above, the MCU 101 according to the second embodiment once decodes received encoded data only when the band is reduced and the received encoded data received from the transmission source device is an I picture. Then change it to a P picture. Even when the bandwidth is reduced, the MCU 101 transmits the received encoded data as it is to the transmission destination device when the received encoded data is a P picture. Since the MCU 101 only needs to change only the I picture to the P picture, it is possible to efficiently reduce the load applied to the network 5 without abruptly increasing the processing load. Note that the CPU 10 that determines whether or not the received encoded data is an I picture in S107 in FIG. 6 functions as the “picture determination unit” of the present invention.

  It goes without saying that the present invention is not limited to the above-described embodiment, and various modifications are possible. For example, the present invention is not limited to the MCUs 1 and 101 that control a video conference. In addition, the present invention can be applied to any apparatus that controls transmission / reception of encoded data between a plurality of apparatuses, such as a server that distributes video. In the above embodiment, the Coding is performed based on the H.264 standard, but other standards may be adopted.

  When the bandwidth of the network 5 with the transmission destination apparatus falls below the threshold value, the MCUs 1 and 101 of the above embodiment change the I picture into a P picture and transmit it. Here, it is also possible to change the ratio of changing the I picture to the P picture in accordance with the band value. For example, the ratio of changing an I picture to a P picture is 0% if the bandwidth value is A or more, 25% if less than A and B or more, 50% if less than B and C, and if less than C, Set to 90%. The MCU sets the rate of recreating the I picture according to the band instead of S4 in FIG. When changing an I picture to a P picture, the number of received I pictures is counted between S4 and S7 in FIG. When the ratio is 25%, every time four I pictures are received, the I picture is changed to a P picture. Similarly, if the ratio is set to 50%, one out of two I pictures is changed to nine out of ten I pictures. In this case, the MCU can increase the rate of making an I picture into a P picture as the bandwidth decreases. Therefore, the MCU can adjust the output bit rate more appropriately according to the amount of bandwidth reduction. It goes without saying that the threshold value in the above embodiment can be changed as appropriate.

  The MCUs 1 and 101 of the above embodiment decode all the encoded data transmitted to the transmission destination device so that the P picture can be quickly regenerated when the bandwidth is reduced, and always store at least the latest decoded transmission data. Yes. However, the decoded transmission data may be generated after the bandwidth is reduced, and the P picture may be regenerated. In addition, the MCUs 1 and 101 of the above embodiment decode the received encoded data and store the decoded received data only when regenerating the P picture. However, all of the received encoded data may be decoded and stored.

  As described in the above embodiment, when a P picture is regenerated from decoded received data, it is desirable to use the picture immediately before the decoded received data as reference data. However, the present invention can be realized without using the immediately preceding picture as reference data.

1,101 MCU
5 Network 10 CPU
12 RAM
13 HDD
122 reception encoded data storage area 123 decoded reception data storage area 124 decoded transmission data storage area

Claims (9)

A communication control device that connects to a plurality of communication devices via a network and controls transmission / reception of encoded data that is encoded moving image data between the plurality of communication devices,
A receiving unit that receives the encoded data transmitted by a transmission source device that is a communication device that transmits encoded data to the network among the plurality of communication devices;
First decoding means for generating decoded reception data which is image data generated as a result of decoding by decoding first encoded data which is the encoded data received by the receiving means;
A transmitting unit that transmits the encoded data to a transmission destination device that is a communication device that is a transmission destination of the encoded data among the plurality of communication devices;
Second decoding means for generating decoded transmission data that is image data generated as a result of decoding by decoding the second encoded data that is the encoded data transmitted by the transmission means;
Storage control means for storing in the storage means the decoded transmission data generated by the second decoding means;
Measuring means for measuring the bandwidth of the network with the destination device;
Decoding reception generated by the first decoding unit from an I picture (Intra-coded Picture) that is encoded data that is encoded using only intra-frame predictive encoding according to the band measured by the measuring unit Generating means for inter-frame encoding data with reference to the decoded transmission data stored in the storage means to generate a P-picture (Predictive-coded Picture) as encoded data;
The transmission means includes
When a P picture is generated by the generation means, the generated P picture is transmitted to the transmission destination device as the second encoded data,
The communication control apparatus, wherein when the P picture is not generated by the generating means, the first encoded data received by the receiving means is transmitted to the transmission destination apparatus as the second encoded data. A band determination unit that determines whether or not the band measured by the measurement unit is lower than a threshold;
The communication control apparatus according to claim 1, wherein the generation unit generates a P picture when the band determination unit determines that the band is lower than a threshold value. The first decoding unit generates the decoded reception data from the first encoded data when the band determination unit determines that the band is lower than a threshold,
The said generation means produces | generates a P picture by carrying out the inter-frame encoding of the decoding reception data produced | generated by the said 1st decoding means from said 1st encoding data other than the said I picture. Communication control device. Picture determining means for determining whether or not the first encoded data received by the receiving means is the I picture when the band determining means determines that the band is lower than a threshold;
The generating means generates a P picture from decoded received data obtained by decoding the I picture only when the picture determining means determines that the first encoded data is the I picture. The communication control apparatus according to claim 2.   5. The storage control unit according to claim 1, wherein at least the latest decoded transmission data among the decoded transmission data generated by the second decoding unit is always stored in the storage unit. 6. Communication control device.   The generation unit generates a P picture by referring to decoded transmission data of second encoded data transmitted by the transmission unit one frame before a P picture to be generated. The communication control apparatus according to any one of 5. From the transmission destination device further comprises an input means for inputting unreceived information that is information indicating that reception of the second encoded data has failed,
The generation means, when the unreceived information is input by the input means, is transmitted to the transmission destination apparatus before the second encoded data in which the reception failure is indicated by the unreceived information. 6. The communication control device according to claim 1, wherein a P picture is generated with reference to the decoded transmission data of encoded data. A communication control method for controlling transmission / reception of encoded data, which is encoded moving image data, performed by a communication control device connected to a plurality of communication devices via a network, between the plurality of communication devices. ,
A receiving step of receiving the encoded data transmitted by a transmission source device that is a communication device that transmits encoded data to the network among the plurality of communication devices;
A first decoding step of generating decoded reception data that is image data generated as a result of decoding by decoding the first encoded data that is the encoded data received by the reception step;
A transmission step of transmitting encoded data to a transmission destination device that is a communication device of a transmission destination of encoded data among the plurality of communication devices;
A second decoding step of generating decoded transmission data that is image data generated as a result of decoding by decoding the second encoded data that is the encoded data transmitted by the transmission step;
A storage control step for storing in the storage means the decoded transmission data generated by the second decoding step;
A measuring step of measuring a bandwidth of the network with the destination device;
Decoding reception generated by the first decoding step from an I picture (Intra-coded Picture) that is encoded data that is encoded using only intra-frame predictive encoding according to the band measured by the measuring step Generating a P picture (Predictive-coded Picture) that is encoded data by inter-coding the data with reference to the decoded transmission data stored in the storage means,
The transmitting step includes
When a P picture is generated by the generation step, the generated P picture is transmitted as the second encoded data to the transmission destination device.
When a P picture is not generated by the generating step, the communication control method, wherein the first encoded data received by the receiving step is transmitted to the transmission destination device as the second encoded data. A communication control program that performs control to transmit and receive encoded data that is encoded moving image data between a plurality of communication devices via a network,
On the computer,
A receiving step of receiving the encoded data transmitted by a transmission source device that is a communication device that transmits encoded data to the network among the plurality of communication devices;
A first decoding step of generating decoded reception data that is image data generated as a result of decoding by decoding the first encoded data that is the encoded data received by the reception step;
A transmission step of transmitting encoded data to a transmission destination device that is a communication device of a transmission destination of encoded data among the plurality of communication devices;
A second decoding step of generating decoded transmission data that is image data generated as a result of decoding by decoding the second encoded data that is the encoded data transmitted by the transmission step;
A storage control step for storing in the storage means the decoded transmission data generated by the second decoding step;
A measuring step of measuring a bandwidth of the network with the destination device;
Decoding reception generated by the first decoding step from an I picture (Intra-coded Picture) that is encoded data that is encoded using only intra-frame predictive encoding according to the band measured by the measuring step Generating the P-picture (Predictive-coded Picture) as encoded data by performing inter-frame encoding with reference to the decoded transmission data stored in the storage means,
Furthermore, in the transmission step,
When a P picture is generated by the generation step, the generated P picture is transmitted to the transmission destination device as the second encoded data,
A communication control program for causing the transmission destination device to transmit the first encoded data received in the reception step as the second encoded data when a P picture is not generated in the generation step.

Images