JP2011109176A - Device and method for multiplexing video, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To transmit optimum high-quality video according to a transmission rate without lowering a coding rate excessively. <P>SOLUTION: A coding unit 201 codes and multiplexes each of a plurality of streams at a plurality of coding rates. A reference information creation unit 203 calculates a quality evaluation value to each of the plurality of streams coded at the plurality of coding rates, and creates reference information for identifying a combination where a sum total of the coding rates of the plurality of streams becomes within a transmission band and the minimum of the quality evaluation value becomes higher. A command conversion unit 211 instructs a combination of a prescribed number of streams to be selected by a data management unit 205 based on the reference information. The data management unit 205 takes out a prescribed number of coded streams from a data storage unit 202. A data transmission unit 207 transmits the prescribed number of streams. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a video multiplexing apparatus, a video multiplexing method, and a program for transmitting video having a plurality of synchronized stream configurations within a certain transmission band.

  As an encoding method for compressing a plurality of camera images together, the international standard encoding method H.264 is used. Multi-view video coding (MVC) is standardized as H.264 Annex H (see Non-Patent Document 1, for example). In MVC, a plurality of encoded video data is compressed and encoded in a multiplexed form. Encoded data of frames having the same time stamp are continuously multiplexed. That is, the encoded data is a stream in which a plurality of video data is multiplexed. Since frames with the same time stamp are continuously multiplexed, the decoder can decode and output MVC encoded data, thereby successively decoding and outputting a plurality of camera images with the same time stamp. it can.

  FIG. 26 is a block diagram showing a configuration of a video transmission system according to the prior art. In the figure, the conventional video transmission system includes a multiplexing device 1, a video transmission server 2, and a video reproduction client 3. In the multiplexing apparatus 1, the encoding unit 101 encodes and multiplexes M input streams at the same fixed encoding rate, and generates multiplexed data. The data storage unit 102 stores multiplexed data including the M streams.

  In the video transmission server 2, the data transmission unit 103 extracts N (≦ M) streams from the data storage unit 102 and sends them to the video reproduction client 3. Here, in order to transmit N streams, a transmission rate corresponding to N × fixed coding rate is required.

  In the video reproduction client 3, the data receiving unit 104 receives N streams. The decoding unit 105 decodes N streams. The video display unit 106 displays a partial video.

  Here, a video composed of M streams (whole stream) is referred to as “whole video”, a video composed of N (≦ M) streams (partial streams) is referred to as “partial video”, A video composed of streams is defined as “unit video”. That is, the whole video is composed of M unit videos, and the partial video is composed of N unit videos.

Hideaki Kimata, Shinya Shimizu, Yutaka Kunita, Megumi Isogai, and Yoshimitsu Ohtani, "Panorama video coding for user-driven interactive video application," IEEE International Symposium on Consumer Electronics (ISCE) 2009, 2009.

  In the above-described conventional technology, there is a problem in that a partial video composed of N partial streams cannot be transmitted when transmission rate <N × fixed coding rate. Further, when the encoding rate is lowered too much so that the transmission rate ≧ N × fixed encoding rate, there is a problem that the image quality is deteriorated.

  The present invention has been made in view of such circumstances, and its purpose is video multiplexing capable of transmitting an optimal high-quality video according to the transmission rate without excessively reducing the encoding rate. An apparatus, a video multiplexing method, and a program are provided.

  In order to solve the above-described problem, the present invention is a video multiplexing apparatus that encodes and multiplexes a plurality of encoded streams and transmits the plurality of streams in a predetermined transmission band. Encoding means for encoding and multiplexing at an encoding rate; selection means for selecting a predetermined number of streams from a plurality of streams encoded at a plurality of code rates by the encoding means; and the encoding means An evaluation value calculating means for calculating a quality evaluation value for each of a plurality of streams encoded at a plurality of encoding rates, and a sum of encoding rates of the predetermined number of streams selected by the selecting means is the predetermined Reference information for identifying a combination of the predetermined number of streams that is within the transmission band of the predetermined number of streams and that has a higher minimum quality evaluation value of the predetermined number of streams. Reference information creating means, selection instruction means for instructing a combination of the predetermined number of streams to be selected by the selecting means based on the reference information created by the reference information creating means, and the selecting means And a transmission means for transmitting the predetermined number of streams finally selected by the video multiplexing apparatus.

  The present invention further comprises video dividing means for generating a plurality of streams by dividing the input video data into small areas in the above-mentioned invention, wherein the encoding means includes a plurality of parts divided by the video dividing means. Each of the streams is encoded and multiplexed at a plurality of encoding rates.

  The present invention further includes designation means for designating the combination of the predetermined number of streams in the above invention, and the selection instruction means is instructed to change the combination of the predetermined number of streams by the designation means. And a new instruction for the combination of the predetermined number of streams to be selected by the selection unit based on the reference information created by the reference information creation unit.

  In the above invention, the present invention further comprises designation means for designating the combination of the predetermined number of streams, and the reference information creating means is instructed to change the combination of the predetermined number of streams by the designation means. Then, the sum of the coding rates of the predetermined number of streams included in the combination instructed to be changed is within the predetermined transmission band, and the quality of the predetermined number of streams included in the combination instructed to be changed Reference information for specifying a combination of the predetermined number of streams, in which the lowest evaluation value becomes higher, is newly created.

  In order to solve the above-described problem, the present invention is a video multiplexing method for encoding and multiplexing a plurality of encoded streams and transmitting the encoded streams in a predetermined transmission band. Encoding at a plurality of encoding rates, multiplexing, selecting a predetermined number of streams from a plurality of streams encoded at the plurality of encoding rates, and encoding at the plurality of encoding rates Calculating a quality evaluation value for each of the plurality of streams, a sum of encoding rates of the predetermined number of streams is within the predetermined transmission band, and a minimum quality evaluation value of the predetermined number of streams A step of creating reference information for identifying a combination of the predetermined number of streams, which has a higher value, and a selection method based on the created reference information. A step of instructing a combination of said predetermined number of streams finally selected, a video multiplexing method characterized by including the step of transmitting the stream of the predetermined number.

  According to the present invention, in the above invention, the step of generating a plurality of streams by dividing the input video data into small regions, and encoding each of the plurality of divided streams at a plurality of encoding rates. And the step of multiplexing.

  According to the present invention, in the above invention, when the step of designating the combination of the predetermined number of streams and the combination of the predetermined number of streams are instructed to be changed, the selection should be made based on the reference information. A step of newly instructing a combination of the predetermined number of streams.

  According to the present invention, in the above invention, the step of designating a combination of the predetermined number of streams and the predetermined number included in the combination instructed to change when the combination of the predetermined number of streams is instructed to be changed. The predetermined number of streams in which the sum of the encoding rates of the streams is within the predetermined transmission band, and the minimum value of the quality evaluation values of the predetermined number of streams included in the combination instructed to change is higher And a step of newly creating reference information for specifying a combination of these.

  Further, in order to solve the above-described problem, the present invention encodes and multiplexes a plurality of encoded streams and transmits each of the plurality of streams to a computer of a video multiplexing apparatus that transmits in a predetermined transmission band. An encoding function for encoding and multiplexing at a plurality of encoding rates, a selection function for selecting a predetermined number of streams from a plurality of streams encoded at a plurality of encoding rates by the encoding function, and the encoding function An evaluation value calculation function for calculating a quality evaluation value for each of a plurality of streams encoded at a plurality of encoding rates, and a sum of encoding rates of the predetermined number of streams selected by the selection function is the predetermined value The combination of the predetermined number of streams that is within the transmission band and that has the lowest minimum quality evaluation value of the predetermined number of streams is identified. A reference information creation function for creating information, a selection instruction function for instructing a combination of the predetermined number of streams to be selected by the selection function based on the reference information created by the reference information creation function, and the selection function A program that realizes a transmission function for transmitting the predetermined number of streams finally selected by.

  According to the present invention, it is possible to transmit an optimum high-quality video according to the transmission rate without excessively reducing the encoding rate.

It is a block diagram which shows the structure of the video transmission system by 1st Embodiment of this invention. It is a flowchart for demonstrating the evaluation value calculation procedure with respect to each data ID by this 1st Embodiment. It is a flowchart for demonstrating the preparation procedure of the reference information required in the case of command conversion in this 1st Embodiment. It is a conceptual diagram which shows the corresponding information of stream ID and data ID in the 1st embodiment. It is a flowchart for demonstrating the detailed procedure of the reference information preparation in this 1st Embodiment. It is a conceptual diagram which shows the priority data ID information for the image quality improvement in the 1st embodiment. It is a conceptual diagram which shows the example of creation of the stream request information in the first embodiment. It is a conceptual diagram which shows the reference information in this 1st Embodiment. It is a flowchart for demonstrating the command transmission procedure of the video reproduction client 30 in this 1st Embodiment. It is a flowchart for demonstrating the command reception of the video transmission server 20 in this 1st Embodiment, and a data transmission procedure. It is a flowchart for demonstrating the data reception of the video reproduction client 30 in this 1st Embodiment, and a video display procedure. It is a block diagram which shows the structure of the video transmission system by this 2nd Embodiment. It is a conceptual diagram which shows an example of the video division | segmentation by the video division part 301 in this 2nd Embodiment. It is a flowchart for demonstrating the calculation procedure of SNR with respect to each data ID in this 2nd Embodiment. It is a conceptual diagram which shows data ID provided with the bit rate (500 kbps) in the 2nd embodiment. It is a conceptual diagram which shows data ID provided with the bit rate (1 Mbps) in the 2nd embodiment. It is a conceptual diagram which shows data ID provided with the bit rate (1.5 Mbps) in this 2nd Embodiment. It is a conceptual diagram which shows the correspondence information of tile ID in this 2nd Embodiment and video coding data ID. It is a conceptual diagram which shows the example of the display pattern in the 2nd embodiment. It is a conceptual diagram which shows the example of creation of the stream request information in the second embodiment. It is a conceptual diagram which shows the reference information in this 2nd Embodiment. It is a block diagram which shows the structure of the video transmission system by 3rd Embodiment of this invention. It is a block diagram which shows the structure of the video transmission system by 4th Embodiment of this invention. It is a flowchart for demonstrating the reference information preparation procedure in the multiplexing apparatus 10 by 5th Embodiment of this invention. It is a conceptual diagram which shows the example of the ascending order sort of data ID which made the evaluation value the key in the multiplexing apparatus 10 in this 5th Embodiment. It is a block diagram which shows the structure of the video transmission system by a prior art.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
A. First Embodiment FIG. 1 is a block diagram showing a configuration of a video transmission system according to a first embodiment of the present invention. In the multiplexing apparatus 10, the encoding unit 201 receives video having synchronized M stream configurations, encodes and multiplexes each stream at a plurality of encoding rates, and generates video data. The data storage unit 202 stores this video data. The reference information creation unit 203 creates reference information using the video data stored in the data storage unit 202 and stores the reference information in the reference information storage unit 204. Note that data multiplexed in the order of reference information and video data may be stored in the data storage unit 202.

  In the video transmission server 20, the data management unit 205 makes N (≦ M) stream requests to the data storage unit 202 in accordance with the “stream request information” received from the command reception unit 206 from the video reproduction client 30. Thus, N streams are extracted from the M streams stored in the data storage unit 202 and transferred to the data transmission unit 207. The data transmission unit 207 transmits N streams to the video reproduction client 30. The command receiving unit 206 receives a command from the video playback client 30. The reference information transmission unit 208 takes out the reference information data from the reference information storage unit 204 and sends it to the video reproduction client 30.

  In the video reproduction client 30, the data reception unit 212 receives N streams. The decoding unit 213 decodes N streams. The video display unit 214 displays the decoded partial video. The reference information receiving unit 216 receives the reference information transmitted from the video transmission server 20 and stores it in the reference information storage unit 210. The stream combination designation unit 215 accepts a change input for stream combination designation. When the stream combination designation is changed, the command conversion unit 211 refers to the reference information stored in the reference information storage unit 210 and converts “stream combination information” into “stream request information”. The command transmission unit 209 transmits a command to the video transmission server 20 in accordance with “stream request information”.

  It should be noted that the data storage unit 202 and the reference information storage unit 204 in the multiplexing device 10 are functionally integrated, the data transmission unit 207 and the reference information transmission unit 208 in the video transmission server 20 are functionally integrated, and the data in the video playback client 30 is integrated. A configuration in which the reception unit 212 and the reference information reception unit 216 are functionally integrated is also possible.

  In FIG. 1, in the video playback client 30, the command conversion unit 209 converts “stream combination information” into “stream request information” with reference to “reference information”. By having “reference information” with the video playback client 30, it is also possible to implement a configuration in which the command transmission unit 209 directly transmits “stream combination information” as a command without performing command conversion in the command conversion unit 211. is there. In the case of this configuration, “stream combination information” can be converted to “stream request information” by referring to the reference information on the video transmission server 20 side.

Next, the video multiplexing / transmission method of the first embodiment will be described in detail.
In the first embodiment, for example, when video composed of 12 synchronized input streams is transmitted at 10 Mbps, reference information is transmitted before video data is transmitted. The reference information creation procedure in the reference information creation unit 203 in the multiplexing apparatus 10 shown in FIG. 1 uses an evaluation value calculation procedure for a result obtained by encoding each input stream at a plurality of bit rates, and an evaluation value calculation result. The reference information creation procedure.

  FIG. 2 is a flowchart for explaining an evaluation value calculation procedure for each data ID by the encoding unit 201 and the reference information creation unit 203 according to the first embodiment. First, the encoding unit 201 assigns stream IDs to a plurality of synchronized input streams. Next, each input stream is encoded at a plurality of encoding rates (bit rates) (step S001), and a data ID is assigned to the encoded stream (step S002). Subsequently, the reference information creation unit 203 creates correspondence information between the stream ID and the video encoded data ID (step S003), and finally calculates an evaluation value for each data ID (step S004).

  FIG. 3 is a flowchart for explaining a reference information creation procedure by the reference information creation unit 203 necessary for command conversion by the command conversion unit 211 according to the first embodiment. Before creating reference information, the reference information creating unit 203 first sets a “transmission rate” indicating a transmission band and a “stream combination” composed of a combination of data IDs (step S201). The “base bit rate” indicating the encoding rate to be set and the “improved bit rate” indicating the encoding rate larger than the basic encoding rate are set (step S202).

  Next, the number of data IDs targeted for image quality improvement is calculated from the data IDs at the base bit rate in the “stream combination” (step S203). Next, using the evaluation value for each data ID calculated in step S004, order information of the data ID that needs to be improved in image quality preferentially is created (step S204). The number of data IDs calculated in step S203 is changed from the data ID at the base bit rate in the “stream combination” to the data ID at the improved bit rate (step S205). Finally, reference information including correspondence information between the “stream combination information” and the “stream request information” composed of the data ID is created (step S206).

  For example, transmission rate: 10 Mbps, base bit rate: 500 kbps, improved bit rate: 1 Mbps, “stream combination” is stream ID (7, 8, 9, 13, 14, 15, 19, 20, 21, 25, 26, 27 ), When the evaluation value is an average SNR (average value of SNR (signal-noise ratio) of all frames), correspondence information between the stream ID and the data ID is as shown in FIG.

  Further, the SNR of each frame is calculated by the following equation (1).

SNR = 10 * log ₁₀ (MAX ² / MSE) (1)

  The unit is decibel [dB]. MAX is the maximum pixel value that the original image can take. For example, the value of MAX when the bit depth is 8 bits is 255. The MSE in Equation (1) is a mean square error, and the square sum of the difference between the tile-size original image and the local decoded image is divided by the number of pixels of the tile size. Desired.

  In the case of an encoding mode (for example, full intra mode) in which a local decoded image cannot be obtained by the encoding unit 201, local decoded image creation (decoding) needs to be performed separately from the encoding procedure. .

  FIG. 5 is a flowchart for explaining detailed procedures for creating reference information in the multiplexing apparatus 10 according to the first embodiment. The reference information creation unit 203 first sets the “stream combination” (stream ID (7, 8, 9, 13, 14, 15, 19, 20, 21, 25, 26, 27), Nt = 12), transmission The bit rate (Max_bitrate) is set to 10 Mbps (step S301). Next, the base bit rate (tile_bitrate_base) = 500 kbps and the improved bit rate (tile_bitrate_up) = 1 Mbps are set (step S302).

  Next, Nt (number of stream IDs of “stream combination”) × tile_bitrate_base (base bit rate) <Max_bitrate (transmission bit rate) (first condition), and M1 (Max_bitrate−Nt × tile_bitrate_base)> M2 It is determined whether (tile_bitrate_up-tile_bitrate_base) or a second condition of M1 = M2 is satisfied (step S303).

  If the first condition or the second condition is not satisfied, tile_bitrate_base = tile_bitrate_base-Δtile_bitrate_base and tile_bitrate_up = tile_bitrate_up-Δtile_bitrate_up are set (step S304), and the process returns to step S303.

  On the other hand, in the setting condition described above, Nt = 12 based on the number of stream IDs of the “stream combination”, so Nt × title_bitrate_base = 12 × 0.5 Mbps = 6 Mbps <Max_bitrate = 10 Mbps, which satisfies the first condition, Since tile_bitrate_up = 1 Mbps, M1 = 10−12 × 0.5 = 4 Mbps, M2 = 1−0.5 = 0.5 Mbps, and the second condition of M1> M2 is satisfied (YES in step S303).

  When the first condition and the second condition are satisfied, Mup = INT (4 / 0.5) = 8 is calculated (step S305), and the data IDs at the 12 base bit rates in the “stream combination” are calculated. Sorting in ascending order by using the SNR of the Y component for the key as a key (step S306). Thereby, the sort result shown in FIG. 6 is obtained.

  Next, eight are selected from Mup = 8 in ascending order of SNR (step S307), the correspondence information of “data ID” shown in FIG. 4 is referred to, and the data ID at the base bit rate tile_bitrate_base is as shown in FIG. In addition, the data ID is changed to the improved bit rate tile_bitrate_up (step S308). Finally, as shown in FIG. 8, reference information including correspondence information between “stream combination” and “stream request information” composed of data IDs is created (step S309).

  Next, a series of procedures from command transmission from the video playback client 30 to video display in the video transmission system according to the first embodiment will be described with reference to FIGS.

  FIG. 9 is a flowchart for explaining the command transmission procedure of the video reproduction client 30. First, the “stream combination” is designated by the stream combination designation unit 215 (step S501), the “stream combination” is converted to “stream request information” by the command conversion unit 211 (step S502), The command transmission unit 209 transmits a command (step S503).

  FIG. 10 is a flowchart for explaining command reception and data transmission procedures of the video transmission server 20. First, the command reception unit 206 receives a command (step S401), interprets the request data ID included in the command (step S402), and sends a stream corresponding to the request data ID from the data storage unit 202 to the data transmission unit. Transfer to 207 (step S403). Finally, the data transmission unit 207 transmits the stream (step S404).

  FIG. 11 is a flowchart for explaining the data reception and video display procedure of the video playback client 30. First, the data receiving unit 212 receives a stream (step S601), and then the decoding unit 213 decodes the stream (step S602). Finally, the decoded data is displayed on the video display unit 214 (step S603).

B. Second Embodiment Next, a second embodiment of the present invention will be described.
FIG. 12 is a block diagram showing the configuration of the video transmission system according to the second embodiment. It should be noted that portions corresponding to those in FIG. In the multiplexing apparatus 10, the video dividing unit 301 divides input video data into M streams (tile division). The encoding unit 201 encodes and multiplexes each stream at a plurality of encoding rates to create video data.

  As described above, in the second embodiment, the video dividing unit 301 divides input video data into a plurality of synchronized streams. For example, when the input video size is 3840 × 1920 pixels and the tile size is 640 × 320 pixels, the division result shown in FIG. 13 is obtained. At this time, if the video display area size is 1920 × 1080 pixels, the video display pattern is 12 patterns.

  FIG. 14 is a flowchart for explaining the SNR calculation procedure for each data ID. First, the input video of 3840 × 1920 pixels is divided into 6 × 6 = 36 tiles as shown in FIG. 13 (step S101). Next, each tile is encoded at a plurality of bit rates (for example, 500 kbps, 1 Mbps, and 1.5 Mbps) (step S102). Subsequently, a data ID is assigned to the encoding result of each tile (step S103). For example, FIGS. 15, 16, and 17 show data IDs assigned at a plurality of stages of bit rates (500 kbps, 1 Mbps, and 1.5 Mbps).

  Next, correspondence information between the tile ID and the encoded video data ID as shown in FIG. 18 is created (step S104). Finally, an average SNR for each data ID (the SNR of all frames) from the tile-size original image obtained after tile division of the input image and the tile-size local decoded image obtained in the process of the encoding unit 201. (Average value) is calculated (step S105).

  Here, the SNR of each frame is calculated by Expression (1) as in the first embodiment. In the case of an encoding mode (for example, full intra mode) in which a local decoded image cannot be obtained by the encoding unit 201, local decoded image creation (decoding) needs to be performed separately from the encoding procedure. is there.

  In the second embodiment, the reference information creation procedure required for command conversion in the command conversion unit 211 is the same as that in the first embodiment (FIG. 3), and therefore only different points will be described. To do.

  For example, as shown in FIG. 13, an example in which the tile size is 640 × 320 pixels, the input video size is 3840 × 1920 pixels, and the video display area size is 1920 × 1080 pixels will be described. In the case as shown in FIG. 13, when it is assumed that the video display area is not enlarged / reduced and only parallel movement is performed, the total display pattern is 12 patterns of display patterns P1 to P12 as shown in FIG. . The stream to which each tile ID is assigned as an input image, the result of encoding at 500 kbps shown in FIG. 15, the result of encoding at 1 Mbps shown in FIG. 16, and the result of encoding at 1.5 Mbps shown in FIG. When the data ID is assigned to the converted result, the correspondence relationship between the tile ID and the data ID can be represented in FIG.

  Here, in the second embodiment, a case where the transmission rate is 10 Mbps, the base bit rate is 500 kbps, the improved bit rate is 1 Mbps, or 1.5 Mbps, and the evaluation value is an average SNR (average value of SNR of all frames) will be described. The reference information creation procedure in the multiplexing apparatus 10 according to the second embodiment is basically the same as that in the first embodiment (FIG. 5), and will be described with reference to FIG.

  In step S301, the “display pattern P6” shown in FIG. 19 is set as the “stream combination”. Since the transmission rate is 10 Mbps, Max_bitrate = 10 Mbps is set. Next, the case where the base bit rate title_bitrate_base = 500 kbps and the improved bit rate tile_bitrate_up = 1 Mbps are set in step S302 will be described.

  Since the number of tiles in the “display pattern P6” shown in FIG. 19 is 12 tiles, Nt = 12. Therefore, in step S303, Nt × title_bitrate_base = 12 × 0.5 Mbps = 6 MbpsMax_bitrate = 10 Mbps, which satisfies the condition, and further, when tile_bitrate_up = 1 Mbps, M1 = 10−12 × 0.5 = 4 Mbps, M2 = 1-0 .5 = 0.5 Mbps, which satisfies the condition of M1> M2.

  If the above condition is satisfied, Mup = INT (4 / 0.5) = 8 can be calculated in step S305. Next, the data ID at the base bit rate corresponding to the 12 tile IDs in the “display pattern P6” is selected from the correspondence information between the tile ID and the video encoded data ID shown in FIG. 18, and in step S306, Sorting in ascending order using the average SNR of the Y component relating to the data ID as a key. Thereby, the sort result shown in FIG. 6 is obtained.

  Next, from Mup = 8 calculated in step S305, eight are selected in ascending order of SNR in step S307, and in step S308, the correspondence information of “data ID” shown in FIG. 18 is referred to, and the base bit rate tile_bitrate_base is selected. Is changed to the data ID at the improved bit rate tile_bitrate_up. Finally, in step S309, as shown in FIG. 8, reference information including correspondence information of “stream request information” composed of “stream combination” and data ID is created.

  On the other hand, when tile_bitrate_up = 1.5 Mbps is set, in step S303, M1 = 10−12 × 0.5 = 4 Mbps, M2 = 1.5−0.5 = 1.0 Mbps, and the condition of M1> M2 Therefore, in step S305, Mup = INT (4 / 1.0) = 4 can be calculated.

  Next, in step S306, sorting is performed in ascending order using the SNRs of the Y components related to the data IDs corresponding to the number of twelve tiles in the display pattern P6 as keys. Thereby, the sort result shown in FIG. 6 is obtained. Next, from Mup = 4 calculated in S305, four are selected in ascending order of SNR in step S307, and in step S308, the correspondence information of “data ID” shown in FIG. 18 is referred to, and the base bit rate tile_bitrate_base is set. As shown in FIG. 20, the data ID is changed to the data ID at the improved bit rate tile_bitrate_up. Finally, in step S309, as shown in FIG. 21, reference information including correspondence information between “stream combination” and “stream request information” composed of the data ID is created.

  In the second embodiment, a series of procedures from command transmission to video display in the video playback client is the same as that in the first embodiment described above (FIGS. 9 to 11), and a description thereof will be omitted.

C. Third Embodiment Next, a third embodiment of the present invention will be described.
FIG. 22 is a block diagram showing the configuration of the video transmission system according to the third embodiment. It should be noted that portions corresponding to those in FIG. In the multiplexing apparatus 10, the evaluation value calculation unit 403 calculates an evaluation value using the video data stored in the data storage unit 402 and stores the evaluation value in the calculation value storage unit 404. Note that data multiplexed in the order of evaluation values and video data may be stored in the data storage unit 402.

  In the video transmission server 20, the evaluation value transmission unit 408 extracts the evaluation value information from the evaluation value storage unit 404 and sends it to the video reproduction client.

  In the video reproduction client 30, the evaluation value receiving unit 415 receives the evaluation value information transmitted from the video transmission server 20 and stores it in the evaluation value storage unit 416. When the stream combination designation is changed, the reference information creation unit 418 extracts the evaluation value information from the evaluation value storage unit 416, creates the reference information, and stores the reference information in the reference information storage unit 210.

  The data storage unit 202 and the evaluation value storage unit 404 in the multiplexing apparatus 10 are functionally integrated, the data transmission unit 207 and the evaluation value transmission unit 408 in the video transmission server 20 are functionally integrated, and the data in the video playback client 30 is integrated. A configuration in which the receiving unit 212 and the evaluation value receiving unit 415 are functionally integrated is also possible.

  The third embodiment is a modification of the above-described first embodiment. As can be seen by comparing the configuration of FIG. 1 with the configuration of FIG. 22, the multiplexing apparatus 10 does not create reference information. Reference information is created by the video playback client 30.

  Therefore, the information transmitted to the video transmission client 30 is not the information shown in FIG. 8 and FIG. 21, but the correspondence information between the stream ID and the video encoded data ID shown in FIG. 4 and the sort shown in FIG. This is the result (here, the average SNR calculation result for all data IDs). The creation of the reference information in the video playback client 30 is the same as in the first embodiment described above. That is, the setting of “stream combination” in step S301 in FIG. 5 is performed by the stream combination designation unit 215 shown in FIG. 22, and the reference information according to the procedure shown in FIG. 5 according to the change in designation of “stream combination”. Create.

D. Fourth Embodiment Next, a fourth embodiment of the present invention will be described.
FIG. 23 is a block diagram showing the configuration of the video transmission system according to the fourth embodiment. In addition, the same code | symbol is attached | subjected to the part corresponding to FIG.1, FIG.12, FIG.22, and description is abbreviate | omitted. As is apparent from FIG. 23, the video transmission system according to the fourth embodiment is different from the video transmission system according to the first embodiment shown in FIG. The configuration of the third embodiment shown in FIG.

  That is, in the video transmission system according to the fourth embodiment, in the multiplexing device 10, the video dividing unit 301 divides input video data into M streams (tile division), and the evaluation value calculating unit 403 An evaluation value is calculated using the video data stored in the data storage unit 503 and stored in the calculated value storage unit 404. In the video transmission server 20, the evaluation value transmission unit 408 extracts the evaluation value information from the evaluation value storage unit 404 and sends it to the video reproduction client 30. In the video reproduction client 30, the evaluation value receiving unit 415 receives the evaluation value information transmitted from the video transmission server 20, stores the evaluation value information in the evaluation value storage unit 416, and the reference information creation unit 418 receives the stream combination. When the designation is changed, the evaluation value information is extracted from the evaluation value storage unit 416, the reference information is created, and the reference information is stored in the reference information storage unit 210.

  The fourth embodiment is a modification of the above-described first embodiment. As can be seen by comparing the configuration of FIG. 12 with the configuration of FIG. 23, the multiplexing apparatus 10 does not create reference information. Reference information is created by the video playback client 30.

  Therefore, the information transmitted to the video transmission client 30 is not the information shown in FIGS. 8 and 21, but the information shown in FIGS. 18 and 6 (here, the average SNR calculation result for all data IDs). . The creation of the reference information in the video playback client 30 is the same as in the second embodiment described above, and the “stream combination” in step S301 in FIG. 5 is set in the stream combination designation unit 215 in FIG. Reference information may be created according to the procedure shown in FIG. 5 in accordance with the change in designation of “stream combination”.

E. Fifth Embodiment Next, a fifth embodiment of the present invention will be described.
The configuration of the video transmission system according to the fifth embodiment is the same as the configuration of the video transmission system according to the second embodiment shown in FIG. In the fifth embodiment, the transmission rate is 11 Mbps, the base bit rate is 500 kbps, the improved bit rate is 1 Mbps, and 1.5 Mbps (two-stage improvement), and the evaluation value is an average SNR (average value of SNR of all frames). The reference information creation procedure in the case of having been performed will be described.

  FIG. 24 is a flowchart for explaining a reference information creation procedure in the multiplexing apparatus 10 according to the fifth embodiment. First, a “transmission rate” indicating a transmission band is set to 11 Mbps, and a “stream combination” configured by a combination of data IDs (stream ID = (7, 8, 9, 13, 14, 15, 19, 20) is set. , 21, 25, 26, 27)) (step S 701), and then “base bit rate” = 500 kbps indicating the basic encoding rate, indicating an encoding rate higher than the basic encoding rate “Improved bit rate” = 1 Mbps and 1.5 Mbps are set (step S702).

  Next, as shown in FIG. 25, the data ID is sorted in ascending order using the evaluation value (average SNR indicated by SNR_Y500) as a key (step S703), and the data ID given to the encoded data by the base bit rate = Of (107, 108, 109, 113, 114, 115, 119, 120, 121, 125, 126, 127), the data ID = 121 indicating the lowest SNR was encoded at 1 Mbps on one stage. The data ID assigned to the data is changed to 221 (step S704). After the change, data ID = (107, 108, 109, 113, 114, 115, 119, 120, 221, 125, 126, 127).

  As a result, the total bit rate is 11 × 0.5 Mbps + 1 × 1 Mbps = 6.5 Mbps <transmission rate: 11 Mbps (NO in step S705), and the process returns to S703. When the data ID = 221 is further changed after the sorting and the data ID change are performed a plurality of times, the data ID is changed to the data ID = 321 given to the data encoded at 1.5 Mbps which is one step higher. .

  If YES in step S705, it indicates that the total bit rate exceeds the transmission rate, so the previous data ID change is restored (step S706). In this case, the data ID change is performed 10 times, and the result shown in FIG. 25 can be obtained. Here, SNR_Y indicates an average SNR corresponding to data corresponding to the changed data ID. Next, reference information is created based on the data IDs indicated by hatching in FIG. 25 (step S707).

  FIG. 25 shows the data ID before change and the average SNR for the data ID associated with FIG.

  According to the first to fifth embodiments described above, N images suitable for the transmission rate from video data encoded and multiplexed at a plurality of bit rates with respect to a video having a plurality of synchronized stream configurations. Thus, even if transmission rate <N × fixed coding rate, partial video composed of N streams can be transmitted, and transmission rate ≧ N × fixed code. Therefore, it is possible to transmit an optimal high-quality video according to the transmission rate without reducing the encoding rate too much so that the encoding rate is achieved.

DESCRIPTION OF SYMBOLS 10 Multiplexer 20 Video transmission server 30 Video reproduction client 201 Encoding part 202 Data storage part 203 Reference information creation part 204 Reference information storage part 205 Data management part 206 Command reception part 207 Data transmission part 208 Reference information transmission / reception part 209 Command transmission Unit 210 reference information storage unit 211 command conversion unit 212 data reception unit 213 decoding unit 214 video display unit 215 stream combination designation unit 216 reference information reception unit

Claims (9)

A video multiplexing device that encodes and multiplexes a plurality of encoded streams and transmits in a predetermined transmission band,
Encoding means for encoding and multiplexing each of the plurality of streams at a plurality of encoding rates;
Selecting means for selecting a predetermined number of streams from a plurality of streams encoded at a plurality of code rates by the encoding means;
Evaluation value calculation means for calculating a quality evaluation value for each of a plurality of streams encoded at a plurality of encoding rates by the encoding means;
The sum of the encoding rates of the predetermined number of streams selected by the selection means is within the predetermined transmission band, and the minimum value of the quality evaluation values of the predetermined number of streams is higher. A reference information creating means for creating reference information for identifying a combination of streams;
Selection instruction means for instructing a combination of the predetermined number of streams to be selected by the selection means based on the reference information created by the reference information creation means;
And a transmission means for transmitting the predetermined number of streams finally selected by the selection means. Video division means for generating a plurality of streams by dividing the input video data into small areas;
The encoding means includes
2. The video multiplexing apparatus according to claim 1, wherein each of the plurality of streams divided by the video division unit is encoded and multiplexed at a plurality of encoding rates. A designating unit for designating a combination of the predetermined number of streams;
The selection instruction means includes
When the designation unit instructs to change the combination of the predetermined number of streams, based on the reference information created by the reference information creation unit, the combination of the predetermined number of streams to be selected by the selection unit The video multiplexing apparatus according to claim 1, wherein a new instruction is issued. A designating unit for designating a combination of the predetermined number of streams;
The reference information creating means includes
When the designation unit instructs to change the combination of the predetermined number of streams, the sum of the encoding rates of the predetermined number of streams included in the combination instructed to change is within the predetermined transmission band, and The reference information for identifying the combination of the predetermined number of streams is newly created, wherein the minimum value of the quality evaluation values of the predetermined number of streams included in the combination instructed to be changed becomes higher. Item 3. The video multiplexing device according to Item 1 or 2. A video multiplexing method for encoding and multiplexing a plurality of encoded streams and transmitting in a predetermined transmission band,
Encoding and multiplexing each of the plurality of streams at a plurality of encoding rates;
Selecting a predetermined number of streams from a plurality of streams encoded at the plurality of code rates;
Calculating a quality evaluation value for each of a plurality of streams encoded at the plurality of encoding rates;
The combination of the predetermined number of streams is specified such that the sum of the encoding rates of the predetermined number of streams is within the predetermined transmission band, and the lowest quality evaluation value of the predetermined number of streams is higher. Creating a reference information;
Instructing a combination of the predetermined number of streams to be selected based on the created reference information;
And finally transmitting the predetermined number of selected streams. The video multiplexing method comprising: Generating a plurality of streams by dividing input video data into small areas;
The video multiplexing method according to claim 5, further comprising: encoding and multiplexing each of the plurality of divided streams at a plurality of encoding rates. Designating a combination of the predetermined number of streams;
A step of newly instructing a combination of the predetermined number of streams to be selected based on the reference information when the combination of the predetermined number of streams is instructed to be changed. The video multiplexing method according to claim 5 or 6. Designating a combination of the predetermined number of streams;
When the change combination of the predetermined number of streams is instructed to be changed, the sum of the encoding rates of the predetermined number of streams included in the change instructed combination is within the predetermined transmission band, and the change is instructed. A step of newly creating reference information for identifying a combination of the predetermined number of streams, wherein a minimum quality evaluation value of the predetermined number of streams included in the combination becomes higher. The video multiplexing method according to claim 5 or 6. To a computer of a video multiplexing apparatus that encodes and multiplexes a plurality of encoded streams and transmits them in a predetermined transmission band,
An encoding function for encoding and multiplexing each of a plurality of streams at a plurality of encoding rates;
A selection function for selecting a predetermined number of streams from a plurality of streams encoded at a plurality of code rates by the encoding function;
An evaluation value calculation function for calculating a quality evaluation value for each of a plurality of streams encoded at a plurality of encoding rates by the encoding function;
The predetermined number of streams, wherein the sum of the encoding rates of the predetermined number of streams selected by the selection function is within the predetermined transmission band, and the minimum value of the quality evaluation values of the predetermined number of streams is higher. A reference information creation function for creating reference information for identifying a combination of streams;
A selection instruction function for instructing a combination of the predetermined number of streams to be selected by the selection function based on the reference information created by the reference information creation function;
A program for realizing a transmission function for transmitting the predetermined number of streams finally selected by the selection function.

Images