JP5478346B2 - Format conversion apparatus, video distribution apparatus, multiplexing apparatus, format conversion method, video distribution method, and program - Google Patents

Description

  The present invention is based on the current H.264 standard. H.264 format encoding apparatus, video distribution apparatus, multiplexing apparatus, format conversion method, video distribution method, encoding and multiplexing high-resolution video that cannot be supported by the H.264 video encoding system, and distributing and displaying it to various terminals, And the program.

  Conventionally, H.M. In the H.264 video encoding method, video of 4K × 2K (horizontal pixel number 4000 × vertical pixel number 2000) or less can only be encoded (see, for example, Non-Patent Document 1), and the video exceeds 4K × 2K. On the other hand, there was no means for encoding. Therefore, for high-resolution video exceeding 4K × 2K, a technique is used in which the high-resolution video is divided into a plurality of small-area videos and then encoded (see, for example, Non-Patent Document 2). In this non-patent document 2, the small area video is regarded as video from a plurality of cameras, and the international standard encoding method H.264 is used. Coding is performed using multi-view video coding (MVC) defined by H.264 Annex H.

  In MVC, a plurality of encoded video data is compressed and encoded in a multiplexed form. Encoded data of frames having the same time stamp are continuously multiplexed. That is, the encoded data is a stream in which a plurality of video data is multiplexed. Since frames with the same time stamp are continuously multiplexed, the decoder can decode and output MVC encoded data, thereby successively decoding and outputting a plurality of camera images with the same time stamp. it can.

  FIG. 14 is a block diagram showing a configuration of a video transmission system according to the prior art. In the figure, the conventional video transmission system includes a multiplexing device 1, a video distribution device 2, and a video reception terminal 3. In the multiplexing apparatus 1, the encoding unit 101 is, for example, an MVC encoding unit, and encodes and multiplexes M input streams at the same fixed encoding rate to create multiplexed data. The stream storage unit 102 stores multiplexed data including the M streams.

  In the video distribution device 2, the stream transmission unit 103 extracts N (≦ M) streams from the M multiplexed streams stored in the stream storage unit 102 and sends them to the video reception terminal 3. Here, in order to transmit N streams, a transmission rate corresponding to N × fixed coding rate is required.

  In the video receiving terminal 3, the data receiving unit 104 receives N streams. The decoding unit 105 decodes N streams. The video display unit 106 displays a partial video.

  Here, a video composed of M streams (whole stream) is referred to as “whole video”, a video composed of N (≦ M) streams (partial streams) is referred to as “partial video”, A video composed of streams is defined as “unit video”. That is, the whole video is composed of M unit videos, and the partial video is composed of N unit videos.

Okubo et al .: "Revised third edition H.264 / AVC textbook", Impress R & D Hideaki Kimata, Shinya Shimizu, Yutaka Kunita, Megumi Isogai, and Yoshimitsu Ohtani, “Panorama video coding for user-driven interactive video application,” IEEE International Symposium on Consumer Electronics (ISCE) 2009, 2009.

  In the above-described conventional technique using MVC, in order to deliver high-resolution video exceeding HD (High Definition) size to various terminals within a limited transmission band such as a real network. There were the following problems.

In particular,
(1) When transmission rate <N × fixed coding rate, partial video composed of N partial streams cannot be video-distributed below the transmission band.
(2) Video can only be played back by a terminal equipped with an MVC decoder.
(3) When the video viewing area of the video receiving terminal is expanded, the processing load on the video receiving terminal increases. Generally, when the user enlarges the video gaze area (for example, in FIG. 4 described later, enlargement from the gaze area 1 to the gaze area 2), the number of unit videos that must be decoded increases. Processing load increases.
(4) The processing load on the distribution device increases with an increase in the number of terminals accessing the distribution device.
There is a problem.

  The present invention has been made in view of such circumstances, and its purpose is to reduce the amount of re-encoding processing computation, to reduce the processing load on the video distribution apparatus, and to improve distribution performance. A format conversion device, a video distribution device, a multiplexing device, a format conversion method, a video distribution method, and a program are provided.

  In order to solve the above-described problem, the present invention is a format conversion apparatus for converting the format of a compression-encoded video stream, and analyzes encoding information of M synchronized streams managed by identification information. A stream analysis unit, a stream cut-out unit that cuts out N (≦ M) streams constituting a rectangular video to be re-encoded from the M synchronized streams based on the identification information, and the stream analysis unit And a re-encoding unit that re-encodes the N streams extracted by the stream cut-out unit into L (1 ≦ L ≦ N−1). This is a characteristic format conversion device.

  In order to solve the above-described problems, the present invention generates a video by dividing a high-resolution video, adding identification information to each divided video, and encoding and multiplexing each divided video at a plurality of bit rates. A format converter for converting the format of the received video stream, a stream analyzer for analyzing encoding information of M synchronized streams managed by the identification information, and the M number of streams based on the identification information A stream cutout unit that cuts out N (≦ M) streams constituting a rectangular video to be recoded from the synchronized stream; and the stream cutout unit that reuses the encoded information analyzed by the stream analysis unit. And a re-encoding unit that re-encodes the N streams cut out by L into L (1 ≦ L ≦ N−1). That.

  In order to solve the above-described problem, the present invention is an apparatus for converting the format of a compression-encoded video stream, the decoding unit decoding M synchronized streams managed by identification information, A video cutout unit that cuts out the video decoded by the decoding unit based on the identification information, and a video clipped by the video cutout unit by reusing the encoded information obtained at the time of decoding by the decoding unit And a re-encoding unit that re-encodes the signal into L (1 ≦ L ≦ M−1).

  In order to solve the above-described problems, the present invention generates a video by dividing a high-resolution video, adding identification information to each divided video, and encoding and multiplexing each divided video at a plurality of bit rates. A device for converting the format of the received video stream, a decoding unit for decoding M synchronized streams managed by the identification information, and a video decoded by the decoding unit based on the identification information The video cutout unit to be cut out and the encoding information obtained at the time of decoding by the decoding unit are reused, and the video cut out by the video cutout unit is re-encoded to L (1 ≦ L ≦ M−1) And a re-encoding unit for performing the format conversion.

  In order to solve the above-described problem, the present invention provides a video distribution device that distributes a compression-encoded video stream to a video reception terminal, the reception unit receiving a command from the video reception terminal, and the reception A stream analysis unit that analyzes encoding information of a stream indicated by a command received by the reception unit, a stream cutout unit that extracts a stream specified by the command received by the reception unit, and a stream analysis unit A re-encoding unit for re-encoding the stream cut out by the stream cut-out unit, and transmitting the video re-encoded by the re-encoding unit to the video receiving terminal A video distribution device comprising: a video distribution device.

  In the present invention according to the above invention, the stream is generated by dividing a high-resolution video, adding identification information to each divided video, and encoding and multiplexing each divided video at a plurality of bit rates. When the re-encoding unit re-encodes below the transmission band of the line to which the video receiving terminal is connected, the re-encoding unit performs pre-encoding on each divided video on condition that the transmission band is below Of the plurality of bit rates, the encoding information in the case of encoding at the maximum bit rate is reused, and the stream cut out by the stream cutout unit is re-encoded.

  In order to solve the above-described problem, the present invention provides a video distribution device that distributes a compression-encoded video stream to a video reception terminal, the reception unit receiving a command from the video reception terminal, and the reception A decoding unit that decodes the stream indicated by the command received by the unit, a video cutout unit that cuts out the video indicated by the command received by the receiving unit from the video decoded by the decoding unit, A re-encoding unit that re-encodes the video cut out by the video cut-out unit, reusing the encoded information obtained at the time of decoding by the decoding unit, and the video re-encoded by the re-encoding unit A video transmission device comprising: a transmission unit that transmits the video to the video reception terminal.

  In the present invention according to the invention described above, the stream is generated by dividing a high-resolution video, adding identification information to each divided video, and encoding and multiplexing the divided video at a plurality of bit rates. When the re-encoding unit re-encodes below the transmission band of the line to which the video receiving terminal is connected, the re-encoding unit is pre-encoded with each divided video on condition that the transmission band is below Among the plurality of bit rates, the encoding information obtained when encoding is performed at the maximum bit rate is reused, and the video clipped by the video cutout unit is recoded.

  In order to solve the above-described problem, the present invention is a multiplexing apparatus that streams and stores high-resolution video, and includes a video dividing unit that divides the video, and a division that is divided by the video dividing unit. An encoding unit that assigns identification information to each video, encodes and multiplexes the divided video at a plurality of bit rates to generate a stream, and a stream storage unit that stores a stream generated by the encoding unit It is the multiplexing apparatus characterized by providing.

  The present invention is characterized in that, in the above-mentioned invention, the encoding unit encodes the tiles divided into tiles at different encoding rates.

  The present invention is characterized in that, in the above invention, the video dividing unit converts the resolution of the input video into a plurality of resolutions before the tile division of the video.

  In order to solve the above-described problem, the present invention is a format conversion method for converting the format of a compression-encoded video stream, and includes encoding information of M synchronized streams managed by identification information. Analyzing, based on the identification information, cutting out N (≦ M) streams constituting the rectangular video to be re-encoded from the M synchronized streams, and re-encoding the encoded information. And re-encoding the extracted N streams. A format conversion method comprising:

  In order to solve the above-described problem, the present invention is a format conversion method for converting a compression-encoded video stream, and decoding M synchronized streams managed by identification information; and Cutting out the decoded video based on the identification information, and re-encoding the cut-out video by reusing the encoded information obtained at the time of decoding. This is the format conversion method.

  In order to solve the above-described problem, the present invention is a video distribution method for distributing a video stream created by a multiplexing device as a video stream compressed and encoded from a video distribution device to a video receiving terminal. As a preliminary preparation for video distribution by the multiplexing device, a step of dividing an input video into tiles, a step of encoding and multiplexing the divided video by the multiplexing device and creating a stream, and the multiplexing When the apparatus stores the created stream as an accumulation stream, and the video receiving terminal receives a gaze area designation from the user, the command associated with the gaze area designation is transmitted to the video distribution apparatus. And a stored stream corresponding to the command is received from the multiplexing device by the video distribution device. A step of analyzing encoded information of the extracted accumulated stream by the video distribution device, and a stream corresponding to a command from the video receiving terminal is cut out from the accumulated stream by the video distribution device. Re-encoding encoded information obtained by the analysis by the video distribution device, re-encoding the clipped stream, and re-encoding the stream re-encoded by the video distribution device. A video that is distributed to the video receiving terminal; and a stream that is distributed from the video distributing device by the video receiving terminal as a video of the gaze area designated by the gaze area designation. Distribution method.

  In order to solve the above-described problem, the present invention is a video distribution method for distributing a video stream created by a multiplexing device as a video stream compressed and encoded from a video distribution device to a video receiving terminal. As a preliminary preparation for video distribution by the multiplexing device, a step of dividing an input video into tiles, a step of encoding and multiplexing the divided video by the multiplexing device and creating a stream, and the multiplexing When the apparatus stores the created stream as an accumulation stream, and the video receiving terminal receives a gaze area designation from the user, the command associated with the gaze area designation is transmitted to the video distribution apparatus. And a stored stream corresponding to the command is received from the multiplexing device by the video distribution device. A step of decoding the extracted accumulated stream by the video distribution device, a step of cutting out a video indicated by the command from the decoded video by the video distribution device, and the video distribution device Re-encoding the encoded information obtained at the time of decoding, re-encoding the clipped video, and distributing the re-encoded stream to the video receiving terminal by the video distribution device And a step of displaying a stream distributed from the video distribution device by the video receiving terminal as a video of a gaze area designated by the gaze area designation.

  The present invention is characterized in that, in the above-mentioned invention, the step of generating a stream by encoding and multiplexing encodes each tile divided into tiles at different encoding rates.

  The present invention is characterized in that, in the above-mentioned invention, before the step of dividing the input video into tiles, the method further includes a step of converting the resolution of the input video into videos of a plurality of resolutions.

  In order to solve the above-described problem, the present invention provides a reception function for receiving a command from the video reception terminal to a computer of a video distribution apparatus that distributes a compressed and encoded video stream to the video reception terminal, A stream analysis function for analyzing encoded information of a stream indicated by a command received by the reception function, a stream cut-out function for cutting out a stream indicated by the command received by the reception function, and an analysis by the stream analysis function A re-encoding function for re-encoding the encoded information, re-encoding the stream cut out by the stream cut-out function, and a transmission function for transmitting the video re-encoded by the re-encoding function to the video receiving terminal A program characterized by being executed.

  In order to solve the above-described problem, the present invention provides a reception function for receiving a command from the video reception terminal to a computer of a video distribution apparatus that distributes a compressed and encoded video stream to the video reception terminal, A decoding function for decoding the stream indicated by the command received by the reception function, a video cutout function for cutting out the video indicated by the command received by the reception function from the video decoded by the decoding function, and the decoding The re-encoding function for re-encoding the video cut out by the video cut-out function, re-encoding the video re-encoded by the re-encoding function, reusing the encoded information obtained at the time of decoding by the encoding function A program for executing a transmission function for transmission to a video receiving terminal.

  According to the present invention, it is possible to reduce the amount of re-encoding processing computation, reduce the processing load on the video distribution apparatus, and improve the distribution performance.

It is a block diagram which shows the structure of the video delivery system containing the video delivery apparatus by this 1st Embodiment. It is a conceptual diagram which shows the example of a tile division | segmentation of the video data by the video division | segmentation part 200, and the provision example of tile division | segmentation information (tile ID). FIG. 3 is a conceptual diagram showing an example of data ID assignment when each tile divided into tiles shown in FIG. 2 is encoded at 500 Kbps, 1 Mbps, and 1.5 Mbps. It is a conceptual diagram which shows the correspondence table of tile ID which shows the spatial position of an input image | video, and data ID. It is a block diagram which shows the structure of the format conversion apparatus 208 by this 1st Embodiment. It is a flowchart for demonstrating operation | movement of the format conversion apparatus 208 by this 1st Embodiment. It is a sequence diagram for demonstrating a series of operation | movement of the video delivery system of this 1st Embodiment. It is a conceptual diagram for demonstrating the tile division | segmentation of input video data, and the example of provision of tile ID. It is a conceptual diagram for demonstrating the determination method of tile ID in the video receiving terminal 30 by this 1st Embodiment. It is a block diagram which shows the structure of the video delivery system containing the video delivery apparatus by this 2nd Embodiment. It is a block diagram which shows the structure of the format converter 308 by this 2nd Embodiment. It is a flowchart for demonstrating operation | movement of the format conversion apparatus 308 by this 2nd Embodiment. It is a sequence diagram for demonstrating a series of operation | movement of the video delivery system of this 2nd Embodiment. It is a block diagram which shows the structure of the video transmission system by a prior art.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

A. First Embodiment First, a first embodiment of the present invention will be described. In the first embodiment, a stream (high-resolution video is divided, identification information is assigned to each divided video, and each divided video is encoded and multiplexed at a plurality of bit rates. And a format conversion device 208 that cuts out and re-encodes the stream without decoding the video stream generated by the video distribution device.

  FIG. 1 is a block diagram showing a configuration of a video distribution system including a video distribution apparatus according to the first embodiment. In FIG. 1, the video distribution system according to the first embodiment includes a multiplexing device 10, a video distribution device 20 including a format conversion device 208, and a video reception terminal 30. The multiplexing apparatus 10 includes a video dividing unit 200, an encoding unit 201, and a stream storage unit 202. The video dividing unit 200 tiles a video (M synchronized streams) (see, for example, FIG. 2 described later), and creates a tile video composed of video data in a small area. The encoding unit 201 encodes and multiplexes these tile videos with encoding parameters (for example, a bit rate, a QP value, a video quality value, etc.) that specify a plurality of encoding rates, and creates video data.

  The stream accumulation unit 202 is encoded and multiplexed for each video in the tile managed by tile division information (for example, see FIG. 2 described later) and tile ID (for example, see FIG. 2 described later). Data ID (for example, see FIG. 3) is assigned to video data and stored. In addition, the stream storage unit 202 stores a correspondence table (for example, see FIG. 4) between tile IDs and data IDs.

  Note that the encoding unit 201 can generate a plurality of synchronization streams managed by ID, For example, multi-view video coding (MVC) defined in H.264 Annex H can be used.

  The video distribution device 20 includes a format conversion device 208 including a stream analysis unit 203, a stream cutout unit 204, and a re-encoding unit 205, a stream transmission unit 206, and a command reception unit. The stream analysis unit 203 extracts a stream from the stream storage unit 202 in the multiplexing apparatus 10 and performs stream analysis, and encodes information (reference picture number, motion vector information, variable block size motion compensation) obtained from the stream analysis. At least one piece of information is extracted from block division pattern information, intra prediction mode information, and the like.

  The command receiving unit 207 receives a command including a data ID or gaze area information from the video receiving terminal 30. Based on the command, the stream cutout unit 204 cuts out a stream (N (≦ M) streams) related to the data ID or gaze area information. The re-encoding unit 205 re-encodes the extracted N streams to L (1 ≦ L ≦ N−1) by reusing at least one of the encoded information. The stream transmission unit 206 distributes the re-encoded video to the video receiving terminal 30. The command may include information on a transmission band to which the video receiving terminal 30 is connected and information such as specifications of the video receiving terminal 30.

  FIG. 2 is a conceptual diagram illustrating an example of tile division of video data by the video division unit 200 and an example of giving tile division information (tile ID). For example, when the input video size is 5120 × 1280 pixels and the tile size is 640 × 320 pixels, the input video is divided into tiles as shown in FIG. A tile ID from 0 to 31 is assigned to each tile. 1, 2, 3, 9, 10, 11, 17, 18, and 19 of the tile are the gaze region 1, and 0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, It is assumed that 13, 14, 16, 17, 18, 19, 20, 21, 22, 24, 25, 26, 27, 28, 29, and 30 are the gaze region 2.

  FIG. 3 is a conceptual diagram showing an example of data ID assignment when each tile divided into tiles shown in FIG. 2 is encoded at 500 Kbps, 1 Mbps, and 1.5 Mbps. For example, when each tile shown in FIG. 2 is encoded with three types of bit rates of 500 Kbps, 1 Mbps, and 1.5 Mbps, data IDs of 100 to 131, 200 to 231 and 300 to 331 are sequentially shown in FIG. Is granted.

  FIG. 4 is a conceptual diagram showing a correspondence table between tile IDs and data IDs indicating spatial positions of the input video. The correspondence table associates the tile ID assigned to the high resolution tile shown in FIG. 2 with the data ID at the bit rate shown in FIG. That is, tile IDs 0, 1,..., 31 are associated with data IDs 100, 101,..., 131 at a bit rate of 500 Kbps, and 200, 201,. , 231 are associated with each other, and data IDs 300, 301,..., 331 with a bit rate of 1.5 Mbps are associated with each other.

  Thus, encoding in advance at a plurality of bit rates can improve the encoding efficiency while reducing the processing load on the format conversion device 208 in the video distribution device 20. The stream storage unit 202 stores multiplexed data obtained by multiplexing encoded data managed by these IDs, and also stores a correspondence table between tile IDs and data IDs as shown in FIG.

  FIG. 5 is a block diagram showing the configuration of the format conversion device 208 according to the first embodiment. The format conversion device 208 reuses the reuse of the encoded information, the stream cutout unit 204 that extracts the stream related to the input ID from the stream storage unit 202, the stream analysis unit 203 for extracting the encoded information, and the like. And a re-encoding unit 205 that performs encoding.

  FIG. 6 is a flowchart for explaining the operation of the format conversion apparatus 208 according to the first embodiment. When there is an external ID designation (step Sa1), the format conversion device 208 cuts out a stream related to the ID (step Sa2) and analyzes the stream (step Sa3). The encoded information extracted from the analysis is reused, and the N streams are re-encoded to L (1 ≦ L ≦ N−1) (step Sa4), and a re-encoded stream is created.

  FIG. 7 is a sequence diagram for explaining a series of operations of the video distribution system according to the first embodiment. First, in the multiplexing apparatus 10, as preparation for video distribution, the input video is divided into tiles (step S11), encoded and multiplexed to create a stream (step 12), and stream accumulation is performed (step 13).

  Here, when a gaze area is designated from the video receiving terminal 30 (step S14), a command associated therewith is transmitted to the video distribution apparatus 20 (step S15), and a stream request is sent from the video distribution apparatus 20 to the multiplexing apparatus 10. It is transmitted (step S16). From the multiplexing apparatus 10, an accumulation stream corresponding to the command is transmitted (step S17).

  Next, the video distribution device 20 analyzes the stored stream (step S18), cuts out the stream corresponding to the command from the video receiving terminal 30 from the stored stream (step S19), and the encoding obtained as a result of the analysis By reusing the information, re-encoding is performed with a reduced calculation load (step S20). The re-encoded stream obtained as a result is distributed to the video receiving terminal 30 (step S21), and the gaze area is displayed on the video receiving terminal 30 (step S22).

  Here, in the above-described operation, a case will be described in which resolution conversion is performed before the input video is divided into tiles in step S11. FIG. 8 is a conceptual diagram for explaining an example of tile division of input video data and assignment of tile IDs. In the above-described operation, before the input video is divided into tiles in step S11, as shown in FIG. 8, after the input video is converted into a plurality of resolutions, the input video is divided into tiles, tile IDs are assigned, and encoding / multiplexing is performed. Create a stream and store the stream.

  In the example illustrated in FIG. 8, after conversion to high resolution, medium resolution, and low resolution, tile division is performed for each resolution of video data. In the example of FIG. 8, the high-resolution video data is divided into 32 tiles each having a tile size of 640 × 320 pixels, and the medium-resolution video data has a tile size of 640 × 320 pixels. Divided into 8 tiles, low-resolution video data is divided into 2 tiles each having a tile size of 640 × 320 pixels. The number given to each tile is tile division information (tile ID).

  Subsequently, in step S14, a distribution request for the re-encoded stream is made to the video distribution apparatus 20 using the tile ID associated with the resolution applied to the change in the gaze area designation, and the re-encoded stream is received. The processing load on the video receiving terminal 30 can also be reduced. For example, in the video receiving terminal 30, when the gaze area designation is changed from the gaze area 1 to the gaze area 2 shown in FIG. 4 described above, the number of decoding target tiles increases and the load on the video receiving terminal 30 increases. Therefore, it is possible to distribute a low-resolution video by converting the input video to a plurality of resolutions.

  Here, FIGS. 9A and 9B are conceptual diagrams for explaining a tile ID determination method in the video reception terminal 30 according to the first embodiment. For example, as shown in FIG. 9A, when the coordinates of the upper left vertex of the gaze area specified by the video receiving terminal 30 are (x, y) = (960, 100), the four vertices of the gaze area are included. The tile IDs are 1, 4, 25, and 28. The tile IDs included in the quadrilateral area having the four vertices are 1, 2, 3, 4, 9, 10, 11, 12, 17, 18, 19 , 20, 25, 26, 27, 28.

  From the video receiving terminal 30, a video distribution request for tile ID groups: 1, 2, 3, 4, 9, 10, 11, 12, 17, 18, 19, 20, 25, 26, 27, 28, and a gaze area Information (for example, the gaze area size and the coordinates of the upper left vertex) is transmitted to the video distribution device 20 as a command. The command is notified to the stream cutout unit 204, the stream analysis unit 203, and the stream storage unit 202, and the stream is taken out and cut out. Next, when the stream composed of the tile ID group including the gaze area is re-encoded, the encoding information (reference picture number, motion vector information, variable block size motion compensation) extracted by the stream analysis unit 203 is re-encoded. Reuse block division pattern information and in-screen prediction mode information).

  Hereinafter, an example in which motion vector information is reused as encoded information will be described. In general, in a general video encoding method such as MPEG4, a video is divided into grid-like blocks, a motion vector is obtained for each block (hereinafter referred to as a macroblock), and motion compensation is performed for each macroblock. The resulting motion compensation error is orthogonally transformed and quantized.

  Encoding information (reference picture number, motion vector information, block division pattern information at variable block size motion compensation, intra prediction mode information, etc.) obtained at the time of encoding is assigned to each macroblock.

  As shown in FIG. 9B, when the re-encoding target area is configured from each tile, the re-encoding target macro block matches the macro block at the time of input video encoding. Can be reused as is. However, if the bit rate at the time of encoding is different in the motion vector search, the image quality of the reference image is different, so that the motion vector result is different. Therefore, when the re-encoding target area (total 16 tiles) in FIG. 9B is re-encoded at 16 Mbps, the motion vector information at the time of encoding at each tile 1 Mbps may be reused. Thereby, the calculation load of motion vector search can be reduced.

B. Second Embodiment Next, a second embodiment of the present invention will be described. Here, only the parts different from the first embodiment will be described. In the second embodiment, a stream (high-resolution video is divided, identification information is assigned to each divided video, and each divided video is encoded / multiplexed at a plurality of bit rates. And a format conversion device 308 that extracts and re-encodes the video after the video distribution device 20 decodes the video stream and converts it back to video.

  FIG. 10 is a block diagram showing a configuration of a video distribution system including a video distribution apparatus according to the second embodiment. In FIG. 10, the video distribution system according to the second embodiment includes a multiplexing device 10, a video distribution device 20 including a format conversion device 308, and a video reception terminal 30. The format converter 308 is different from the first embodiment. In the second embodiment, the stream analysis unit 203 is a decoding unit 303, and the stream cutout unit 204 is replaced with a video cutout unit 304. That is, in the second embodiment, re-encoding is performed after returning the accumulated stream to video once. Further, regarding the reuse of motion vector information, the reuse method differs depending on the cutout method in the video cutout unit 304.

  When the video cutout unit 304 cuts out the video in the recoding target area shown in FIG. 9B, the motion vectors of all the macroblocks in the recoding target area can be reused. On the other hand, when a video is cut out in the gaze area of FIG. 9B, since the input video is encoded in units of tiles, tiles located on each side of the gaze area (tile ID = 1, 2, 3, 4, 9, 12, 17, 20, 25, 26, 27, 28), the motion vector of the macro block may refer to an area outside the gaze area and cannot be reused. Therefore, only the motion vectors of tiles (tile ID = 10, 11, 18, 19) existing in the gaze area are reused.

  FIG. 11 is a block diagram showing the configuration of the format conversion device 308 according to the second embodiment. The format conversion device 308 extracts a stream related to the input ID from the stream storage unit 202 and decodes it, a video cutout unit 304 that cuts out the decoded video, and obtains the cut out video at the time of decoding. The re-encoding unit 205 is configured to re-encode the encoded information into L (1 ≦ L ≦ M−1).

  FIG. 12 is a flowchart for explaining the operation of the format conversion apparatus 308 according to the second embodiment. When there is an ID designation from the outside (step Sb1), the format conversion apparatus 308 extracts a stream related to the ID (step Sb2) and performs decoding (step Sb3). A video is cut out from the decoded video (step Sb4), the encoded information obtained at the time of decoding is reused, the cut video is re-encoded (step Sb5), and the re-encoded stream is re-encoded. Create.

  FIG. 13 is a sequence diagram for explaining a series of operations of the video distribution system according to the second embodiment. First, in the multiplexing apparatus 10, as an advance preparation for video distribution, the input video is divided into tiles (step S31), encoded and multiplexed to create a stream (step S32), and stream accumulation is performed (step S33).

  Here, when a gaze area is designated from the video receiving terminal 30 (step S34), a command associated therewith is transmitted to the video distribution apparatus 20 (step S35), and a stream request is sent from the video distribution apparatus 20 to the multiplexing apparatus 10. It is transmitted (step S36). The multiplexed stream corresponding to the command is transmitted from the multiplexer 10 (step S37).

  Next, the video distribution device 20 decodes the accumulated stream (step S38), and the video related to the command from the video receiving terminal 30 is cut out from the decoded video (step S39) and obtained at the time of decoding. By reusing the encoded information, re-encoding is performed with a reduced calculation load (step S40). As a result, the obtained re-encoded stream is distributed to the video receiving terminal 30 (step S41), and the gaze area is displayed on the video receiving terminal 30 (step S42).

  1 and the format conversion device 308 shown in FIG. 10, tile ID = 0, 1, 2, 3, 8, 9, 10, 11, 16, 17, 18, in FIG. The unit video of each tile of 19, 24, 25, 26, and 27 is a single video stream tile, and the unit video of each tile of ID = 40, 41, 44, 45, and 80 is a single video stream. It may be re-encoded into a plurality of video streams.

According to the first and second embodiments described above, the following effects can be obtained.
1) The amount of computation of re-encoding processing can be reduced, and high-resolution video can be distributed at a transmission rate or lower.
2) A plurality of synchronous streams managed by ID (for example, a stream after MVC encoding) can be converted into another encoding format and video can be distributed.
3) Even when accessed simultaneously from various video receiving terminals 30, it is possible to distribute a video including a video watching area requested from the video receiving terminal.
4) When re-encoding a plurality of synchronization streams (for example, streams after MVC encoding) managed by the ID, encoding parameters included in the synchronization stream can be used.

DESCRIPTION OF SYMBOLS 10 Multiplexer 20 Video distribution apparatus 30 Video receiving terminal 200 Video dividing part 201 Encoding part 202 Stream storage part 203 Stream analysis part 204 Stream cut-out part 205 Re-encoding part 206 Stream transmission part 207 Command receiving part 208,308 Format conversion Device 303 Decoding unit 304 Video clipping unit

Claims (17)

A format converter that divides a high-resolution video, assigns identification information to each divided video, and converts the format of the video stream generated by encoding and multiplexing each divided video at a plurality of bit rates. And
By analyzing the coding information of the synchronized streams of M present managed by the identification information, the reference picture numbers, the sign of the motion vector information, variable block-size block division pattern information at the time of motion compensation, intra prediction mode information A stream analysis unit that extracts at least one of the conversion information ;
A stream cutout unit that cuts out N (≦ M) streams constituting a rectangular video to be re-encoded from the M synchronized streams based on the identification information;
The reuse the encoded information analyzed by the stream analyzing unit, the re-encoding unit for re-encoding the N streams cut out in L present (1 ≦ L ≦ N-1 ) by the stream extraction unit A format conversion apparatus comprising: and. A format converter that divides a high-resolution video, assigns identification information to each divided video, and converts the format of the video stream generated by encoding and multiplexing each divided video at a plurality of bit rates. And
A decoding unit that obtains motion vector information by decoding M synchronized streams managed by the identification information;
A video cutout unit that cuts out the video decoded by the decoding unit based on the identification information;
A re-encoding unit that reuses the motion vector information obtained at the time of decoding by the decoding unit and re-encodes the video clipped by the video cut-out unit into L (1 ≦ L ≦ M−1) A format conversion apparatus comprising: and. Video distribution that divides high-resolution video to video receiving terminals, assigns identification information to each divided video, and distributes the video stream generated by encoding and multiplexing each divided video at multiple bit rates A device,
A receiving unit for receiving a command from the video receiving terminal;
By analyzing the coding information of the stream indicated by the command received by the receiving unit , the reference picture number, the motion vector information, the block division pattern information at the time of variable block size motion compensation, the code of the intra prediction mode information A stream analysis unit that extracts at least one of the conversion information ;
A stream cutout unit that cuts out a stream indicated by the command received by the reception unit;
Said reuse the encoded information analyzed by the stream analyzing unit, re-encoding unit for re-encoding the stream extracted by the stream cutout portion,
A video distribution apparatus comprising: a transmission unit that transmits the video re-encoded by the re-encoding unit to the video reception terminal. The re-encoding unit includes:
When re-encoding below the transmission band of the line to which the video receiving terminal is connected, the maximum bit among a plurality of bit rates pre-encoded in each divided video on condition that the transmission band is below The video distribution apparatus according to claim 3 , wherein the encoded information when encoded at a rate is reused, and the stream extracted by the stream extraction unit is re-encoded. Video distribution device that divides a high-resolution video to a video receiving terminal, assigns identification information to each divided video, and distributes the video stream generated by encoding and multiplexing the divided video at a plurality of bit rates Because
A receiving unit for receiving a command from the video receiving terminal;
Decoding a stream indicated by the command received by the receiving unit to obtain motion vector information ;
A video cutout unit that cuts out a video indicated by a command received by the reception unit from the video decoded by the decoding unit;
A re-encoding unit that reuses the motion vector information obtained at the time of decoding by the decoding unit and re-encodes the video clipped by the video cut-out unit;
A video distribution apparatus comprising: a transmission unit that transmits the video re-encoded by the re-encoding unit to the video reception terminal. The re-encoding unit includes:
When re-encoding below the transmission band of the line to which the video receiving terminal is connected, the maximum bit among a plurality of bit rates pre-encoded in each divided video on condition that the transmission band is below The video distribution apparatus according to claim 5 , wherein the video information extracted at the rate is reused, and the video clipped by the video cutout unit is recoded. A multiplexing device for streaming and storing high-resolution video,
A video dividing unit that tiles the high-resolution video;
An encoding unit that assigns identification information to each divided video that has been tile- divided by the video dividing unit, encodes and multiplexes the divided video at a plurality of bit rates, and generates a stream;
A multiplexing apparatus comprising: a stream storage unit that stores the stream generated by the encoding unit. The encoding unit includes:
The multiplexing apparatus according to claim 7 , wherein each divided video divided into tiles is encoded at a different encoding rate. The video dividing unit
Before tiling of the image, the multiplexing apparatus according to claim 7 or 8, characterized in that the resolution conversion of the input image to the image of the plurality of resolutions. A format converter that divides a high-resolution video, assigns identification information to each divided video, and converts the format of the video stream generated by encoding and multiplexing each divided video at a plurality of bit rates. A format conversion method,
By analyzing the coding information of the synchronized streams of M present managed by the identification information, the reference picture numbers, the sign of the motion vector information, variable block-size block division pattern information at the time of motion compensation, intra prediction mode information A stream analysis step for extracting at least one of the conversion information ;
A stream cutout step of cutting out N (≦ M) streams constituting the rectangular video to be re-encoded from the M synchronized streams based on the identification information;
Re-encoding step of re-encoding the N streams extracted by the stream cutout step to L (1 ≦ L ≦ N−1) by reusing the encoded information analyzed by the stream analysis step A format conversion method characterized by including and. A format converter that divides a high-resolution video, assigns identification information to each divided video, and converts the format of the video stream generated by encoding and multiplexing each divided video at a plurality of bit rates. A format conversion method,
Decodes the synchronized stream of the M which has been managed by the identification information, and decoding step of obtaining the motion vector information,
A video cutout step of cutting out the video decoded by the decoding step based on the identification information;
Re-encoding step of re-encoding the video extracted by the video clipping step into L (1 ≦ L ≦ M−1) by reusing the motion vector information obtained at the time of decoding by the decoding step A format conversion method characterized by including and. A video distribution device generates a video stream generated by dividing a high-resolution video in a multiplexing device , assigning identification information to each divided video, and encoding and multiplexing each divided video at a plurality of bit rates. a video distribution method to deliver the image receiving terminal from,
The multiplexer is, as advance preparation for video distribution, the steps of the tile dividing the input image of high resolution,
The multiplexing device includes the steps of generating a video stream the tiled image by encoding and multiplexing,
The multiplexing device includes the steps of the video stream the generated that accumulate,
The video receiving terminal, when there is a gaze area designation from the user, transmitting a command associated with the gaze area designation to the video distribution device;
Retrieving the video distribution device, a video stream corresponding to the command from the multiplexer,
The video distribution device, by analyzing the coding information of the retrieved video stream, the reference picture number, motion vector information, variable block-size block division pattern information at the time of motion compensation, the encoding of the intra-frame prediction mode information Extracting at least one piece of information ;
The video distribution device, a stream corresponding to the command from the video receiving terminal, a step of cutting out from said video stream,
A step wherein the video distribution device, which reuses the encoded information obtained by the analysis, re-encodes the cutout stream,
The video distribution device, and the step of distributing said re-encoded stream to the image receiving terminal,
The video receiving terminal, the video distribution method characterized by including the step of displaying the stream delivered from the video distribution device, as an image of the watch area specified by the watch area specified. A video distribution device generates a video stream generated by dividing a high-resolution video in a multiplexing device , assigning identification information to each divided video, and encoding and multiplexing each divided video at a plurality of bit rates. a video distribution method to deliver the image receiving terminal from,
The multiplexer is, as advance preparation for video distribution, the steps of the tile dividing the input image of high resolution,
The multiplexing device includes the steps of generating a video stream the tiled image by encoding and multiplexing,
The multiplexing device includes the steps of the video stream the generated that accumulate,
The video receiving terminal, when there is a gaze area designation from the user, transmitting a command associated with the gaze area designation to the video distribution device;
Retrieving the video distribution device, a video stream corresponding to the command from the multiplexer,
The video distribution device, and decoding the retrieved video stream, obtaining a motion vector information,
A step of cutting out an image in which the video distribution device is instructed by the command from the decoded video,
A step wherein the video distribution device, which reuses the motion vector information obtained during the decoding, re-encoding the extracted video,
The video distribution device, and the step of distributing said re-encoded stream to the image receiving terminal,
The video receiving terminal, the video distribution method characterized by including the step of displaying the stream delivered from the video distribution device, as an image of the watch area specified by the watch area specified. The step of creating a video stream by encoding and multiplexing includes:
The video distribution method according to claim 12 or 13 , wherein each of the tiled divided videos is encoded at a different encoding rate. Prior to the step of the tile dividing the input image, video distribution method according to claim 12 or 13, further comprising the step of converting the resolution of the input image to the image of the plurality of resolutions. The program for functioning a computer as a video delivery apparatus of Claim 3 or 4 . A program for causing a computer to function as the video distribution apparatus according to claim 5 or 6 .

Images