JP5654414B2 - Video encoding apparatus, video encoding method, and program - Google Patents

Description

  The present invention relates to a video encoding device, a video encoding method, and a program for encoding video.

In recent years, research and development of super-high-definition video systems that far exceed high-definition such as super high-definition systems and 4k digital cinema have been actively conducted. There are two forms of viewing: a form that is displayed as it is on a high-definition display and a form that is cut out and displayed.
With regard to the form to cut out and display a part, system development is being promoted so that a part of the ultra-high-definition panoramic video can be viewed with high image quality using the currently popular high-definition television and network. ing.
By using this system, for example, on a high-definition television in an ordinary home, a user wants a person or a part of the venue to be enlarged and displayed from an image taken in a large venue such as a soccer stadium or a concert hall. be able to. As a result, it is possible to deliver a sense of realism in the S seats of soccer stadiums and concert halls to the home.

As a general delivery method that can be used in this system, a method of dividing an ultra-high-definition video into small divided images for each frame and transmitting all the divided images, for example, a division including a display target area specified by the user There is a method of transmitting only a part.
As a method for transmitting a divided image including the display target region, for example, a technique described in Non-Patent Document 1 has been proposed.
However, in such a method, encoding is performed at a uniform bit rate regardless of the subject included in the display target region, so that encoding is performed for a divided image including a moving object and a divided image including only a still region such as a background. There was a problem that the image quality later varied greatly. In order to cope with this problem, a data format created by encoding / multiplexing each divided image at a different bit rate according to the contents of each divided image and a portion that maximizes the subjective image quality of the display target area of the user A distribution method has been proposed (see Non-Patent Document 2).

Hideaki Kimata, Shinya Shimizu, Yutaka Kunita, Megumi Isogai, and Yoshimitsu Ohtani, "Panorama video coding for user-driven interactive video application," IEEE International Symposium on Consumer Electronics (ISCE) 2009, 2009. Masayuki Inoue, Hideaki Kizen, Katsuhiko Fukasawa, Nobuhiko Matsuura, “Study of Distribution Method in Interactive Panorama Video Distribution System”, Research Report-Audio Visual Complex Information Processing (AVM), Vol.2010-AVM-69 No.9, 2010.

However, in the technique proposed by Non-Patent Document 2, it is necessary to encode a plurality of divided images by a combination of a plurality of resolutions and a plurality of encoding amounts on the distribution side. That is, if the number of resolution patterns is n, the number of divided images in a frame image of resolution n is α _n , and the number of patterns of coding amount (coding rate) is β _n , (α _n × β _n ) divisions. The image needs to be encoded. This means that the number of divided images to be compression-encoded is greatly increased. Therefore, since a considerable amount of time is required for compression encoding, an increase in the number of divided images causes a significant increase in the length of compression encoding.

  The present invention has been made in view of such circumstances, and the purpose thereof is a video encoding device capable of reducing the processing time when creating encoded data based on a divided image obtained by dividing an original image, It is to provide a video encoding method and program.

The present invention has been made to solve the above-described problems, and a video encoding device according to an aspect of the present invention converts a resolution of an input image according to a predetermined resolution, and the resolution is converted. A resolution conversion unit that outputs conversion data, an area division unit that divides the resolution conversion data input from the resolution conversion unit according to a predetermined number of divisions, and outputs a plurality of divided division data; An encoding unit that performs encoding according to a plurality of different determined encoding rates on the divided data in parallel, and outputs encoded data for each encoded rate; and the encoding A multiplexing unit that multiplexes data for each input image; and the resolution conversion unit that executes resolution conversion processing on the input image, and performs the division processing on the resolution conversion data. To execute the dividing unit, a coding process for the divided data is performed on the coding unit, and a control unit to execute the multiplexing process in the multiplexing unit with respect to the encoded data, the resolution converting unit includes a plurality There are ( n ) input images, and the input images are input to any one of the n resolution conversion units in a time cycle in order of time frame by frame, and each of the n resolution conversion units selects an input frame. Are also converted to the same resolution, and there are n region dividing units, and each of the n region dividing units divides resolution conversion data input from the resolution conversion unit corresponding to one-to-one into the same division number m. The number of the encoding units is m according to the division number m of the region, and each of the m encoding units handles the same plurality of encoding rates and executes encoding according to the encoding rate to be handled. Multiple subs to The segment data of the same region from the n region segmentation units is input to correspond to the input order of the input image, and each sub-encoding process unit is stored in the buffer of each encoding unit. encoding in parallel the coding rate determined we were divided data.

One aspect of the present invention is the video encoding device described above, wherein the buffer of the encoding unit is a buffer that is used in common by the plurality of sub-encoding processing units.

A video encoding method according to an aspect of the present invention includes a resolution conversion step in which a resolution conversion unit converts the resolution of an input image in accordance with a predetermined resolution, and outputs resolution conversion data in which the resolution is converted. A division unit divides the resolution conversion data input from the resolution conversion unit according to a predetermined number of divisions, and outputs a plurality of divided division data, and is specified in the input image A control step of inputting region information indicating a display target region and controlling the encoding unit to execute an encoding process on the divided data including the display target region, and the encoding unit are different from each other in advance. Coding corresponding to a plurality of coding rates is performed on the divided data in parallel, and coded data for each coded coding rate is output. Of the method, a multiplexing step of multiplexing the encoded data for each of the input image, the resolution conversion unit is a plurality of (n), the time order of the input image frames, either inside of the n Are input to one resolution conversion unit in a time cycle, and each of the n resolution conversion units converts the input frames to the same resolution, and there are n region dividing units, Each region dividing unit divides the resolution conversion data input from the resolution conversion unit corresponding to one-to-one into the same division number m, and there are m encoding units according to the division number m of the region. , Each of the m encoding units includes a plurality of sub-encoding processing units that handle the same plurality of encoding rates and execute encodings according to the encoding rates to be handled, respectively. Is said n Is inputted as the divided data of the same region from the region dividing unit corresponding to the input order of the input image is encoded in parallel with encoding rate decided we were divided data from each sub-coding unit buffer Steps.

The present invention has been made to solve the above-described problem, and a program according to one embodiment of the present invention converts a resolution of an input image according to a predetermined resolution and converts the resolution of the computer. Resolution conversion means for outputting converted data, area dividing means for dividing the resolution conversion data input from the resolution conversion means according to a predetermined number of divisions, and outputting a plurality of divided divided data, predetermined Encoding means for executing encoding according to a plurality of different encoding rates in parallel with respect to the divided data and outputting encoded data for each of the encoded encoding rates; Multiplexing means for multiplexing for each input image, according to a predetermined encoding parameter, the resolution conversion processing for the input image is applied to the resolution conversion means. And executing the dividing process on the resolution conversion data by the area dividing unit, causing the encoding unit to execute the encoding process on the divided data, and executing the multiplexing process on the encoded data by the multiplexing unit. A plurality of ( n ) resolution conversion means, and the input image is input to any one of the n resolution conversion means in a time cycle in order of time frame by frame, means for converting the frame inputted to the resolution conversion unit either the same resolution, the area dividing means is also located the n well, the n resolution inputted from the resolution conversion unit corresponding to one-to-one to each area dividing means There are m means for dividing the converted data into the same number of divisions m, and the encoding means is m in accordance with the division number m of the area, and each of the m encoding means is the same A plurality of sub-encoding processing units that handle a number of encoding rates and execute encoding according to the encoding rate to be handled, and the buffer of each encoding unit has the same region from the n region dividing units divided data is input to correspond to the input order of the input image, each sub-encoding means causes encoded in parallel with encoding rate decided we were divided data from the buffer to the means, to function as the It is a program.

  According to the present invention, it is possible to shorten the processing time when creating encoded data of a display target area based on a divided image obtained by dividing an original image.

It is a figure which shows the structural example of the video coding apparatus 100 in the 1st Embodiment of this invention. It is a figure which shows the structural example of the file 300 assumed in each embodiment of this invention. It is a timing chart which shows the compression coding process which the video coding apparatus 100 in the 1st Embodiment of this invention performs. It is a flowchart which shows the process sequence example of the compression encoding which the video encoding apparatus 100 in the 1st Embodiment of this invention performs. It is a figure which shows the structural example of the video coding apparatus 100 in the 2nd Embodiment of this invention. It is a timing chart which shows the compression coding process which the video coding apparatus 100 in the 2nd Embodiment of this invention performs. It is a flowchart which shows the process sequence example of the compression encoding which the video coding apparatus 100 in the 2nd Embodiment of this invention performs. It is a figure which shows the structural example of the video coding apparatus 100 in the 3rd Embodiment of this invention. It is a timing chart which shows the compression coding process which the video coding apparatus 100 in the 3rd Embodiment of this invention performs. It is a flowchart which shows the process example of a compression encoding process which the video encoding apparatus 100 in the 3rd Embodiment of this invention performs. It is a figure which shows the other structural example about the encoding part 130 in the 3rd Embodiment of this invention. It is a flowchart which shows the process sequence example (1st example, 2nd example) for setting each parallel number of the resolution conversion part 110 in the embodiment of this invention, the area division part 120, and the encoding part 130. It is a flowchart which shows the process sequence example (3rd example) for setting each parallel number of the resolution conversion part 110 in the embodiment of this invention, the area division part 120, and the encoding part 132. 14 is a flowchart illustrating an example of a processing procedure for increasing the number of resources (step S605) in FIG. It is a figure which shows the modification about the correspondence of the resolution conversion part 110 and the area division part 120. It is a figure which shows typically the compression encoding of an ultra high resolution image | video. It is a figure which shows typically the reproduction | regeneration processing of the ultra-high resolution image | video by which compression encoding was carried out. It is a figure which shows typically the example of a delivery aspect of the compression-encoded super high resolution video. It is a figure which shows typically the example of a delivery aspect of the compression-encoded super high resolution video.

[About split video distribution]
First, with reference to FIGS. 16 to 19, a video division distribution system as a premise of the present invention will be described. The video divided distribution system is a system that divides an original image into a plurality of divided images and transmits a divided image including a designated display target area to the playback device side.
FIG. 16 schematically shows an example of compression encoding of ultra-high resolution video in this video distribution / distribution system.
A frame image 200 shown in FIG. 16A shows one frame as a panoramic ultra-high resolution video. In the compression coding of the frame image 200, as shown in FIG. 16B, the entire area of the frame image 200 is divided into a divided area 210 {210-1, 210-2,... With a predetermined number of horizontal / vertical pixels. -Divide by 210-32}. As an example, here, the division is performed by 32 (4 rows × 8 columns) divided regions 210-1, 210-2,... 210-32.

  Then, each of the divided areas 210 {210-1, 210-2,... 210-32} is compression-coded, and each of the divided areas 210 {210-1, 210-2,. Generate encoded data. Then, as shown in FIG. 16C, the encoded data of these divided areas 210 {210-1, 210-2,... 210-32} is managed as one file 300. This file 300 is an encoded video corresponding to one frame. By sequentially arranging the files 300 in units of frames, video stream data of one file is formed.

  FIG. 17 schematically shows the reproduction processing of the file 300 shown in FIG. FIG. 17A shows a file 300 similar to that in FIG. Here, it is assumed that the display target area 400 in the frame image 200 of FIG. In this case, the display target area 400 shown in FIG. 17B includes a part of each of the six divided areas 210-11, 210-12, 210-19, 210-20, 210-27, and 210-28. Includes area. Therefore, as reproduction processing, these divided areas 210-11, 210-12, 210-19, 210-20, 210-27 and 210-28 are read from the file 300, and decompression processing is executed. Then, the decoded images of the divided areas 210-11, 210-12, 210-19, 210-20, 210-27 and 210-28 are synchronized, and the image portion of the display target area 400 is displayed and output. A moving image is reproduced by sequentially executing this reproduction process for each frame.

  Here, assume a video distribution system in which video stream data composed of the file 300 is transferred from a server to a playback device via a network and played back as described above with reference to FIG. In this case, the video stream data to be distributed has a structure in which, for example, the files 300 corresponding to each frame are arranged, so that the amount of data per unit reproduction time is considerable, so that it is considerably high in transmission. Data rate is required. In the present situation where the bandwidth of the network is limited, it is difficult to transmit video stream data having such a high data rate.

  Therefore, as described in Non-Patent Document 2, a technique has been proposed in which only encoded data in a divided area required on the playback device side is partially extracted from video stream data and transferred. 18 and 19 schematically show an example of video distribution by this technique.

  FIG. 18A shows the structure of the file 300 corresponding to one frame image 200. First, the frame image 200 is subjected to resolution conversion with a plurality of resolutions. Thereby, based on the frame image 200, a plurality of frame images having different predetermined resolutions are generated. That is, as shown by the horizontal axis in FIG. 18A, the file 300 has a structure in which a plurality of different resolutions are arranged as frame images 200 having the same image content.

  Next, the frame image 200 whose resolution has been converted as described above is divided into a plurality of divided regions 210 in the same manner as described above with reference to FIGS. 16 (a) and 16 (b). In the example of FIG. 18A, one frame image 200 is divided into 32 divided regions 210-1 to 210-32 as in FIG. 16 and FIG.

  These divided regions 210 perform compression encoding with a plurality of different encoding amounts (bit rates). As a result, as shown by the vertical axis in FIG. 18A, a plurality of frame images each composed of divided regions 210-1 to 210-32 each encoded with a different code amount corresponding to one resolution. Will be arranged. That is, in the file 300 shown in FIG. 18A, the video at the position of one divided region 210 is provided with α, which is the same number as the encoding amount of α. The distribution server (not shown) stores video stream data in which the file 300 having the structure shown in FIG. 18A is arranged for each frame.

  Here, it is assumed that the image of the display target area 400 in FIG. 18B is reproduced and displayed from the file 300 shown in FIG. FIG. 18C shows an example of a display image 500 obtained by reproducing the display target area 400 of FIG.

  The display target area 400 shown in FIG. 18B includes eight divided areas 210-10, 210-11, 210-12, 210-13, 210-18, 210-19, 210- in the frame image 200. Each of the image portions 20 and 210-21 is formed. In order to improve the video quality of the display target area within the limited band, it is necessary to allocate a high coding amount to the divided area with low quality. In order to determine the video quality, quality evaluation information (for example, PSNR information) exists for all frames in all divided regions. While the band is allowed, the coding amount is increased in order from the lowest quality divided area. In the example of FIG. 18, four divided areas 210-11, 210-12, 210-19 and 210-20 are improved to the highest encoding amount, and the remaining divided areas are improved to the second encoding amount. Is shown.

  Further, when displaying an ultra-high resolution panoramic image, it is assumed that a partial area in the frame image 200 is zoomed in (enlarged) to be reproduced and displayed, or zoomed out (reduced) to be reproduced and displayed. Yes. FIG. 19 schematically shows an example of data distribution corresponding to reproduction by zoom-out.

  FIG. 19A shows the structure of the file 300 similar to that of FIG. Here, it is assumed that the frame image 200 is zoomed out (reduced) and reproduced and displayed as shown in a display image 500 in FIG. As shown in FIG. 19B, the display target area 400 corresponding to the display image 500 includes four divided areas 210-21, 210-22, 210-24, and 210-25 having a lower resolution. Formed by each image portion. In the case of zoom-out display, compared with the case of FIG. 18, for example, the image range to be displayed is widened by the same four divided areas, so the divided areas are selected from lower resolution frames. Therefore, the distribution server reads the divided areas 210-21, 210-22, 210-24, and 210-25 from the frame image 200 having the lower resolution and transfers them to the playback device as shown in FIG. To do.

  On the other hand, although illustration is omitted, when a wide range is displayed by zooming out (reducing), conversely, a high resolution above a certain level becomes excessive. Therefore, the distribution server in this case reads out the divided area 210 necessary for reproduction of the display target area 400 from the frame image 200 having the lower resolution among the plurality of resolutions, and transfers it to the reproduction apparatus.

In the video distribution of the mode shown in FIGS. 18 and 19, the divided area 210 required corresponding to the area to be reproduced by the reproduction apparatus is selected and transferred. Furthermore, the encoding amount of the divided area 210 to be transferred is appropriately selected according to the occupation ratio of the video portion in the display target area 400. Further, the resolution of the divided area 210 to be transferred is appropriately selected according to the display magnification of the display video 500. As a result, it is possible to suppress the data rate of the distribution data and distribute the ultra-high resolution video in a network environment with a band limitation.
For simplification of description, the video division distribution system which is the premise of the present invention has been described on the assumption that distribution is performed in units of frames. Needless to say, when encoding is performed using an encoding method such as H.264, it is not a frame unit but a GOP unit.

<First Embodiment>
Next, an example of a video encoding device according to an embodiment of the present invention will be described. FIG. 1 is a diagram illustrating an example of a configuration of a video encoding device according to the present embodiment. The video encoding apparatus according to the first embodiment described below is an example of an apparatus used in the video division distribution system described above.

[Configuration of video encoding device]
FIG. 1 shows a configuration example of a video encoding apparatus 100 as the first embodiment of the present invention. The video encoding apparatus 100 shown in this figure includes a resolution conversion unit 110, a region division unit 120, an encoding unit 130, a multiplexing unit 140, and a control unit 150. The resolution converter 110 includes a plurality of resolution converters, and includes, for example, a first resolution converter 110-1 to a third resolution converter 110-3. The region dividing unit 120 includes a plurality of region dividing units, and includes, for example, a first region dividing unit 120-1 to a third region dividing unit 120-3. The encoding unit 130 includes a plurality of encoding units that can encode the same data in parallel at different encoding rates. For example, the first encoding unit 130-1 to the fourth encoding unit 130- 4 is included. In the present embodiment, each of the first encoding unit 130-1 to the fourth encoding unit 130-4 constitutes a separate device.

  The buffers 111-1 to 111-3 are provided corresponding to the first resolution conversion unit 110-1 to the third resolution conversion unit 110-3, respectively. The buffers 111-1 to 111-3 hold data to be processed by the first resolution conversion unit 110-1 to the third resolution conversion unit 110-3, respectively.

  Buffers 121-1 to 121-3 are provided corresponding to first region dividing unit 120-1 to third region dividing unit 120-3, respectively. The buffers 121-1 to 121-3 hold data to be processed by the first area dividing unit 120-1 to the third area dividing unit 120-3, respectively.

The buffers 131-1 to 131-4 are provided corresponding to the first encoding unit 130-1 to the fourth encoding unit 130-4, respectively. The buffers 131-1 to 131-4 hold data to be processed by the first encoding unit 130-1 to the fourth encoding unit 130-4, respectively.
The buffer 141 holds data to be processed by the multiplexing unit 140.

  The input video is video signal data in a predetermined format with ultra-high resolution, for example, called 4k resolution or 8k resolution. Here, for convenience of explanation, it is assumed that the input is performed in units of frame images. The frame image data as the input video branches to the buffers 111-1 to 111-3, and a different frame image is supplied for each frame. The buffers 111-1 to 111-3 hold the supplied frame images.

  The first resolution conversion unit 110-1 to the third resolution conversion unit 110-3 input frame images from the buffers 111-1 to 111-3, respectively, and frames at a predetermined resolution or a plurality of resolutions. A resolution conversion process for generating an image is performed. For example, a plurality of predetermined resolutions are set in each of the first resolution conversion unit 110-1 to the third resolution conversion unit 110-3. The plurality of resolutions set in the first resolution conversion unit 110-1 to the third resolution conversion unit 110-3 are the same resolution. The first resolution conversion unit 110-1 to the third resolution conversion unit 110-3 convert an input frame image to a set resolution, and output a frame image corresponding to each resolution. As a result, a plurality of frame images each converted to a different resolution are generated by one resolution conversion unit 110. Note that the number of resolution patterns and the resolution value in each resolution pattern are determined in advance, and one frame image is converted into a resolution corresponding to the number of patterns. In this embodiment, for convenience of explanation, it is assumed that the same resolution is set in the first resolution conversion unit 110-1 to the third resolution conversion unit 110-3. The first resolution conversion unit 110-1 to the third resolution conversion unit 110-3 each output a frame image converted to a set resolution.

  In the buffers 121-1 to 121-3, data of a plurality of frame images corresponding to a plurality of resolution patterns generated by the first resolution conversion unit 110-1 to the third resolution conversion unit 110-3 are respectively stored. Entered.

  The first region dividing unit 120-1 to the third region dividing unit 120-3 respectively receive the frame image data held in the buffers 121-1 to 121-3, and convert the frame image to a predetermined size ( The area dividing process is executed so as to divide the image into divided areas of (resolution). That is, if one frame image 200 input by the region dividing unit 120 is the one shown in FIG. 16A, the region dividing unit 120 displays the frame image 200 as shown in FIG. By dividing the image by a predetermined number of horizontal / vertical pixels, it is converted into a format composed of 32 divided regions 210 {210-1, 210-2,... 210-32}. The first region dividing unit 120-1 to the third region dividing unit 120-3 respectively convert the frame images converted to the resolution set for the same frame image into 32 divided regions 210 {210-1, 210. -2, ... 210-32}.

  The data of the divided area 210 output from the first area dividing unit 120-1 to the third area dividing unit 120-3 is appropriately input to any one of the buffers 131-1 to 131-4. The buffers 131-1 to 131-4 hold the input data of the divided area 210.

  The first encoding unit 130-1 to the fourth encoding unit 130-4 input the data held in the buffers 131-1 to 131-4, respectively, and execute the compression encoding process. Although the compression encoding method here is not particularly limited, as an example, a multi-view video encoding method (MVC) can be adopted. In the following, compression encoding is simply referred to as encoding.

  Data of the encoded divided region obtained by encoding the divided region 210 by the first encoding unit 130-1 to the fourth encoding unit 130-4 is input to the buffer 141 and held therein.

  The multiplexing unit 140 receives the data of the encoded divided area held in the buffer 141, and executes a multiplexing process for multiplexing the data of the encoded divided area in units of frames into one file. . Note that the number of frames per frame depends on the encoding method. When a multi-view video coding method (MVC: MultiView Video Coding) is adopted, the number of frames is in GOP units. As a result, as shown in FIG. 16A, a file 300 corresponding to one frame image 200 has a structure including a number of divided regions 210 corresponding to a combination of a plurality of resolutions and a plurality of encoding amounts. Then, the multiplexing unit 140 sequentially outputs the file 300 corresponding to this frame unit as an output video. Thus, video stream data is formed by concatenating the files 300 that are sequentially output.

  Note that the parallel number of each component included in the resolution conversion unit 110, the region division unit 120, and the encoding unit 130 illustrated in FIG. 1 corresponds to a specific example of encoding processing of the video encoding device 100 described later. It is only an example.

  The control unit 150 performs control on the resolution conversion unit 110, the region division unit 120, the encoding unit 130, and the multiplexing unit 140.

The resolution conversion unit 110, the region division unit 120, the encoding unit 130, and the multiplexing unit 140 are realized by, for example, a DSP (Digital Signal Processor), and the control unit 150 controls the resolution conversion unit 110 by controlling the DSP. Further, the signal processing circuit can be formed by changing the parallel numbers of the area dividing unit 120 and the encoding unit 130.
Further, the control unit 150 instructs data distribution and processing timing to the resolution conversion unit 110, the region division unit 120, the encoding unit 130, and the multiplexing unit 140 according to the set configuration of the signal processing circuit.

  If the video stream data generated by the video encoding apparatus 100 having the above configuration is distributed via, for example, a network, a distribution server (not shown) is provided on the network, and the video stream data formed as described above is provided. Will be remembered. When the distribution server performs video distribution, for example, as described with reference to FIGS. 18 and 19, only the data of the divided area 210 corresponding to the image of the portion to be displayed by the playback device in the frame image 200 is obtained. To deliver. Further, the distribution server may appropriately select and distribute the encoding amount of the divided area 210 to be transferred according to the occupancy rate of the video portion in the display target area 400. Further, the resolution of the divided area 210 to be transferred may be appropriately selected according to the display magnification of the display video 500 and distributed. As a result, it becomes possible to deliver an ultra-high resolution video via a limited bandwidth network.

[Example of encoding processing timing]
Next, an example of the execution timing of the encoding process executed by the video encoding apparatus 100 having the configuration shown in FIG. 1 will be described. In the following description, the structure of the file 300 shown in FIG. 2 is assumed. Here, for simplification of description, it is assumed that one frame image 200 is divided into two regions, a first divided region 210-1 and a second divided region 210-2. In addition, the encoding amount is assumed to be encoded at two encoding rates γ1 and γ2 of the first encoding amount and the second encoding amount. The resolution is α. The file 300 in FIG. 2 has the number of divided areas 210 expressed by (number of divisions “2” × number of coding rate patterns “2”). Actually, the number of resolution patterns, the number of divisions, and the number of coding rate patterns are very large, and the number of divided regions 210 in the file 300 is enormous. In general, when the number of divisions at the resolution n is α _n and the encoding rate at the resolution n is β _n , the number of divided areas in the file 300 is Σ (α _n × β _n ).
Can be expressed as
Of the data corresponding to the file 300, the divided area encoded by the first encoding unit 132-1 is stored in the buffer 131-1. Similarly, the divided areas encoded by the encoding units 132-2 to 132-4 are stored in the buffers 131-2 to 131-4, respectively. It should be noted that the number of divided areas and the number of encoding units are not necessarily the same, and are not necessarily assigned one-to-one. Allocation of divided areas and encoding is performed by the control unit 150.
For example, the first encoding unit 132-1 encodes the first divided region with the first encoding amount, the second encoding unit 132-2 encodes the second divided region with the first encoding amount, When the third encoding unit 132-3 encodes the first divided region with the second encoding amount, and the fourth encoding unit 132-4 encodes the second divided region with the second encoding amount An example will be described. In this case, the control unit 150 stores the encoding parameter (for example, the resolution of the input video, the resolution of the area division, the number of patterns of resolution, the number of patterns of encoding amount, etc.) in the buffer 131-1. Stores the first divided area, the buffer 131-2 stores the second divided area, the buffer 131-3 stores the first divided area, and the buffer 131-4 stores the second divided area. The
Also, the first encoding unit 132-1 encodes the first divided region with the first encoding amount and the second encoding amount, and the second encoding unit 132-2 converts the second divided region into the first code. When encoding with the encoding amount and the second encoding amount, the following control is performed. That is, the control unit 150 stores the first divided area in the buffer 131-1 and the second divided area in the buffer 131-2. Therefore, data is not stored in the buffer 131-3 and the buffer 131-4.

  The timing chart of FIG. 3 shows an example of the encoding process executed by the video encoding device 100 of the first embodiment shown in FIG. 1 over time. First, as shown in the figure, the input video is sequentially input, for example, at a timing corresponding to the frame period, such as frames 1, 2, 3, 4,.

  The buffers 111-1 to 111-3 corresponding to the resolution conversion processing sequentially hold the frames 1, 2, 3, 4... Input as described above at the timing shown in the drawing. Specifically, the buffer 111-1 holds frame 1, the buffer 111-2 holds frame 2, and the buffer 111-3 holds frame 3. As described above, when the data holding by the buffers 111-1 to 111-3 is completed, the buffer 111-1 holds the frame 4, and thereafter the frame holding by the buffers 111-1 to 111-3 is similarly performed. Is repeated.

  The first resolution conversion unit 110-1 inputs the data of frame 1 held in the buffer 111-1, and executes resolution conversion processing. That is, frame images having different resolutions α1 (hereinafter referred to as resolution conversion data) are generated from the frame 1 data. Note that this process can be bypassed if different resolutions are not created. For example, the first resolution may be the same as the input video.

  Next, the second resolution conversion unit 110-2 inputs the frame 2 held in the buffer 111-2, executes resolution conversion processing, and generates resolution conversion data of the resolution α1. Next, the third resolution conversion unit 110-3 inputs the frame 3 held in the buffer 111-3, executes resolution conversion processing, and generates resolution conversion data of the resolution α1. Next, since the frame 4 is held in the buffer 111-1, the first resolution conversion unit 110-1 inputs the frame 4 and executes resolution conversion processing to generate resolution conversion data of the resolution α1. As described above, the first to third resolution conversion units 110-1 to 110-3 sequentially input the frame data and cyclically execute the resolution conversion process.

  As described above, the resolution conversion data of the frame 1 having a plurality of resolutions generated by the first resolution conversion unit 110-1 is output to the buffer 121-1 corresponding to the region division processing and held therein. Next, the resolution conversion data of frame 2 having a plurality of resolutions generated by the second resolution conversion unit 110-2 is output to the buffer 121-2 and held therein. Next, resolution conversion data of the frame 3 having a plurality of resolutions generated by the third resolution conversion unit 110-3 is output to the buffer 121-3 and held therein. Next, the resolution conversion data of the frame 4 having a plurality of resolutions generated by the first resolution conversion unit 110-1 is output to the buffer 121-1, and held therein.

  The first area dividing unit 120-1 receives resolution conversion data of frame 1 having a plurality of resolutions from the buffer 121-1, and divides the data into a first divided area 210-1 and a second divided area 210-2 as an area dividing process. To do. In the figure, the data of the first divided area 210-1 of frame 1 is represented as a divided area 1-1, and the data of the second divided area 210-2 of frame 1 is represented as a divided area 1-2. .

  Next, the second area dividing unit 120-2 inputs resolution conversion data of frame 2 having a plurality of resolutions from the buffer 121-2, and similarly, the first divided area (divided area 2-1) and the second divided area. Divide into (divided areas 2-2). Next, the third area dividing unit 120-3 inputs resolution conversion data of the frame 3 having a plurality of resolutions from the buffer 121-3, and the first divided area (divided area 3-1) and the second divided area (divided area). Divide into 3-2). Next, the first area dividing unit 120-1 inputs resolution conversion data of the frame 4 having a plurality of resolutions from the buffer 121-1, and the first divided area (divided area 4-1) and the second divided area (divided area). 4-2). As described above, the first region dividing unit 120-1 to the third region dividing unit 120-3 also cyclically input frame data having a plurality of resolutions cyclically and execute the region dividing process. In the following description, the same notation as in FIG. 3 will be used to describe the data shown in FIG.

  Of the divided regions 1-1 and 1-2 generated by the first region dividing unit 120-1, the divided region 1-1 includes a buffer 131-1 and a second code corresponding to the first encoding unit 132-1. Are input in parallel to the buffer 131-2 corresponding to the generating unit 132-2 and are held here. On the other hand, the divided region 1-2 is input in parallel to the buffer 131-3 corresponding to the third encoding unit 132-3 and the buffer 131-4 corresponding to the fourth encoding unit 132-4. Retained.

  Next, among the divided regions 2-1 and 2-2 generated by the second region dividing unit 120-2, the divided region 2-1 includes a buffer 131-1 corresponding to the first encoding unit 130-1, and The data is input in parallel to the buffer 131-2 corresponding to the second encoding unit 130-2 and held here. The divided region 2-2 is input in parallel to and held in the buffer 131-3 corresponding to the third encoding unit 130-3 and the buffer 131-4 corresponding to the fourth encoding unit 130-4. The

  Next, among the divided regions 3-1 and 3-2 generated by the third region dividing unit 120-3, the divided region 3-1 includes a buffer 131-1 corresponding to the first encoding unit 130-1, and The signals are input in parallel to and held in the buffer 131-2 corresponding to the second encoding unit 132-2. The divided area 3-2 is input in parallel to and held in the buffer 131-3 corresponding to the third encoding unit 132-3 and the buffer 131-4 corresponding to the fourth encoding unit 132-4. The

  Next, among the divided regions 4-1 and 4-2 generated by the fourth region dividing unit 120-3, the divided region 4-1 includes a buffer 131-1 corresponding to the first encoding unit 130-1, and The signals are input in parallel to and held in the buffer 131-2 corresponding to the second encoding unit 132-2. The divided area 4-2 is input in parallel to the buffer 131-3 corresponding to the third encoding unit 132-3 and the buffer 131-4 corresponding to the fourth encoding unit 132-4, and is held here. The

  As a result, the buffers 131-1 and 131-2 corresponding to the first encoding unit 132-1 and the second encoding unit 132-2, respectively, are divided regions 1-1, 2-1, and 3-1. And 4-1. Also, the buffers 131-3 and 131-4 corresponding to the third encoding unit 132-3 and the fourth encoding unit 132-4, respectively, are divided regions 1-2, 2-2, 3-2, and 4-2 is held sequentially.

  As described with reference to FIG. 2, here, it is assumed that encoding is performed using two encoding amounts, ie, a first encoding amount and a second encoding amount. Among the first encoding unit 132-1 to the fourth encoding unit 132-4, the first encoding unit 132-1 and the second encoding unit 132-2 each have first data on the first divided region. Encoding is performed using the encoding amount and the second encoding amount. The third encoding unit 132-3 and the fourth encoding unit 132-4 perform encoding with the first encoding amount and the second encoding amount, respectively, on the data of the second divided region.

  The first encoding unit 132-1 starts with the divided region 1 of the frame 1 among the data of the divided regions 1-1, 2-1, 3-1 and 4-1 held in the buffer 131-1. -1 is input, encoding is performed using the first encoding amount, and the encoded divided region 1-1-1 is generated. Thereafter, the first encoding unit 132-1 sequentially receives the divided areas 2-1, 3-1 and 4-1 of the frames 2, 3, and 4 and inputs the encoded divided areas 2-1-1, 3- 1-1 and 4-1-1 are generated.

  Of the data of the divided areas 1-1, 2-1, 3-1 and 4-1 held in the buffer 131-2, the second encoding unit 132-2 first selects the divided area 1-1. Input and perform encoding with the second encoding amount to generate an encoded divided region 1-1-2. Thereafter, the second encoding unit 132-2 sequentially inputs the divided areas 2-1, 3-1, and 4-1, and the encoded divided areas 2-1-2, 3-1-2, and 4-1. -2 is generated.

  Of the data in the divided areas 1-2, 2-2, 3-2 and 4-2 held in the buffer 131-3, the third encoding unit 132-3 first selects the divided area 1-2. Input and perform encoding using the first encoding amount to generate an encoded divided region 1-2-1. Thereafter, the third encoding unit 132-3 sequentially inputs the divided areas 2-2, 3-2, and 4-2, and the encoded divided areas 2-2-1, 3-2-1, and 4-2. -1 is generated.

  Of the data of the divided areas 1-2, 2-2, 3-2 and 4-2 held in the buffer 131-4, the fourth encoding unit 132-4 first selects the divided area 1-2. Input and perform encoding with the second encoding amount to generate an encoded divided region 1-2-2. Thereafter, the third encoding unit 132-3 sequentially inputs the divided areas 2-2, 3-2, and 4-2, and the encoded divided areas 2-2-2, 3-2-2, and 4-2. -2 is generated.

  As described above, the first encoding unit 132-1 to the fourth encoding unit 132-4 execute the encoding process, so that the two first divided regions and the second divided region in one frame are Encoding with one encoding amount and encoding with a second encoding amount are performed. That is, in one frame, the first divided region based on the first coding amount, the first divided region based on the second coding amount, the first divided region based on the second coding amount, and the second divided region based on the second coding amount. Four patterns of encoded divided areas with the divided areas are obtained. Then, four patterns of encoded divided areas in one frame are sequentially formed for each frame as time elapses. Each of the data of these encoded divided areas is composed of a plurality of divided areas corresponding to a plurality of resolutions.

  Each time the first encoding unit 132-1 sequentially generates the data of the encoded divided regions 1-1-1, 1-2-1, 3-1-1 and 4-1-1 as described above. Then, these data are transferred to the buffer 141 corresponding to the multiplexing unit 140. Similarly, every time the second encoding unit 132-2 sequentially generates data of the encoded divided areas 1-1-2, 2-1-2, 3-1-2, and 4-1-2. , Transferred to the buffer 141. Each time the third encoding unit 132-3 sequentially generates the data of the encoded divided areas 1-2-1, 2-2-1, 3-2-1 and 4-2-1, the buffer 141 Forward to. Each time the fourth encoding unit 132-4 sequentially generates the data of the encoded divided areas 1-2-2, 2-2-2, 3-2-2, and 4-2-2, the buffer 141 Forward to. As a result, the buffer 141 sequentially holds the data transferred as described above at the timing shown in FIG.

  The multiplexing unit 140 is a timing at which all of the four encoded division regions 1-1-1, 1-1-2, 1-2-1, 1-2-2 corresponding to the frame 1 are held in the buffer 141. Then, these encoded divided areas are input and a multiplexing process is executed to generate a file corresponding to frame 1 as shown in the figure. Next, in the multiplexing unit 140, all of the four encoded divided areas 2-1-1, 2-1-2, 2-2-1, and 2-2-2 corresponding to the frame 2 are held in the buffer 141. At this timing, these encoded divided areas are input and the multiplexing process is executed to generate a file corresponding to frame 2.

  Thereafter, the multiplexing unit 140 performs the multiplexing process at the timing at which all the corresponding encoded divided areas are held in the buffer 141 for each of the frames 3, 4,. Generate the file. In this way, stream data as an output video is obtained by sequentially generating a file in units of frames. Alternatively, frame-by-frame files are sequentially combined as one file to obtain an output video file.

  As shown in the processing timing of FIG. 3 above, the process of sequentially encoding the frames by the first encoding unit 132-1 to the fourth encoding unit 132-4 is also shown in FIG. It is executed in parallel. In other words, the present embodiment is equivalent to the case where no segmentation is performed for the time required for the encoding process by providing a plurality of encoding units 130 in parallel and causing them to execute the encoding process in parallel. It is what you are trying. As a result, in the present embodiment, it is possible to greatly reduce the time required for the encoding process regardless of whether the frame image 200 is divided by the divided region 210.

  Note that the actual processing time for each frame varies. For example, the resolution conversion unit 110 and the region division unit 120 have different processing times for each frame depending on the resolution of data to be processed. Further, in the encoding process by the encoding unit 130, for example, the processing time is different for each of an I (Intra) picture, a P (Predictive) picture, and a B picture (Bi-directional Predictive). Therefore, buffers 111, 121, 131, and 141 are provided for the resolution conversion unit 110, the region division unit 120, the encoding unit 130, and the multiplexing unit 140, respectively. Thereby, the processing timing can be synchronized by absorbing the difference in the processing time.

[Example of encoding procedure]
The flowchart in FIG. 4 illustrates an example of a processing procedure executed by the video encoding device 100 according to the first embodiment. This figure shows a procedure for processing one frame at the processing timing shown in FIG. The processing shown in this figure can be regarded as being appropriately executed by each unit shown in FIG.

  In step S101, the control unit 150 substitutes an initial value “1” for the variable n indicating the frame number. Next, in step S102, the control unit 150 converts the frame n supplied as the input video into the buffer 111 corresponding to the ((n · mod · m)) resolution converter 110- (n · mod · m)). -Transfer to (n · mod · m). m is the number of resolution converters 110 and is “3” here. Further, (n · mod · m) indicates a remainder when n is divided by m. However, when m is “3”, (n · mod · m) can take any of “0”, “1”, and “2”, but when “0”, “m” Take the value. Thus, as shown in FIG. 3, the buffers 111 corresponding to the first to third resolution conversion units 110-1 to 110-3 sequentially circulate through the frames 1, 2, 3, 4. -1 to 111-3. In order to simplify the notation, (n · mod · m) will be replaced with “N” in the following.

  Next, in step S103, the Nth resolution conversion unit 110-N executes resolution conversion processing. That is, data of frame n with the first to i-th resolution is generated from frame n. In step S104, the Nth resolution conversion unit 110-N transfers the data of the frame n with the first to ith resolutions to the buffer 121-N corresponding to the Nth region division unit 120-N and holds it. Let

  Next, in step S105, the Nth region dividing unit 120-N inputs the data of the frame n with the first to i-th resolutions held in the buffer 121-N and executes the region dividing process. That is, the data of frame n is divided into a first divided area 210-1 and a second divided area 210-2. In correspondence with the notation in FIG. 3, the image is divided into divided areas n−1 and n−2. In the following description of FIG. 4, the notation of FIG. 3 is used.

  In step S106, the Nth region dividing unit 120-N transfers the data of the divided region n-1 to the buffer 131-1, which corresponds to the first encoding unit 132-1, and holds the data. In step S107, the Nth region dividing unit 120-N transfers and holds the data of the divided region n-1 to the buffer 131-2 corresponding to the second encoding unit 132-2.

  In step S108, the Nth region dividing unit 120-N transfers the data of the divided region n-2 to the buffer 131-3 corresponding to the third encoding unit 132-3, and holds the data. Further, in step S109, the Nth region dividing unit 120-N transfers and holds the data of the divided region n-2 to the buffer 131-4 corresponding to the fourth encoding unit 132-4.

  Next, in step S110, the first encoding unit 132-1 receives the data of the divided region n-1 held in the buffer 131-1, encodes the first encoded amount, and performs encoding division. Data of region n-1-1 is generated. Then, the first encoding unit 132-1 transfers and stores the data of the encoded division region n-1-1 to the buffer 141 corresponding to the multiplexing unit 140.

  In addition, in step S111, the second encoding unit 132-2 receives the data of the divided area n-1 held in the buffer 131-2, encodes it with the second encoding amount, and encodes the encoded divided area. Data of n-1-2 is generated. Then, the second encoding unit 132-2 transfers the data of the encoded divided region n-1-2 to the buffer 141 to hold it.

  In addition, in step S112, the third encoding unit 132-3 inputs the data of the divided region n-2 held in the buffer 131-3, encodes the first encoded amount, and encodes the encoded divided region. Data of n-2-1 is generated. Then, the third encoding unit 132-3 transfers the data of the encoded divided region n-2-1 to the buffer 141 to hold it.

  In addition, in step S113, the fourth encoding unit 132-4 inputs the data of the divided region n-2 held in the buffer 131-4, encodes it with the second encoding amount, and encodes the encoded divided region. Data of n-2-2 is generated. Then, the fourth encoding unit 132-4 transfers the data of the encoded divided region n-2-2 to the buffer 141 to hold it. As described with reference to FIG. 3, the encoding processing in steps S110 to S113 is performed in parallel at the same time.

  The multiplexing unit 140 holds the data of the four encoded divided regions n-1-1, n-1-2, n-2-1 and n-2-2 obtained in steps S110 to S113. Then, in step S114, a process of multiplexing these is executed. Thereby, a file corresponding to the frame n is formed. When the processing up to step S114 ends, the control unit 150 increments the variable n in step S115 and returns to the processing in step S102. By executing the processing in this way, the file 300 can be sequentially formed in units of frames.

  In distributing the video stream data generated by the video encoding device 100, a distribution mode according to FIGS. 18 and 19 can be assumed. In other words, the encoding amount and resolution data necessary for reproduction are further selected from the divided areas necessary for reproduction by the reproduction apparatus and distributed. This can be applied to a service in which, for example, a user does not specify a viewing area, but distributes a part of the input video that has been edited so as to be appropriately enlarged or reduced on the distribution side. As a process of the video encoding apparatus 100 corresponding to such a distribution mode, the following can be considered.

  First, in the video encoding apparatus 100 shown in FIG. 1, the resolution conversion unit 110 and the region division unit 120 respectively perform resolution conversion processing and a divided region 210 using a predetermined number of resolution patterns set in advance as described above. A divided area process is performed to divide into two. Then, the data of these divided areas 210 is held in the buffer 131 in the encoding unit 130, for example, as shown in FIG. Up to this stage, the processing procedure is the same as that described above with reference to FIGS.

  In addition, in response to a playback data request from the playback device, the control unit 150 determines the divided regions to be transmitted to the playback device, and the combination of the coding amount of the data in these divided regions and the resolution pattern. Then, the control unit 150 instructs the encoding unit 130 that executes the encoding process according to the determined combination of the segmented area and the encoding amount from among all the encoding units 130. At this time, the control unit 150 designates a resolution pattern to be encoded to the encoding unit 130 that has instructed the encoding process. In response to this, the encoding unit 130 that has received an instruction for the encoding process selects the frame of the designated resolution pattern from the frame data corresponding to each of the plurality of resolution patterns held in the corresponding buffer 131. Only the data is input and the encoding process is executed.

  As a specific example, assuming the case shown in FIG. 2 with respect to the divided areas and the encoding amount, the resolution conversion unit 110 performs resolution conversion processing with two patterns of resolutions α1 and α2 in total. In addition, it is assumed that the data necessary for reproduction is the divided area 210-1, the resolution is α1, and the data is encoded with the second encoding amount. In this case, among the encoding units 132-1 to 132-4 in FIG. 1, only the encoding unit 132-2 that processes the divided region 210-1 and executes the processing based on the second encoding amount is encoded. Will be executed. In addition, when inputting data from the buffer 131-2, the encoding unit 132-2 performs encoding processing by inputting only the data of the resolution α1 frame out of the resolution α1 and resolution α2 frames. Become. Accordingly, steps S110, S112, and S113 are omitted from the flowchart of FIG. Further, in this case, the multiplexed file for each frame is formed by the data of the compressed divided area encoded by the encoding unit 132-2.

  By performing the processing as described above, only a part of the encoding units 130 corresponding to the combination of the divided area 210 and the encoding amount necessary for reproduction perform the encoding process, and further the code for executing the encoding process The encoding unit 130 may perform the encoding process only on the frames of the resolution pattern necessary for reproduction. As a result, the number of parallel encoding processes is reduced, the processing load is reduced, and the number of frames processed by one encoding unit 132 is also reduced, thereby shortening the processing time. As for the resolution conversion process, for example, the control unit 150 may be configured to cause the resolution conversion unit 110 to execute the resolution conversion process using only the resolution pattern necessary for reproduction. Further, in the case of a distribution system that does not need to execute encoding processing in real time, the video encoding device 100 uses all combinations of resolution patterns and encoding amount patterns to encode all divided regions. Can be done. Then, the video stream data generated in this way is stored in the storage device. In response to a request from the playback device, the distribution server reads the data of the divided area 210 based on the combination of the resolution pattern and the encoding amount pattern necessary for playback from the video stream data stored in the storage device, and reads the data in a predetermined format. Is transmitted to the playback device as content data.

<Second Embodiment>
[Configuration of video encoding device]
FIG. 5 shows a configuration example of the video encoding device 100 according to the second embodiment. In this figure, the same parts as those in FIG. The video encoding apparatus 100 shown in this figure includes two first encoding units 130-1 and a second encoding unit 130-2 as parts for executing the encoding process.

  In addition, the first encoding unit 130-1 includes therein a first sub encoding unit 132-1, a buffer 131-1, a second sub encoding unit 132-2, and a buffer 131-2. The second encoding unit 130-2 includes therein a third sub encoding unit 132-3, a buffer 131-3, a fourth sub encoding unit 132-4, and a buffer 131-4.

  The first sub-encoding unit 132-1 to the fourth sub-encoding unit 132-4 correspond to the first encoding unit 132-1 to the fourth encoding unit 132-4 in FIG. Execute. That is, in the first encoding unit 130-1, the first sub-encoding unit 132-1 receives the data of the first divided area 210-1 held in the buffer 131-1, and receives the first encoding amount. The encoding process by is executed. The second sub-encoding unit 132-2 receives the data of the first divided area 210-1 held in the buffer 131-2, and executes an encoding process using the second encoding amount. Therefore, the 1st encoding part 130-1 becomes a site | part which performs the encoding process corresponding to a 1st division area.

  On the other hand, the 2nd encoding part 130-2 becomes a site | part which performs the encoding process corresponding to a 2nd division area. That is, the third sub-encoding unit 132-3 receives the data of the second divided area 210-2 held in the buffer 131-3 and executes the encoding process using the first encoding amount. The fourth sub-encoding unit 132-4 receives the data of the second divided area 210-2 held in the buffer 131-4 and executes an encoding process using the second encoding amount.

[Example of encoding processing timing]
The timing chart of FIG. 6 shows an example of the execution timing of the encoding process executed by the video encoding apparatus 100 as the second embodiment shown in FIG. In this figure, the description of the same processing as in FIG. 3 is omitted.

  The encoding process in FIG. 6 is performed by replacing the first encoding unit 132-1 to the fourth encoding unit 132-4 shown in FIG. 3 with the first sub encoding unit 132-1 to the fourth sub code. The execution unit 132-4 executes. Here, the encoding process timing executed by the first sub-encoding unit 132-1 to the fourth sub-encoding unit 132-4 is the first sub-encoding unit 132-1 to the fourth sub-encoding unit in FIG. This is the same as the encoding process executed by 132-4. Therefore, the execution timing of each process shown in FIG. 6 is the same as that in FIG.

  However, in the previous FIG. 3, for example, the first region dividing unit 120-1 transfers the data of the divided region 1-1 obtained by the dividing process to the first encoding unit 132-1, and further performs the second encoding. It was made to transfer to the part 132-2. Further, the first region dividing unit 120-1 is configured to transfer the data of the divided region 1-2 to each of the third encoding unit 132-3 and the fourth encoding unit 132-4. That is, in the first embodiment, when the first area dividing unit 120-1 transfers the data of the divided area, the transfer is performed as many times as the number of patterns of the encoding amount. That is, the operation of transferring the same data a plurality of times is executed, and the data transmission amount increases.

  On the other hand, the first area dividing unit 120-1 of the second embodiment transfers the data of the divided area 1-1 to the first encoding unit 130-1. The first encoding unit 130-1 stores and holds the data transferred in this way in the buffer 131-1 corresponding to the first sub-encoding unit 132-1 as shown in FIG. Next, the first encoding unit 130-1 transfers the data of the divided area 1-1 held in the buffer 131-1 as described above to the buffer 131-2 corresponding to the second sub encoding unit 132-2. To copy and hold. Accordingly, the first area dividing unit 120-1 needs to individually transfer the data of the divided area 1-1 to each of the first sub-encoding unit 132-1 and the second sub-encoding unit 132-2. Will disappear. That is, the operation of transferring the data in the divided area 1-1 may be performed only once regardless of the number of patterns of the encoding amount.

  In addition, the first area dividing unit 120-1 transfers the data of the divided area 1-2 to the second encoding unit 130-2. The second encoding unit 130-2 also stores and holds the transferred data in the buffer 131-3 corresponding to the third sub-encoding unit 132-3, and then stores the data in the buffer 131-3. The held data is copied and held in the buffer 131-4 corresponding to the fourth sub-encoding unit 132-4. As a result, the first region dividing unit 120-1 may transfer the data of the divided region n-2 only once regardless of the number of patterns of the encoding amount.

  As described above, in this embodiment, the area dividing unit 120 can perform the transfer operation for holding the data of the divided area in the subsequent buffer only once. As a result, the data transfer amount does not increase, and for example, the processing load can be reduced and the encoding process can be further speeded up.

[Example of encoding procedure]
The flowchart in FIG. 7 illustrates an example of a processing procedure executed by the video encoding device 100 according to the second embodiment. This figure shows a procedure for processing one frame at the processing timing shown in FIG.

  In FIG. 7, the processing of steps S201 to S205 is the same as that of steps S101 to S105 of FIG. In step S206, the Nth region dividing unit 120-N transfers the data of the divided region n-1 to the first encoding unit 130-1. In response to this, in step S207, the first encoding unit 130-1 transfers the data transferred from the N-th region dividing unit 120-N to the buffer 131-1 corresponding to the first sub-encoding unit 132-1. Store and hold. In the same step S207, the first encoding unit 130-1 copies the data stored in the buffer 131-1 as described above to the buffer 131-2 corresponding to the second sub-encoding unit 132-2. Hold.

  Next, in step S208, the Nth region dividing unit 120-N transfers the data of the divided region n-2 to the second encoding unit 130-2. In response to this, in step S209, the second encoding unit 130-2 stores and holds the transferred data in the buffer 131-3 corresponding to the third sub-encoding unit 132-3. Also, the second encoding unit 130-2 copies and stores the data stored in the buffer 131-3 to the buffer 131-4 corresponding to the fourth sub encoding unit 132-4. Subsequent steps S210 to S215 are the same as steps S110 to S115 in FIG.

  With the processing procedure of FIG. 7, the data transfer of the divided areas n-1 and n-2 by the Nth area dividing unit 120-N is performed once each at steps S206 and S208. That is, the transfer of one divided region by the region dividing unit 120 is performed once regardless of the number of encoding patterns.

<Third Embodiment>
[Configuration of video encoding device]
FIG. 8 illustrates a configuration example of the video encoding device 100 according to the third embodiment. In this figure, the same parts as those in FIG. 1 and FIG.

  The video encoding device 100 illustrated in FIG. 8 includes a first encoding unit 130-1 and a second encoding unit 130-2 as a configuration corresponding to the encoding process. Moreover, the 1st encoding part 130-1 is equipped with the 1st sub-encoding part 132-1 and the 2nd sub-encoding part 132-2 in the inside, and the 2nd encoding part 130-2 is the inside. In addition, a third sub-encoding unit 132-3 and a second sub-encoding unit 132-3 are provided. This is the same as in FIG.

  However, the first encoding unit 130-1 in FIG. 8 includes only one buffer 131A-1, and the first sub encoding unit 132-1 and the second sub encoding unit 132-2 are both buffers 131A. Input data from -1. Similarly, the second encoding unit 130-2 includes only one buffer 131A-2, and the third sub encoding unit 132-3 and the fourth sub encoding unit 132-4 receive data from the buffer 131A-2. Enter. That is, in the third embodiment, the buffer 131A in one encoding unit 130 is shared by a plurality of sub encoding units 130.

[Example of encoding processing timing]
The timing chart of FIG. 9 shows an example of the execution timing of the encoding process executed by the video encoding device 100 as the third embodiment shown in FIG. In this figure, description of processing that is the same as in FIGS. 3 and 5 is omitted.

  The first area dividing unit 120-1 transfers the data of the divided areas 1-1 and 102 generated by the area dividing process to the buffer 131A-1 in the first encoding unit 130-1, respectively. The data is transferred to the buffer 131A-2 in the conversion unit 130-2. Thereby, the buffer 131A-1 holds the data of the divided area n-1, and the buffer 131A-2 holds the data of the divided area n-2.

  Then, the first sub-encoding unit 132-1 in the first encoding unit 130-1 receives the data of the divided region n-1 from the buffer 131A-1 and performs an encoding process corresponding to the first encoding amount. Execute. Also, the second sub-encoding unit 132-2 in the same first encoding unit 130-1 also inputs the data of the divided region n-1 from the buffer 131A-1 and encodes corresponding to the second encoding amount Execute the process.

  In the second encoding unit 130-2, the third sub-encoding unit 132-3 inputs the data of the divided region n-2 from the buffer 131A-2 and performs encoding corresponding to the first encoding amount. Execute the process. In addition, the fourth sub-encoding unit 132-4 in the same second encoding unit 130-1 also inputs the data of the divided region n-2 from the buffer 131A-2 and performs encoding corresponding to the second encoding amount. Execute the process. Then, the first to third region dividing units 120-1 to 120-3 sequentially execute the same processing as described above cyclically according to the frame order.

  By processing the data of the divided area as described above, in the third embodiment, it is necessary to perform the process of copying the data of the divided area between the plurality of buffers 131 in one encoding unit 130. Disappears. As a result, the time required for data copying between the buffers 131 is reduced. As a result, the time required for encoding processing as the video encoding device 100 can be further reduced.

[Example of encoding procedure]
FIG. 10 is a flowchart illustrating an example of a processing procedure executed by the video encoding device 100 according to the third embodiment. This figure shows a procedure for processing one frame at the processing timing shown in FIG.

  The processing in steps S301 to S305 in FIG. 10 is the same as that in steps S201 to S205 in FIG. In step S306, the N-th region dividing unit 120-N converts the data of the divided region n-1 among the divided regions n-1 and n-2 generated by the region dividing process into the first encoding unit 130-1. Is transferred to the buffer 131A-1. In step S307, the Nth region dividing unit 120-N transfers the data of the divided region n-2 to the buffer 131A-2 corresponding to the second encoding unit 130-2.

  Steps S308 to S313 are the same as steps S210 to S215 in FIG. However, in the processing of steps S308 and S309, both the first sub-encoding unit 132-1 and the second sub-encoding unit 132-2 use the data of the divided region n-1 held in the buffer 131A-1. input. Similarly, in the processing of steps S310 and S311, both the third sub-encoding unit 132-3 and the fourth sub-encoding unit 132-4 store the data of the divided region n-2 held in the buffer 131A-2. Enter. With this procedure, the divided area data can be transferred to the sub-encoding unit 132 without copying the divided area data between the buffers.

[Another configuration example of the encoding unit]
FIG. 11 shows another configuration example of the part corresponding to the encoding process in the third embodiment. FIG. 11 includes four first to fourth sub-encoding units 132-1 to 132-4 in the first encoding unit 130-1, and four fourth to fifth sub-coding units in the second encoding unit 130-2. An example including 8 sub-encoding units 132-5 to 132-8 is shown.

  In this configuration, the first encoding unit 130-1 and the second encoding unit 130-2 correspond to the divided regions n-1 and n-2, respectively. This corresponds to the case of encoding with 4 patterns of 4 encoding amounts.

  The first encoding unit 130-1 includes four first sub encoding units 132-1 to 132-4 and one buffer 131A- corresponding to these sub encoding units 132 in common. 1 is provided. Similarly, the second encoding unit 130-2 includes four fifth sub-encoding units 132-5 to third sub-encoding unit 132-8 and one corresponding to these sub-encoding units 132 in common. A buffer 131A-2 is provided.

  The data of the first divided area transferred to the first encoding unit 130-1 is held in the buffer 131A-1. Then, the first sub-encoding unit 132-1 to the fourth sub-encoding unit 132-4 receive the data of the first divided area from the buffer 131A-a, and the first encoding amount to the fourth code, respectively. Encoding processing based on the quantization amount is executed.

  In the second encoding unit 130-2, similarly to the above, the data of the second divided area is held by the buffer 131A-2. Then, the fifth sub-encoding unit 132-5 to the eighth sub-encoding unit 132-8 respectively perform encoding processing using the first to fourth encoding amounts.

  In FIG. 8, two sub-encoding units 132 correspond to one buffer 131A. However, the sub-encoding unit 132 sharing the buffer 131A is not limited to this, and as illustrated in FIG. 11 described above, for example, the number of sub-encoding units corresponding to the encoding amount pattern is provided to one buffer 131A. It can be set as the structure which respond | corresponds to an encoding part.

<Parallel number setting of resolution conversion unit, region division unit, and encoding unit>
In the present embodiment, the combination of the parallel numbers of the resolution conversion unit 110, the region division unit 120, and the sub-encoding unit 130 is not limited to one pattern, and can be changed as appropriate. For example, when the video encoding apparatus 100 is realized by a DSP, the setting can be changed as appropriate.

  There are various ways to set the parallel number. The first and second examples will be described with reference to the flowcharts of FIGS. 12A and 12B, respectively. Note that the processing shown in FIGS. 12A and 12B can be regarded as being executed by the control unit 150.

  A first example of FIG. 12A will be described. In FIG. 12A, in step S401, the control unit 150 determines the number of frames i1 that can be subjected to resolution conversion processing and region division processing simultaneously in parallel based on the resource amount of the video encoding device 100. Next, in step S402, the control unit 150 forms i1 resolution conversion units 110 and the same i1 region division units 120.

  Further, in step S403, the control unit 150 determines the number j1 of encoding processing units that can be processed in parallel, based on the resource amount as the video encoding device 100. Next, in step S404, the control unit 150 forms j1 sub-encoding units 130. Accordingly, the first to i1th resolution conversion units 110-1 to 110-i1, the first to i1st region dividing units 120-1 to 120-i1, and the first to j1th sub-encoding units 130-1 to 130- A video encoding device 100 including j1 is formed.

  Next, a second example of FIG. 12B will be described. In the second example, it is assumed that the input video source is divided. When a high-resolution video such as 4K or 8K is input, the video signal may be divided. There are actually 4K cameras that use 4 HD-SDIs and output 4 video sources. In step S501, the control unit 150 obtains the parallel number i2 of the resolution converting unit 110 and the region dividing unit 120 by calculating (r × a) using the input video source number r and a predetermined natural number a. In step S502, i2 resolution conversion units 110 and i2 area division units 120 are formed. Thus, by providing in parallel the number of resolution conversion units 110 by an integer multiple of the number of input sources r, it becomes possible to simultaneously execute resolution conversion processing for the number of frames corresponding to the parallel number, and the number of unit frames The time required for per resolution conversion processing is shortened.

Next, in step S503, the control unit 150 determines the number of divided areas in one frame image based on a predetermined resolution value for each resolution pattern. In this case, the number of divided areas at the resolution n is defined as f _n . Next, in step S504, the control unit 150 uses the number of divided regions f _n and the number of encoded amount patterns g _{n of} resolution n for the parallel number j2 of the sub-encoding unit 130 to Σ (f _n × g _n ). Then, the control unit 150 forms j2 sub-encoding units 130 in step S505. Accordingly, the video encoding apparatus 100 includes the first to i-th resolution conversion units 110-1 to 110-i2, the first to i-th region dividing units 120-1 to 120-i2, and the first to j-th sub-encoding. It has the structure provided with the parts 130-1 to 130-j2.

  Next, a third example of setting the parallel number will be described with reference to the flowchart of FIG. In the third example, the parallel number of the resolution converting unit 110, the region dividing unit 120, and the encoding unit 132 is handled as the number of resources. The resource here corresponds to an arithmetic unit that can be assigned to the resolution conversion unit 110, the region division unit 120, and the encoding unit 132. As will be understood from the following description, depending on the parallel number setting in the third example, resources are dynamically allocated so that the processing time until the input video is compression-encoded and output is minimized. Note that a fixed number of resources are allocated in advance to the control unit 150 and the multiplexing unit 140.

  In step S601, the control unit 150 substitutes “1” for a variable i indicating the number of resources to be allocated to each process of resolution conversion, region division, and encoding as an initial setting. In addition, in step S602, the control unit 150 inputs a predetermined unit of video for prior evaluation, and each resolution conversion unit 110-1 corresponding to the current number of resources (i = 1), area division. The unit 120-1 and the encoding unit 132-1 execute compression encoding processing. If the unit of the video input at this time is, for example, a GOP (Group Of Picture) unit, it may be about 1 GOP to several GOPs.

  Next, in step S603, the control unit 150 calculates a processing time required for one resource for one frame processing for each of the resolution conversion processing, the region division processing, and the encoding processing from the result of the processing in step S602. . The processing time of the resolution conversion process per frame calculated here is expressed as x1, and the processing time of the area division processing per frame is expressed as x2. Also, the processing time of the encoding process of one divided area in one frame is represented as x3.

  The speed of the encoding process per frame is represented as s3.

  The processing time required for encoding processing per frame when the number of resources allocated to encoding (corresponding to the number of encoding processing units 132) is "1" x3 × f). For example, when the number of resources “i” assigned to encoding is assigned, the processing time of the encoding process per frame can be expressed as (x3 × (f / i)).

  This means that the processing time per frame is the shortest (the processing speed is the highest) in the encoding process when the number of divided regions f is set to the number of resources i. . This is because the f divided regions in one frame can be encoded simultaneously by the f encoding units 132 in parallel. However, when the number of resources larger than the number of divided regions f is given, the processing time does not become any faster, but since multiple frames can be processed simultaneously, throughput (processing per unit time is possible). Data amount) can be improved. In the encoding process, the throughput when the number of frames y is encoded at the same time corresponds to, for example, a value represented by (s3 · y). At this time, the number of resources i of the encoding process is (f × y).

  In step S604, the control unit 150 performs processing with the longest processing time or the minimum throughput value that is obtained according to the number of currently allocated resources among the resolution conversion processing, region division processing, and encoding processing. Select as assignment target. That is, the one with the lowest processing capability is selected. In the first step S604 executed subsequent to step S603, for example, when the variable i is “1”, the processing time x1 of the resolution conversion processing per frame obtained in step S602, and The processing time s2 of the area division processing per frame is compared with the processing time (x3 × f) of the encoding processing per frame. Then, the process having the maximum value may be selected.

  Next, in step S605, the control unit 150 increases the number of resources i to be allocated to the process selected as the resource allocation target in step S604 by a set number according to a predetermined processing procedure. The processing procedure in step S605 will be described later. Then, the control unit 150 calculates the processing time or throughput of the resource allocation target process when the number of resources i is increased in step S605. In step S605, the calculated processing time or throughput (processing capability) is calculated. It is determined whether it is lower (minimum) than the other two processes. If the processing time is longer than the other two, the processing capability is the lowest. If the throughput is shorter than the other two, the processing capability is the lowest.

  If it is determined in step S607 that the processing capability is the lowest, the control unit 150 returns to the process of step S605 and further increases the number of resources i. If it is determined in step S607 that the processing capability is not the lowest, the control unit 150 proceeds to step S608.

  In step S608, the control unit 150 determines whether resources that can be allocated for the resolution conversion process, the area division process, and the encoding process remain. If it is determined that resources that can be allocated remain, the process returns to step S604. As a result, as the resource allocation target, a process having the lowest processing capability is selected instead of the processing whose processing capability is no longer the lowest due to the allocation of the number of resources so far. By repeating the processing of steps S604 to S608 in this way, the number of resources given to each of the resolution conversion processing, region division processing, and encoding processing is sequentially increased. Then, when it is determined in step S608 that there are no remaining resources that can be allocated, the setting of the number of resources is completed.

  FIG. 14 shows an example of a processing procedure for increasing the number of resources for the resource allocation target processing in step S605 of FIG. First, in step S701, the control unit 150 determines whether or not the currently selected resource allocation target process is an encoding process. Here, when it is determined that it is not the encoding process, the resource allocation target process is either the resolution conversion process or the area division process. Therefore, in step S702, the control unit 150 increments the resource number i for the resolution conversion process or the area division process that is currently the resource allocation target. That is, the resource to be allocated is increased by one. For example, instead of incrementing, it may be possible to add a constant of 2 or more.

  On the other hand, if it is determined in step S701 that the encoding process is a resource allocation target, the control unit 150 proceeds to step S703. In step S704, the control unit 150 determines whether or not the number of resources i currently allocated to the encoding process is larger than the number of divided regions f. Here, when it is determined that the number of resources i is equal to or less than the number of divided regions f, the control unit 150 increments the number of resources i in step S702. At this stage, since the number of encoding units 132 that can be processed in parallel is equal to or less than the number of divided regions f, the processing speed of the encoding process increases as the number of resources i increases. .

  On the other hand, if it is determined that the number of resources i exceeds the number of divided areas f, the control unit 150 multiplies the divided area f by a predetermined natural number α in step S704. It is determined whether or not (f × α) or more. Here, when it is determined that the remaining number of resources that can be allocated is (f × α) or more, the control unit 150 adds (f × α) to the number of resources i in step S705. That is, when the parallel number of the encoding unit 132 becomes equal to or greater than the divided region f, the processing speed cannot be increased any more, but the throughput is improved by increasing the number of frames that can be processed in parallel. Therefore, in step S705, the resource number i is increased by an integral multiple of the divided region f.

  On the other hand, when it is determined in step S704 that the remaining number of resources that can be allocated is less than (f × α), the control unit 150 proceeds to step S706. In step S706, the control unit 150 adds all the remaining remaining resource numbers β to the resource number i. By performing the processing in this way, resources are distributed so that the most efficient compression encoding processing is performed among the resolution conversion processing, the region division processing, and the encoding processing. Especially for encoding processing, first increase the processing speed (shorten the processing time) and increase the number of resources to increase the throughput when the number of resources becomes the same as the number of divided areas. I can let you.

<Modification Example of Correspondence Relationship between Resolution Conversion Unit and Region Division Unit>
In the configuration of the embodiments so far, the resolution conversion unit 110 and the region division unit 120 are made to correspond to each other in a one-to-one relationship after having the same parallel number. However, a configuration in which two or more region dividing units 120 are associated with one resolution conversion unit 110 may be employed. As an example of such a configuration, FIG. 15 illustrates a configuration in which two of the first A area dividing unit 120A-1 and the first B area dividing unit 120B-1 correspond to the first resolution converting unit 110-1. ing.

  Here, as a specific example, the first resolution conversion unit 110-1 generates data of first to fourth resolution frames corresponding to four patterns of resolution as the resolution conversion processing. Then, the first resolution conversion unit 110-1 transfers and holds the data of the first resolution frame and the second resolution frame among the first to fourth resolution frames, for example, to the buffer 121A-1. In addition, the data of the third resolution frame and the fourth resolution frame are transferred to and held in the buffer 121B-1.

  The first A area dividing unit 120A-1 inputs the data of the first resolution frame and the second resolution frame from the buffer 121A-1, and executes the area dividing process. Further, the first B region dividing unit 120B-1 inputs the data of the third resolution frame and the fourth resolution frame from the buffer 121B-1, and executes the region dividing process. In this way, by adopting a structure in which a plurality of area dividing units 120 are assigned to one resolution converting unit 110, the number of resolution frames input by one area dividing unit 120 can be dispersed. Thereby, for example, the processing load of one region dividing unit 120 can be reduced, and a reduction in processing speed can be prevented.

  Note that at the processing timing shown in FIG. 3 and the like, the resolution conversion processing by the plurality of resolution conversion units 110 and the region division processing by the plurality of region division units 120 are sequentially executed at timings according to the frame period of the input video, respectively. ing. However, the plurality of resolution conversion units 110 can perform resolution conversion processing in parallel, and the plurality of region division units 120 can also perform region division processing in parallel at the same time. Therefore, for example, when the processing time of the resolution conversion process is longer than one frame period, the resolution conversion process can be executed so that the processing periods overlap between the resolution conversion units 110. Similarly, when the processing time is longer than one frame period, the region dividing unit 120 can execute the resolution conversion process so that the processing periods overlap between the region dividing units 120. As described above, under the condition that the processing time of the resolution conversion processing or the region division processing is longer than one frame period, the parallel processing is performed by the resolution conversion unit 110 and the region division unit 120, thereby reducing the processing time. It is done.

  In the embodiments described so far, processing is performed in units of frames. However, the present invention is not limited to this. For example, the processing is performed in units of GOP (Group Of Pictures). The configuration of this embodiment can be applied.

  The processing procedure described in this embodiment can be regarded as a method having a series of these procedures. Further, it can be understood as a program for causing a computer to execute these series of procedures or a recording medium for storing the program. Examples of the recording medium include a Blu-ray Disc (registered trademark), a DVD (Digital Versatile Disk), an HDD (hard disk), a memory card, and the like.

The above-described apparatus has a computer system inside. The process of operation is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer system reading and executing this program. The “computer system” herein includes a CPU, various memories, an OS, and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.

In addition, a program for realizing each step is recorded on a computer-readable recording medium, and a program for realizing this function is recorded on a computer-readable recording medium and recorded on the recording medium. Each process may be performed by causing the computer system to read and execute the program.
The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 110 Resolution conversion part 111, 121, 131, 141 Buffer 120 Area division part 130 Coding part 140 Multiplexing part 150 Control part 200 Frame image 210 Division area 300 File 400 Display object area 500 Display image

Claims (4)

A resolution conversion unit that converts the resolution of the input image according to a predetermined resolution and outputs resolution conversion data in which the resolution is converted;
An area dividing unit that divides the resolution conversion data input from the resolution conversion unit according to a predetermined number of divisions and outputs a plurality of divided data;
An encoding unit that performs encoding according to a plurality of different predetermined encoding rates on the divided data in parallel and outputs encoded data for each encoded rate;
A multiplexing unit that multiplexes the encoded data for each input image;
The resolution conversion process for the input image is executed by the resolution conversion unit, the division process for the resolution conversion data is executed by the region division unit, the encoding process for the divided data is executed by the encoding unit, and the code A control unit that causes the multiplexing unit to execute a multiplexing process on the multiplexed data,
There are a plurality ( n ) of resolution conversion units, and the input image is input to any one of the n resolution conversion units in a time cycle in order of time frame by frame, and each of the n resolution conversion units is Convert all input frames to the same resolution,
There are also n region dividing units, and each of the n region dividing units divides resolution conversion data input from the resolution conversion unit corresponding to one-to-one into the same division number m,
There are m encoding units according to the division number m of the region, and each of the m encoding units handles a plurality of the same encoding rates, and executes a plurality of encodings corresponding to the encoding rates to be handled. Sub-encoding processing units, and the sub-encoding processing unit receives the divided data of the same region from the n region dividing units corresponding to the input order of the input images. There video encoding apparatus characterized by encoding in parallel the coding rate determined it was divided data from the buffer. The buffer of the encoding unit is
The video encoding apparatus according to claim 1, wherein the plurality of sub-encoding processing units are buffers commonly used. A resolution conversion step in which the resolution conversion unit converts the resolution of the input image in accordance with a predetermined resolution, and outputs resolution conversion data in which the resolution is converted;
An area dividing step of dividing the resolution conversion data input from the resolution converting section according to a predetermined number of divisions, and outputting a plurality of divided divided data;
A control step of inputting region information indicating a display target region specified in the input image and controlling the encoding unit to execute an encoding process on the divided data including the display target region;
Code in which the encoding unit performs encoding according to a plurality of different predetermined encoding rates in parallel on the divided data and outputs encoded data for each encoded rate Step,
A multiplexing step for multiplexing the encoded data for each input image;
There are a plurality ( n ) of resolution conversion units, and the input image is input to any one of the n resolution conversion units in a time cycle in order of time frame by frame, and each of the n resolution conversion units is Converting all input frames to the same resolution;
There are also n region dividing units, each of the n region dividing units dividing the resolution conversion data input from the resolution conversion unit corresponding to one-to-one into the same division number m,
There are m encoding units according to the division number m of the region, and each of the m encoding units handles a plurality of the same encoding rates, and executes a plurality of encodings corresponding to the encoding rates to be handled. Sub-encoding processing units, and the sub-encoding processing unit receives the divided data of the same region from the n region dividing units corresponding to the input order of the input images. a step of encoding but in parallel with the coding rate determined we were divided data from the buffer,
A video encoding method comprising: Computer
Resolution conversion means for converting the resolution of the input image according to a predetermined resolution and outputting resolution conversion data in which the resolution is converted;
Area dividing means for dividing the resolution conversion data input from the resolution conversion means according to a predetermined number of divisions, and outputting a plurality of divided data;
Encoding means for executing encoding according to a plurality of different predetermined encoding rates in parallel with respect to the divided data and outputting encoded data for each encoded rate,
Multiplexing means for multiplexing the encoded data for each input image,
According to a predetermined encoding parameter, the resolution conversion means for the input image is executed by the resolution conversion means, the division processing for the resolution conversion data is executed by the area division means, and the encoding processing for the divided data is performed by the area conversion means. Control means for causing the encoding means to execute, and causing the multiplexing means to execute a multiplexing process on the encoded data;
There are a plurality ( n ) of resolution conversion means, and the input image is input to any one of the n resolution conversion means in a time cycle in order of time frame by frame, and is input to each of the n resolution conversion means . Means to convert all input frames to the same resolution,
There are also n area dividing means, and means for dividing the resolution conversion data input from the resolution converting means corresponding to each of the n area dividing means on a one-to-one basis into the same division number m.
The number of the encoding means is m according to the division number m of the region, and each of the m encoding means handles the same plurality of encoding rates, and each executes a plurality of encodings according to the encoding rate to be handled. Sub-encoding processing means, and the divided data of the same area from the n area dividing means is input to the buffer of each encoding means so as to correspond to the input order of the input images, and each sub-encoding processing means means causes the coding in parallel the coding rate determined we were divided data from the buffer to,
Program to function as.

Images