JP4978582B2 - Stream creation device - Google Patents

Description

  The present invention relates to an apparatus for creating an arbitrary stream according to a standard after editing a part of a video stream including bit data defined by the standard such as MPEG.

  In the standard of moving picture compression such as MPEG, an image is divided into rectangular areas such as 16 pixels × 16 pixels, and pixel values are compressed in units of the rectangles. This rectangle is called a macroblock, and the x and y coordinates in the entire image are attached. For example, the fourth macro from the left, the top macroblock is represented as MB (3,0). It is common. The macroblock compression method uses the Intra method that compresses pixel values using only the information of the image currently being processed, and searches for similar parts from temporally adjacent images. These are roughly divided into two methods: a relative position difference (motion vector: MV) and a pixel value difference, which are found, and a similar position. The Inter method varies from 1-MV (16x16MV), which attaches one motion vector to the entire macroblock, to 16-MV (4x4MV), which divides it into 16 pixels of 4 x 4 pixels and attaches motion vectors to each. A number of motion vectors can be attached.

  The data written in the stream is not a motion vector as it is, but a variable length encoded difference (difference vector: DMV) with a prediction vector (PMV) obtained based on information of surrounding macroblocks. It is. The difference between the pixel value of the image or the pixel value obtained by the Inter method is divided into blocks of 8 pixels x 8 pixels, and the block coefficient is 8 pixels x 8 pixels, or 8 pixels x 4 pixels, or 4 pixels x 8 pixels. Alternatively, orthogonal conversion is performed in units of 4 pixels × 4 pixels, converted into frequency components, quantization is performed by dividing by a quantization coefficient, and scanning is performed to convert it to one dimension to obtain a coefficient (coef) sequence. Due to the nature of the standard, this coefficient is almost zero, so the number of non-zero coefficients, the number of consecutive coefficient values 0 before the corresponding coefficient (run), the value of the corresponding coefficient (level), and the corresponding coefficient is a block or A set of whether or not (last) is the last coefficient of the divided sub-block is obtained, and this set is variable-length encoded and written to the stream. Also, information on whether all the coefficients in each block are 0 or at least one coefficient other than 0 exists is collected for one macroblock and written in the stream as CodedBlockPattern (CBP). Information on whether all coefficients in a macroblock are 0 or at least one coefficient other than 0 exists may be indicated in the stream as (Coded or not Coded). In some cases, the Coded or not Coded information, the Intra or Inter type, and the information on the number of motion vectors are collectively written in the stream as a macroblock type (MBType). As described above, in a moving image compression standard such as MPEG, syntax elements such as difference vector information and coefficients are variable-length encoded and written in a stream in a compressed state.

  When developing a moving picture compression / decompression apparatus that complies with standards such as MPEG, verification data that covers all the syntax elements defined by the standard is required. This verification data may be provided by a standardization organization, but sufficient verification data necessary for development is not always available. Therefore, a part of the existing stream (bit stream) is edited to create a stream suitable for the verification purpose. At this time, since it is difficult to edit the compressed binary data with a general-purpose binary editor, the original stream is decoded, converted into highly readable data for each syntax element, and the converted data is verified. In general, the data is edited so as to meet the intention, and the data is returned to the stream data again to create the intended stream.

FIG. 8 is a block configuration diagram of the stream analysis apparatus, and FIG. 9 is a processing flow of the conventional stream analysis apparatus.
In general, when developing a compression / decompression device conforming to a standard, a stream analysis device having a function of outputting a stream analysis result using text data or a dedicated GUI is created. FIG. 8 shows a configuration of a portion for outputting the stream analysis result of the decoder. FIG. 9 shows a flow of a conventional stream analysis apparatus.

  First, the next syntax determining unit 11 determines the syntax element to be read out next determined by the standard from the value of the syntax element read in the past, and transmits it to the stream reading unit 12. The stream reading unit 12 reads data corresponding to the information of the specified syntax element from the stream, refers to the variable length code table 10 as necessary, and returns the read result to the next syntax determining unit 11. Based on the returned result, the next syntax determination unit further determines a syntax element to be read next. This is repeated and the stream is decoded. At this time, the result read by the stream reading unit is passed to the analysis result output unit 13 and the analysis result is output. For this reason, the analysis result is output only for the syntax element in which data exists in the standard.

  In FIG. 8, when a stream is input to the stream reading unit 12, the stream reading unit 12 extracts data corresponding to the type of the next syntax obtained in the next syntax determining unit 11 from a part of the stream and changes the data. With reference to the long code table 10, the stream which is a variable length code is analyzed, and the syntax corresponding to the analyzed stream is output. The determined syntax is passed to the analysis result output unit 13 and output as an analysis result. As the output format, the syntax may be output as it is, or may be output as a string of symbols that is easier for humans to understand.

  In FIG. 9, in step S10, the type of the next syntax is specified, in step S11, the data of the specified type of stream is read, and in step S12, the analysis result is output. In step S10, when the next syntax of the specified type should not exist, steps S11 and S12 are skipped, and the next syntax type is specified.

  Patent Document 1 discloses information that enables easy creation of MHEG scenes in various display modes by making it possible to manage and handle shared objects by simple editing operations when creating MHEG (Multimedia Hypermedia Information Coding Experts Group) content. A processing device is disclosed.

Patent Document 2 discloses a digital broadcast data analysis / editing apparatus that can easily and efficiently generate test digital broadcast data by directly editing existing digital broadcast data.
JP 2000-278601 A JP 2002-290999 A

  However, depending on the type of data, there may be an effect on the presence or absence of subsequent data. With conventional stream creation techniques, it is not known until the stream is decoded whether the stream after editing is in violation of the standard. In order to conform to the standard, it was necessary for the editor to repeat decoding and editing until there were no parts that violated the standard. For this reason, it has been difficult to edit highly dependent data from subsequent data.

10 to 14 are diagrams for explaining conventional problems.
For example, there is an original stream compliant with the VC-1 standard as shown in FIG.
If you want to generate a stream with the (2,3) DMV (differential motion vector) changed from (-10,4) to (30,20), in order to comply with the standard, the block following the edited MB position The presence / absence of HybridPred may change and needs to be reflected. Here, HybridPred is data indicating the following information. That is, when obtaining an MV (motion vector) for a macroblock being decoded, a predicted motion vector (PMV) of the macroblock is obtained from the MVs of the surrounding macroblocks and added to the DMV of the macroblock. At this time, it is determined whether or not the difference between the obtained PMV and the surrounding MV is larger than the threshold. If so, the PMV is discarded, and the surrounding MV is used instead of the PMV, and the MV of the macro block is Ask for. At this time, HybridPred indicates which macroblock of surrounding macroblocks is used instead of PMV. Therefore, when the DMV of the surrounding macroblock is changed, the PMV of the macroblock changes, and HybridPred is necessary when it is not necessary or when it is not necessary. In this way, depending on the change of the surrounding macroblock stream, HybridPred may or may not exist for the macroblock to be decoded later, and even if it exists, the value of HybridPred may differ. . This is because the data that affects subsequent data (such as the macroblock type or DMV) will change the affected data (HybridPred) due to the change. The reason why it is not possible to know how the change of the affected data will affect the affected data, and it is necessary to repeat editing and decoding in the past. It has become.

FIG. 11 is a configuration example of a conventional stream editing apparatus, and FIG. 12 is a processing flow of the conventional stream editing apparatus.
In FIG. 11, the stream analysis device 15 analyzes a stream and converts it into syntax. The editing is performed by the user using the editing device 16, and a stream after editing is generated using the stream generating device 17 from the edited syntax.

  In FIG. 12, the original stream is input in step S15, the stream is converted into syntax by the stream analyzer in step S16, and the analysis result is output in step S17. In step S18, the user edits the analysis result, and in step S19, the edited analysis result (edited syntax) is obtained. In step S20, an edited stream is obtained in step S21 by creating a stream from the edited syntax with the stream creation device.

  In the conventional method having the flow shown in FIG. 12 configured as shown in FIG. 11, the analysis result (edited by editing the DM (2) of the MB (2,3) in the output analysis result (FIG. 13) ( From FIG. 14A), even if the stream is generated again, the presence or absence of HybridPred of the subsequent block is not considered.

  Therefore, when the HybridPred of MB (10, 6) must be lost as in this example, the generated stream in FIG. 14B is in violation of the standard. Therefore, in order to prevent such a violation of the standard, the stream creation device requires a function to automatically determine and delete the syntax violation, and such a function is required to create the stream. In order to incorporate it into a device, it is necessary to incorporate a decoder having a function of removing a violation of the standard from a stream once created, and the decoding function becomes complicated.

  An object of the present invention is to provide a stream creation device that realizes an arbitrary stream that does not violate the standard even after editing with a simple configuration.

  The stream creation device of the present invention reads data from a stream composed of bits obtained by encoding image data according to a standard, analyzes the stream, and outputs the analysis result in a readable manner. Is not present in the standard, the first stream analyzing means for outputting dummy data and outputting the analysis result, and the stream after editing the analysis result of the first stream analyzing means and converting it into a stream, The second stream analyzing means for deleting the dummy data and outputting the analysis result when the data corresponding to the dummy data does not exist in the standard by changing the data affecting the subsequent data. And stream creation means for creating a stream from the analysis results of the first and second stream analysis means. That.

  The first stream analysis means outputs an analysis result including dummy data. When this is edited, dummy data is left. When this is converted into a stream and the stream is analyzed in accordance with the standard by the second stream analysis means, data may or may not be necessary where dummy data exists. If the data is necessary, the dummy data is left as it is, and if it is not necessary, the dummy data is deleted. Thereby, the analysis result which arranged the data according to a standard can be obtained. When dummy data is present, the image data is incorrect, but the syntax arrangement is correct and conforms to the standard. By making this a stream, an edited stream according to the standard can be obtained.

  According to the present invention, it is possible to provide a stream creation device that realizes an arbitrary stream that does not violate the standard even after editing with a simple configuration.

  In this embodiment, it is possible to create a stream that conforms to the standard without adding a complicated decoding function to the stream creation tool by slightly modifying the existing decoder and temporarily creating data that violates the standard. And

1-7 is a figure explaining embodiment of this invention.
In the present embodiment, the following two functions are newly provided.
-The next syntax determining unit can specify a syntax element with a dummy to the stream reading unit even when there is no data according to the standard (even when HybridPred does not exist). To be specified). Also, when a dummy syntax element is instructed, the stream reading unit returns a dummy result to the next syntax determining unit and the analysis result output unit without actually reading data from the stream ( HybridPred is output). The behavior when existing in the standard is not changed. In other words, a stream analysis device 20 is provided that reads data according to the standard and always outputs an analysis result even when the standard data does not exist.
-The next syntax determining unit can specify a syntax element in a dummy manner to the stream reading unit even when no data exists in the standard (even when HybridPred does not exist) (HybridPred can be specified) ). Further, even when a dummy syntax element is instructed, the stream reading unit actually reads data from the stream and returns a dummy result to the next syntax determination unit. In the case of a dummy, no output instruction is issued from the stream reading unit to the analysis result output unit (HybridPred is not output). In other words, a stream analysis device 23 is provided that always reads data even when no data exists according to the standard and outputs the analysis result according to the standard.
1 and 2 show the configuration and flow of the apparatus of this embodiment.

In the present embodiment, first, the original stream (FIG. 10) is applied to the stream analysis device 20, and the stream analysis result (FIG. 5) is output. Here, MBType is a macroblock type, and specifies the type of macroblock encoding method. This data is data that affects subsequent data. The DMV is a differential motion vector, and is a basis for calculating a motion vector of a macro block to be processed by being added to a predicted motion vector (PMV). This data is also data that affects subsequent data. CBP is information indicating whether or not there is a coefficient in the macroblock to be processed later. Block is a part in which coefficient values are collectively described for each block. In this analysis result, HybridPred always exists even when it does not exist in the stream (the second HybridPred: 0 should not exist in the standard, but is output in the analysis result) ).

  The analysis result is edited to match the verification item (FIG. 6A). Here, the value of the first DMV is changed. Next, the edited analysis result is applied to the stream creation device to create an intermediate stream (FIG. 6B). The underlined part is the edited part. Here, regardless of the presence / absence of HybridPred in the standard, there is always Hybrid, so the stream is in violation of the standard. This intermediate stream is further applied to the stream analysis device 23 to create an intermediate analysis result (FIG. 7A). In the stream analysis result 23, the HybridPred is always read from the stream, and is output as the analysis result only when necessary in the standard, so the analysis result conforms to the standard. Here, the second HybridPred: 0 is missing. Finally, the intermediate analysis result is applied to the stream creation device 22 to create a stream (FIG. 7B). The underlined part is the edited part. Since this stream is created from the analysis result conforming to the standard, it conforms to the standard.

As described above, a standard-compliant stream can be created even by editing that affects the presence or absence of the subsequent syntax element in the apparatus of this patent.
In the stream editing apparatus of this embodiment shown in FIG. 1, the stream analysis apparatus 20 analyzes the input stream to make a syntax, and inputs the analysis result to the editing apparatus. At this time, even if HybridPred does not exist, HybridPred is output as a dummy. The user edits the analysis result with the editing device 21 and inputs the result to the stream generation device 22. The edited analysis result is first converted into a stream. This stream is a violation of the standard because it is a stream of syntax that includes HybridPred even in a portion where HybridPred originally does not exist. The stream (intermediate stream) including the standard violation is input to the stream analysis device 23. The stream analysis device 23 analyzes the stream including the violation of the standard according to the standard. Then, it can be seen that HybridPred is also included in the portion that should not have HybridPred. Therefore, in this case, HybridPred is not output, but the deleted version is output as an analysis result. As a result, the analysis result is an analysis result that does not include unnecessary HybridPred and that does not violate the standard. Then, the analysis result is streamed by the stream generation device 22 to obtain an edited stream in accordance with the standard.

FIG. 2 is a processing flow of the stream analysis device 20.
In step S30, the next syntax type is determined from the processing result of the previous stream. If the syntax type is a syntax that originally exists, the stream is read out in step S31, the syntax of the analysis result according to the contents of the stream is output in step S32, and the process returns to step S30. In step S30, if the syntax element indicates that it does not exist in accordance with the standard (HybridPred does not exist), the stream is dummy read in step S33 (actually nothing is read). In step S32, syntax that does not originally exist (HybridPred) is output as an analysis result, and the process returns to step S30. This is done for the entire stream.

FIG. 3 is a processing flow of the stream analysis device 23.
When a stream including a violation of the standard is received, the next syntax type is determined in step S35. When the syntax type is determined from the previous stream processing, the stream is read in step S36, the analysis result is output in step S37, and the process returns to step S35. In step S35, if it is determined from the processing of the previous stream that the syntax element does not exist next, the stream is read (reads dummy HybridPred) in step S38, and nothing is performed. Return to S35.

FIG. 4 is an overall processing flow of the present embodiment.
In step S40, the original stream is input. In step S41, the stream analysis apparatus 20 generates an analysis result so as to include a dummy syntax element. In step S42, the analysis result is output on a computer display or the like. In step S43, the user edits the analysis result, and in step S44, the edited analysis result is input to the stream creation device. In step S45, the stream creation device converts the input analysis result into a stream, and outputs an intermediate stream in step S46. In step S47, the stream analysis device 23 analyzes the intermediate stream, and if a dummy syntax element is found, the intermediate analysis result in step S48 is obtained without outputting as an analysis result. In step S49, the intermediate analysis result is input to the stream creation apparatus and converted into a stream, thereby obtaining an edited stream in accordance with the standard in step S50.

  In this way, dummy syntax elements are included in the first analysis, and after editing, the edited analysis result is converted into a bitstream, and this is analyzed once again. Regardless of (), you can create a bitstream that conforms to the standard. If it is necessary to put another syntax element in the place of the dummy syntax element as a result of editing, the dummy syntax element is left in the second analysis process. The dummy syntax element has a random value as a variable value, so if you look at the resulting stream as a bitstream that encodes an image, the image will be corrupted, but the syntax element is where you originally needed it. Is included, so it does not violate the standard.

  As described above, this embodiment is very convenient for creating a bit stream that does not violate the standard as a data structure, even if it is random as an image as a verification stream.

In addition to the above embodiment, the following supplementary notes are disclosed.
(Appendix 1)
When data is read from a first stream including bits obtained by encoding image data according to a standard, the first stream is analyzed, and an analysis result is output, the data affected by the preceding data does not exist in the standard Includes first stream analysis means for outputting dummy data and outputting an analysis result;
By editing the analysis result of the first stream analyzing means and changing the data affecting the subsequent data in the second stream after being streamed, the data corresponding to the dummy data conforms to the standard. A second stream analysis unit that deletes dummy data and outputs an analysis result when it no longer exists;
Stream creation means for creating a third stream from the analysis results of the first and second stream analysis means;
A stream creation apparatus comprising:
(Appendix 2)
If the data corresponding to the dummy data exists in the standard even if the data affecting the subsequent data is changed, the second stream analyzing means outputs the dummy data as it is as an analysis result. The stream creation device according to Supplementary Note 1, wherein:
(Appendix 3)
The data that affects the subsequent data is data indicating the type of block in a coding unit of an image, or a differential motion vector that is a difference between a motion vector and a predicted motion vector. Stream creation device.
(Appendix 4)
The stream creation device according to attachment 3, wherein the data affected by the preceding data is data instructing to use a motion vector of an adjacent block instead of the predicted motion vector.
(Appendix 5)
The stream creation apparatus according to appendix 1, wherein a standard for encoding the image data is MPEG or VC-1.
(Appendix 6)
The stream creation apparatus according to appendix 1, further comprising editing means for editing the analysis result of the first stream analysis means.
(Appendix 7)
When data is read from a first stream consisting of bits encoded with image data according to a standard, the first stream is analyzed, and the analysis result is output in a readable manner, the data affected by the preceding data is in accordance with the standard. Even if it does not exist, output dummy data and output the first analysis result,
Edit the first analysis result,
Creating a second stream from the edited first analysis result;
When the data corresponding to the dummy data does not exist in the standard by changing the data affecting the subsequent data for the second stream after the first analysis result is streamed, Delete the dummy data and output the second analysis result,
Creating a third stream from the second analysis result;
A stream creation method comprising:

It is an apparatus block diagram of embodiment of this invention. It is a processing flow of the stream analyzer 20 of the embodiment of the present invention. It is a processing flow of the stream analyzer 23 of the embodiment of the present invention. It is the whole processing flow of embodiment of this invention. It is a data example (the 1) explaining embodiment of this invention. It is a data example (the 2) explaining embodiment of this invention. It is a data example (the 3) explaining embodiment of this invention. It is a block block diagram of a stream analyzer. It is a processing flow of the conventional stream analysis apparatus. It is FIG. (1) explaining the conventional problem. It is FIG. (2) explaining the conventional problem. It is FIG. (3) explaining the conventional problem. It is FIG. (4) explaining the conventional problem. It is FIG. (5) explaining the conventional problem.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Variable length code table 11th order syntax determination part 12 Stream reading part 13 Analysis result output part 15 Stream analysis apparatus 16 Editing apparatus 17 Stream generation apparatus 20 Stream analysis apparatus 21 Editing apparatus 22 Stream generation apparatus 23 Stream analysis apparatus

Claims (5)

When data is read from a first stream including bits obtained by encoding image data according to a standard, the first stream is analyzed, and an analysis result is output, the data affected by the preceding data does not exist in the standard Includes first stream analysis means for outputting dummy data and outputting an analysis result;
By editing the analysis result of the first stream analyzing means and changing the data affecting the subsequent data in the second stream after being streamed, the data corresponding to the dummy data conforms to the standard. A second stream analysis unit that deletes dummy data and outputs an analysis result when it no longer exists;
Stream creation means for creating a third stream from the analysis results of the first and second stream analysis means;
A stream creation apparatus comprising:   If the data corresponding to the dummy data exists in the standard even if the data affecting the subsequent data is changed, the second stream analyzing means outputs the dummy data as it is as an analysis result. The stream creation apparatus according to claim 1.   The data that affects the subsequent data is data indicating a type of block of a coding unit of an image, or a differential motion vector that is a difference between a motion vector and a predicted motion vector. The stream creation device described.   4. The stream creation apparatus according to claim 3, wherein the data affected by the preceding data is data instructing to use a motion vector of an adjacent block instead of a predicted motion vector.   The stream creation apparatus according to claim 1, wherein a standard for encoding the image data is MPEG or VC-1.

Images