JP2003199057A - Image editing apparatus and method - Google Patents

Abstract

(57) Abstract: Provided is an image editing apparatus and method for simultaneously correcting closed caption (CC) data so as to synchronize with a desired screen when editing an MPEG stream. SOLUTION: A correction information creating unit 8 performs a GO process immediately before a discontinuity point.
In the correction of P, a process of adding or rewriting a display memory erasure code and a non-display memory code to the last part of the CC data is performed. Further, in the correction of the GOP after the discontinuous point, the correction information creating unit 8 changes the CC data to dummy data “00” and performs a picture cutting process until the I picture is reproduced. The number of fields of the last picture of the immediately preceding GOP is adjusted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image editing apparatus and method for editing a compression coded video stream such as an MPEG stream containing closed caption data.

[0002]

2. Description of the Related Art MPEG (Moving Picture)
e Experts Group) is one of the international standards for moving image compression technology standardized by the International Organization for Standardization (ISO), and is a communication medium in that it can compress and reproduce video data at a high compression rate and high quality. In addition to the above, the field of use of storage media has expanded significantly.

In this MPEG, a reference video frame (I picture: Intra-Picture) and a difference frame (B picture: B) from the reference video frame.
Directionally Predictive
-Picture, P picture: Predictive
15 frames (0 ..- Picture), which usually consist of a reference video frame and a difference frame.
5 seconds) is one unit. This group of 15 frames is referred to as GOP (Group of Pict).
ure). This GOP has a sequence header (SH) to enable playback from the middle, and the sequence header contains the initial data and user data required for data playback such as image size and pixel aspect ratio. Contains data. In the MPEG stream, continuous video is created by the connection of the sequence header and GOP. Therefore, editing in GOP units can be easily performed.

DVD-Video standard (DVD Spe
specifications forRead-Only
In Disc Version 1.0), video data is compression-encoded by the above-mentioned MPEG system.
Among the contents stored by using the DVD-Video standard, EIA widely used for TV video signals is included.
Some include closed caption (CC) information in the -608 standard. VBI (Video Blanking) other than the effective image area such as CC information.
Information) is defined in the DVD-Video standard to be recorded in the user data area of the GOP layer in the MPEG data. This DVD-Vid
Regarding the recording / reproducing of CC information in the eo standard, Japanese Patent Laid-Open Nos. 2000-36940 and 2000-9245 are available.
No. 6, Japanese Patent Laid-Open No. 2001-69439, and the like.

[0005]

The MPEG stream is easy to edit in GOP units, but M streams containing CC information.
When editing such as deleting the PEG stream, multiple G
C recorded in the user data area across the OP
A part of C information was deleted and CC information could not be displayed correctly. Therefore, MPE including conventional CC information
In editing the G stream, there is a problem that it is necessary to re-record the CC information so as to synchronize with the desired screen again after the video data is edited.

SUMMARY OF THE INVENTION In view of the above problems, it is an object of the present invention to provide an image editing apparatus and method for simultaneously correcting CC information so as to synchronize it with a desired screen when editing an MPEG stream.

[0007]

An image editing apparatus according to the present invention comprises a correcting means for correcting closed caption data described in a user area of an MPEG stream of a portion to be edited, and a closed caption corrected by the correcting means. Stream creating means for creating a new stream using the caption data.

Further, the image editing method of the present invention corrects the closed caption data described in the user area of the MPEG stream of the part to be edited, and creates a new stream using the corrected closed caption data.

According to the present invention, the CC information can be corrected at the same time when the MPEG stream is edited so as to be synchronized with a desired screen.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

First, the closed caption system will be described.

In the closed caption, the caption signal (character information) is multiplexed on the line 21 in the vertical blanking period of the video signal. With normal caption data,
Caption mode (POPON, PAINT, R
OLLUP), a code indicating the TEXT mode, a display position address and a preamble-A indicating a character color, etc.
A data code representing a character follows the address-Code. In addition to this, display memory erase, non-display memory erase, flip memory code, which controls start / end,
In addition, control codes such as cursor movement, backspace, flash-on, line feed, and deletion between line ends are inserted as necessary. These control codes consist of a 1-byte pair (2 bytes) and are inserted twice for correct reception.

Now, the closed caption code decoder will be described.

FIG. 1 is a block diagram showing the basic structure of a closed caption code decoder. CC code data is input from the input unit 11 to the CC code decoder 12 and decoded. The decoded data is stored in the memory A13 or the memory B14 according to the write memory selection signal.
It is written in either one of. Then, either the data of the memory A13 or the memory B14 is selected by the selector 15 according to the read memory selection signal and displayed on the display.

In the PAINTON mode, the closed caption code decoder writes data in the read memory (display memory). For example, the data is written in the memory A13 and the data is read from the memory A13. Therefore, the decoded data will be displayed immediately.

In the POPON mode, the closed caption code decoder writes data to the memory (non-display memory) which has not been read. For example, data is written to the memory A13 and data is read from the memory B14. Then, the flip-memory code controls the switching between the read / write memories, the data contents written up to that point are displayed at once (the contents of the memory A13 in this example), and the subsequent data writing is performed to the memory B14. Be seen.

FIG. 2 is a diagram for explaining an example of closed caption data.

In the example of FIG. 2, closed caption data is described in the order of the first field and the second field, ABCDE is displayed on the 15th line, and FGHIJKLMNO is displayed on the 15th line after switching. ing. The command "load caption data" is executed by "1420" and "1420". Next, the display start position is 1 by "14 72" and "14 72".
It is determined that there are 5 rows and 4 columns. Next data "4
1 "," 42 "," 43 "," 44 "," 45 "are the letters ABC
Corresponds to DE. The next "142C""142C" erases the display memory, which causes the characters displayed on the display to disappear. Next, an instruction of switching between the memories A and B is executed in "142F" and "142F". Then, the letters "ABCDE" are displayed on the display.

Next, the relationship between the MPEG stream and the closed caption data will be described.

The closed caption system is standardized in the form of being multiplexed on the line 21 of the vertical blanking period of the video signal, but the MPEG standard secures a user data area for describing the information. CC to DVD
When information is recorded, GOP in the MPEG standard
It is standardized to describe line 21 data in the user data area of a layer.

Here, the format of the user data area of the GOP layer in the MPEG standard will be described.

FIG. 3 is a diagram for explaining the format described in the user data area. In FIG.
user_data_start_code is the value 000
001B2h. line2_Indi
"cator" is a fixed value indicating that the data that follows is user data. top_field_fl
ag_of_gop is “1b” when the first display field of the first picture in the GOP is the top field.
In the case of the bottom field, it is described as "0b". num
ber_of_displayed_field_go
p indicates the number of display fields in the GOP, which is the same as the number of line 21 data repeatedly recorded subsequently.

Marker_bits is a fixed value, lin
When e21_switch is "1b", it indicates that the following line 21 data is valid and conforms to the EIA-608 standard. When it is "0b", it indicates that the following line 21 data is invalid. Line 21 contains either field 1 data or field 2 data.

Line21_data () is defined in the format shown in FIG. Figure 4 shows line21_
It is a figure for demonstrating the format of data (). The MSB of line21_data1 is an odd parity bit, and 7 bits represent the data of the character 1 defined by EIA-608. line21_dat
In a2, the MSB is an odd parity bit, and 7 bits represent the data of character 2 defined by EIA-608.

FIG. 5 is a diagram for explaining the relationship between an image, a GOP including the image, and CC data in the GOP layer user data area of the GOP. The example shown in FIG. 5 is G
12 for the image displayed in the 12th frame in OP
The second CC data is represented.

Next, editing of an MPEG stream including CC data will be described with reference to the data shown in FIG.

FIG. 6 shows the structure of CC data when the CC data of FIG. 2 is rewritten for each GOP so as to be described in the user data area. FIG. 7 shows the relationship between each GOP, the CC data (GOP_CC) described in the GOP, and the display period of the CC data.

Here, the editing for deleting GOP3 in FIGS. 6 and 7 is performed. In GOP3_CC, "14 20" which is a control code, "14 20" (caption mode), and "1470" which indicates a display start position are described. If this GOP3_CC is deleted, the edited data will be as shown in FIG.

In GOP4_CC, the caption mode cannot be recognized because it starts in the middle of a series of CC data. Further, since "1470" indicating the display start position is not inserted twice, this cannot be recognized correctly either. Further, as shown in FIG. 2, a preamble indicating a caption mode code, a display position address, a character color, and the like.
Since -Address-Code is not followed by a data code representing a character, intended CC data is not displayed on the playback screen even when the memory is switched.

Further, GOP3, GOP4, GOP1, G
An example of data edited by rearranging in the order of OP2 is shown in FIG. 9, and its configuration is shown in FIG. According to FIG.
GOP3_CC, GOP4 for the reproduced image of GOP1
The _CC data is displayed, and the CC data that does not match the reproduced image at all is displayed.

Therefore, the present invention corrects the CC information at the same time when the MPEG stream is edited so that the CC information is synchronized with the desired screen.

FIG. 11 is a block diagram showing the arrangement of an image editing apparatus for editing an MPEG stream containing CC data according to an embodiment of the present invention.

The image editing apparatus uses the file 1 in which the MPEG stream (1) of the editing source is stored, the MP of the editing source as well.
File 2 in which EG stream (2) is stored, edit processing device 3, and MP newly created by edit processing device 3
File 4 for storing EG stream and playback device 5
Composed by.

The edit processing device 3 includes an MPEG stream analysis unit 6, an edit list creation unit 7, a correction information creation unit 8 and an MPE.
The G stream creating unit 9 is provided. The MPEG stream analysis unit 6 stores the M files stored in the edit source files 1 and 2.
Information on a series of screen groups having the same attributes such as the image size and image rate of the PEG stream, the presence / absence of user data, and GOP time code are acquired. The edit list creating unit 7 creates an edit list by selecting an arbitrary GOP according to the GOP time code obtained by the MPEG stream analyzing unit 6. The correction information creation unit 8 performs a correction process to obtain CC data corresponding to the image to be displayed based on the edit list created by the edit list creation unit 7. The MPEG stream creating unit 9 creates a new MPEG stream based on the edit list created by the edit list creating unit 7 and the CC data corrected by the correction information creating unit 8. Playback device 5 is MP
Display the data generated by the EG stream creation unit 9,
Enables confirmation of display data.

Next, the operation of the image editing apparatus configured as described above will be performed by M stored in the editing source files 1 and 2.
An example in which a new MPEG stream 4 is created by the edit processing device 3 using the PEG streams (1) and (2) will be described.

First, the MPEG stream analysis unit 6 analyzes the MPEG streams (1) and (2) stored in the edit source files 1 and 2, and a screen group having a series of the same attributes such as image size and image rate. Information, bit rate, confirmation of CC data described in the user area, and GOP time code are acquired. The edit list creation unit 7 creates an edit list from the GOP time code obtained by the MPEG stream analysis unit 6. FIG. 12 shows an example of an edit list created by selecting an arbitrary GOP from the MPEG streams (1) and (2). FIG. 13 is a block diagram showing the editing example described with reference to FIG. In FIG. 13, an MPEG stream (3) created by selecting an arbitrary GOP has a structure of 2 GOPs in the order of data (1), (2), and (1). Note that points (a) and (b) in FIG. 13 indicate discontinuous points.

The correction information creating unit 8 determines the discontinuity point (a) based on the result of the edit list created by the edit list creating unit 7.
(B) The CC data described in the user data area of the preceding and following GOPs is corrected.

The correction information creating section 8 will be described in detail below. The CC data caption mode is the POPON mode generally used in movies and the like.

First, the correction of the GOP immediately before the discontinuity will be described. The CC data currently described in the user data area of this GOP indicates that the next flip memory code is confirmed and is displayed on the display by switching the memory. If the flip memory code is not described in this GOP, it will be displayed in the next or subsequent GOP. Currently, the next GOP is an image with completely different content due to editing. Therefore, the correction information creation unit 8 performs a process of adding or rewriting the display memory erasing code and the non-display memory code to the last part of the CC data.

Next, GO after the discontinuity points (a) and (b)
The correction of P will be described. As shown in FIG. 2, the CC data is a preamble-address that represents a caption mode code, a display position address, a character color, and the like.
A data code representing a character follows the s-Code. Therefore, if the CC data described in the user data area is not described from the caption mode code, it will not be correctly displayed on the display even if the memory is switched. Therefore, the correction information creation unit 8 changes the CC data to dummy data "00".

Further, as the GOP unit editing, the GOP after the discontinuity cannot be reproduced unless it starts from the I picture which can be reproduced independently. Therefore, the correction information creation unit 8 performs a picture cutting process until the I picture is reproduced. Further, the correction information creation unit 8 sets the number of fields of the last picture of the GOP immediately before the discontinuity points (a) and (b) so that no problem occurs in the connection with the GOP immediately before the discontinuity point due to this clipping processing. The adjustment process is also performed at the same time.

Here, the processing of discontinuity points will be specifically described using the CC data of FIG.

As shown in FIG. 14, GOP1 and GOP
Cut GOP3 out of 2, GOP3, GOP4
Create new MPEG streams OP1, GOP2, and GOP4.

First, in GOP4 immediately after the discontinuity point,
The number of fields in a picture that cannot be played back is acquired because encoding processing is not performed again. At the same time, the processing of the joint with GOP2 immediately before the discontinuity point is performed.

In this example, GOP4 is a reproduction unit BBIB.
BPBBPBBPBBP cannot be reproduced as it is because the correlation before and after each frame is lost. Therefore, as a method of processing discontinuity points without re-encoding, a method of cutting out two B pictures reproduced before an independently reproducible I picture is adopted. Then, as shown in FIG. 5, the CC data corresponding to the two B pictures 1 and 2 to be cut are also cut and displayed at the same time. Next, CC of GOP4
Since the contents of the data are described from the middle and cannot be displayed correctly, they are all replaced with dummy data "00".

Next, the processing of GOP2 immediately before the discontinuity will be described. If you add the deletion code of the data currently displayed on the display and the data displayed next,
It becomes like 5. In FIG. 15, "14 2E""14
2E "is a non-display memory erase code," 14 2C "" 14
2C "is a display memory erasing code. With this display memory erasing code, the memory A13 and the memory B14 in FIG.
All the data read in both memories will be erased. Then, the number of fields of the last picture of GOP2 is adjusted so that GOP2 and GOP4 can be connected without any problem.

The CC data correction information generated by the correction information creating section 8 is supplied to the MPEG stream creating section 9 in the format shown in FIG. The MPEG stream creating unit 9 creates a new MPEG stream from the MPEG streams (1) and (2) based on the edited CC data created by the correction information creating unit 8 and the edit list created by the edit list creating unit 7. To do. The reproduction device 5 confirms reproduction of the MPEG stream thus generated.

According to the image editing apparatus of this embodiment, even if CC data is described in the user data area,
Smooth CC data can be obtained without displaying CC data inconsistent with the playback screen at the editing point. Also,
It is not necessary to input the CC data again to prevent the display of inconsistent data, and the working time can be suppressed.

However, in order not to display the inconsistent data, dummy data "00" is added to the GOP after the discontinuity point.
By replacing it with, there is a period in which it is not displayed on the screen. Therefore, a method for shortening the non-display period and enabling a smoother CC data display will be described below.

FIG. 17 is a block diagram showing the configuration of another embodiment of the present invention. 17, the same components as those in FIG. 11 are designated by the same reference numerals and the description thereof will be omitted. In the present embodiment, the correction information creating method of the correction information creating unit 10 is different from that of the first embodiment.

A correction information creating method in the correction information creating unit 10 of the second embodiment will be described with reference to FIGS. 17 and 18. Note that FIG. 18 is a diagram for explaining the editing method of the second embodiment.

First, in GOP4 immediately after the discontinuity point,
The number of fields in a picture that cannot be played back is acquired because encoding processing is not performed again. At the same time, the processing of the joint with GOP2 immediately before the discontinuity point is performed. In this example, the first
In the same manner as in the first embodiment, GOP4 is a reproduction unit BBIBBP
Since the correlation before and after each frame of BBPBBPBBP is lost, two B pictures reproduced before the I picture are cut out. Then, as shown in FIG.
The CC data corresponding to is also cut and the display time is adjusted.

Next, GOP4 cannot be displayed accurately because the contents of CC data are described from the middle. Therefore, the CC data content of GOP2 immediately before the discontinuity point is checked. Since no meaningful CC data exists in GOP2_CC, the content of GOP3_CC is rewritten to GOP2_CC. At this time, GOP4
_CC is not corrected.

Next, the processing of GOP2 will be described. As described above, since there is no meaningful CC data in GOP2_CC data, the contents of GOP3_CC are
Rewrite to OP2_CC. At that time, so that CC data that is inconsistent with the playback screen after the discontinuity is not displayed,
A display memory erase code is added to the end of the GOP2_CC data. At the same time, the number of fields of the last picture of GOP2 is adjusted so that GOP2 and GOP4 can be connected without any problem. FIG. 19 shows CC data after correction by the correction information creating unit 10. The CC data correction information generated by the correction information creation unit 10 is supplied to the MPEG stream creation unit 9 in the format shown in FIG.

The MPEG stream creating section 9 uses the edited CC data created by the correction information creating section 10 and the edit list created by the edit list creating section 7 to create a new MPEG from the MPEG streams (1) and (2). Create a stream. The MPEG stream generated in this way is supplied to the reproduction device 5 and reproduction confirmation is performed.

As described above, according to the image editing apparatus of the present embodiment, even if the CC data is described in the user data area, the CC data is smoothly displayed without displaying the CC data inconsistent with the reproduction screen at the editing point. Can be played. In addition, it is not necessary to input the CC data again in order to prevent the display of inconsistent data, and the working time can be suppressed. Furthermore, it is possible to shorten the period in which CC data is not displayed as compared with the first embodiment and obtain smoother CC data.

In the first and second embodiments, G
The case where the CC data is described in the user data area of the OP layer has been described, but the present invention can be similarly processed even when the CC data is described in the user data area of another layer. Further, in the first and second embodiments, the editing unit is the GOP unit, but the present invention can similarly process the editing in the picture unit or in the other GOP unit.

[0058]

According to the present invention, at the time of editing an MPEG stream, the CC information can be corrected at the same time so as to be synchronized with a desired screen.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of a closed caption code decoder.

FIG. 2 is a diagram illustrating an example of closed caption data.

FIG. 3 is a diagram for explaining a format described in a user data area.

FIG. 4 is a diagram for explaining a format of line21_data ().

FIG. 5 is an image, a GOP including the image, and the GO.
The figure for demonstrating the relationship with CC data of the GOP layer user data area of P.

FIG. 6 is a diagram for explaining a structure of CC data when the CC data is rewritten for each GOP so as to be described in a user data area.

FIG. 7 is a diagram showing a relationship between each GOP, CC data (GOP_CC) described in the GOP, and a display period of the CC data.

FIG. 8 is a diagram for explaining edited data for deleting GOP3_CC.

FIG. 9 is a diagram for explaining data when a GOP rearrangement edit is performed.

FIG. 10 is a diagram for explaining a data structure when GOP rearrangement editing is performed.

FIG. 11 is a block diagram showing the configuration of an image editing apparatus.

FIG. 12 is a diagram for explaining an example of an edit list created by selecting an arbitrary GOP from MPEG streams (1) and (2).

FIG. 13 is a diagram for explaining an example of editing created by selecting an arbitrary GOP from MPEG streams (1) and (2).

FIG. 14: GOP3 is cut, GOP1, GOP
FIG. 2 is a diagram illustrating a configuration when a new MPEG stream called GOP4 is created.

FIG. 15 is a diagram for explaining CC data when a deletion code of data currently displayed on the display and an erasing code of data displayed next is added to the CC data of GOP2 immediately before the discontinuity point.

FIG. 16 is a diagram showing generated CC data correction information.

FIG. 17 is a block diagram showing the configuration of another embodiment.

FIG. 18 is a diagram for explaining an editing method according to the second embodiment.

FIG. 19 is a diagram showing CC data after correction by a correction information creating unit.

[Explanation of symbols]

1, 2, 4 files 3 Editing processor 5 playback device 6 MPEG stream analysis unit 7 Editing list creation section 8, 10 Correction information creation unit 9 MPEG stream creation unit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/035 F term (reference) 5C023 AA18 AA40 BA17 CA02 CA05 DA04 EA02 5C063 AB03 AB07 AC01 AC10 CA11 CA23 CA29 CA40 DA03 DA13 DB02 5D044 AB07 BC01 CC05 DE12 DE55 EF03 EF05 FG21 GK12 5D110 AA13 AA29 BB01 BB20 CA05 CB04 CD02 CD04 CD07 CF24 CK26 DE01

Claims (6)

[Claims] 1. An M including closed caption data.
In a PEG stream editing apparatus, a correction means for correcting closed caption data described in a user area of an MPEG stream of a portion to be edited, and a stream for creating a new stream using the closed caption data corrected by the correction means An image editing apparatus comprising: a creating unit. 2. The correction means analyzes a series of closed caption data in the vicinity of the edited portion when the division or partial deletion is performed by editing, and based on the result, analyzes all of the series of closed caption data. The image editing apparatus according to claim 1, wherein the recording position and the control code of a series of closed caption data are corrected unintentionally. 3. The correction of the recording position is performed by taking out significant closed caption data superimposed on an MPEG stream portion deleted by editing and superimposing it on the edited stream. Image editing device. 4. An M including closed caption data.
A method of editing a PEG stream, characterized in that closed caption data described in a user area of an MPEG stream of a portion to be edited is corrected, and a new stream is created using the corrected closed caption data. . 5. When division or partial deletion is performed by editing, a series of closed caption data in the vicinity of the edited portion is analyzed, and based on the result, all of the series of closed caption data is made insignificant, or a series of closed caption data. 5. The image editing method according to claim 4, wherein the recording position of the caption data and the control code are corrected, and the data is edited to be significant even in the edited MPEG stream. 6. The series of closed caption data is corrected by extracting the significant closed caption data superimposed on the MPEG stream portion cut by editing and superimposing it on the edited MPEG stream. 6. The image editing method according to claim 5, wherein the recording position is corrected.