JPH06225219A - Method and device for processing moving image - Google Patents

Abstract

PURPOSE:To prevent the reduction in compression ratio and the deterioration in image quality in encoding/decoding by providing a reception means, a decoding means, a processing means and an image forming means and transmitting digital moving image data and command data through a common transmission line. CONSTITUTION:A transmission-side terminal transmits effect designation data (command data) to a communication line 101 through an interface 17. Then, the transmission-side terminal encodes moving image data in an encoding/ decoding circuit 15 and transmits code data to the communication line 101 through the interface 17. A reception--side terminal receives effect designation data by the interface 17 stores it in RAM 13, receives code data of moving image data and stores it in RAM 13. Then, the reception-side terminal decodes code data stored in RAM 13 by the encoding/decoding circuit 15 and stores obtained image data in RAM 13. Then, the reception-side terminal edits moving image data stored in RAM 13 based on the effect designation data stored in RAM 13.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing method and apparatus, and more particularly to a moving image processing method and apparatus for editing moving images, for example, superimposing and transition effects.

[0002]

2. Description of the Related Art Conventionally, digital moving image data has been edited,
In the case of compression and transmission, it is general to edit the moving image data and then compress the processed moving image data. Here, the editing means, for example, superimposing characters or figures on the moving image, or applying a transition effect described below to the moving image.

The transition effect is, for example, to smoothly connect two moving image sequences that are temporally continuous, while fading out a moving image that is ahead in time, fades a moving image that is behind in time. It is a process to put in. In this case, in the range specified in the transition period, an image in a special state in which the two moving image sequences overlap each other appears.

The edited moving image data thus obtained is encoded by various proposed moving image encoding systems and transmitted to the receiving side through the communication line. On the receiving side, the received coded data is decoded and displayed. 2A and 2B are flowcharts showing a conventional moving image editing procedure. FIG. 2A shows the procedure of the transmitting terminal, and FIG. 2B shows the procedure of the receiving terminal.

In FIG. 1A, the transmitting terminal inputs the moving image data and stores it in the memory in step S1. Then, the transmitting terminal accepts the designation of the effect such as the transition effect in step S2, and then in step S2.
In step 3, the image data stored in the memory is edited based on the designation accepted in step S2.

Next, in FIG. 2B, the receiving side terminal receives the code data in step S6. Then, in step S7, the receiving terminal decodes the received code data and stores the decoded moving image data in the memory. Then, the receiving side terminal displays the moving image on the display in step S8.

[0007]

However, the above-mentioned conventional example has the following problems. That is, in the above conventional example, the edited image data subjected to the transition effect or the like is encoded, but in the existing moving image encoding method that performs compression by utilizing the correlation of the adjacent frames, the transition effect or the like is generated. When the applied special image data is compressed, the frame correlation collapses, making it difficult to perform efficient compression.

In some moving picture coding systems, in order to achieve a high compression rate, the frame rate may be thinned out to about 10 frames per second for coding and transmission. However, it was difficult to realize a smooth transition effect, and the transition of the image was jerky. On the other hand, normally, for displaying subtitles of moving images such as television and movies, white with high brightness is often used to improve visibility, but compression methods such as MPEG based on quantization in the frequency domain are used. , When compressing such a video with subtitles, mosquito noise is likely to occur around the characters, and since the bits are concentrated in the text part, the image quality of the entire image deteriorates compared to when there are no subtitles. there were.

Further, there are many problems in terms of user operability. For example, the subtitle position is usually fixed at the bottom of the screen, and the user cannot switch the display position arbitrarily,
Even if subtitles were not needed, it was not possible to switch this to invisible. Further, there is a problem that it is not possible to watch moving images such as movies while switching between Japanese and English subtitles. When two types of subtitles are required, the user needs to obtain two moving image data that differ only in the subtitle part, and on the other hand, the creator side also depends on the sales area (for example, Japanese area and English area). Therefore, it was necessary to create different editions, which was wasteful.

[0010]

SUMMARY OF THE INVENTION The present invention is intended to solve the above problems, and has the following structure as one means for solving the above problems. That is, receiving means for receiving encoded digital moving image data and command data for instructing editing processing to be applied to the digital moving image data, and decoding means for decoding the encoded digital moving image data. A processing means for editing the digital moving image data decoded by the decoding means using the command data, and an image forming means for forming an image according to the digital moving image data processed by the processing means. The digital moving image data and the command data are transmitted via a common transmission path.

[0011]

According to the above structure, the encoded digital moving image data transmitted through the common transmission line and the command data for instructing the editing process to be applied to the digital moving image data are received, A moving image processing method and apparatus for decoding encoded digital moving image data and forming an image in accordance with the decoded and decoded digital moving image data using command data can be provided. For example, a compression rate in encoding and decoding is provided. Can be prevented and deterioration of image quality can be prevented. In addition, the transition effect efficiently,
Moreover, it can be realized with a good image quality. Also, the superimpose function can be enhanced.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A moving image processing apparatus according to an embodiment of the present invention will be described in detail below with reference to the drawings.

[0013]

[First Embodiment] FIG. 1 is a block diagram showing an example of the configuration of a system for processing a moving image by the moving image processing method according to the first embodiment of the present invention according to the first embodiment of the present invention.
The moving image transmission system shown in the figure is composed of a plurality of terminals a1, terminals b2, c3, ... Connected by communication lines, and the terminal a1 has the following structure.

An input port 11 receives moving image data from an image input device 19 such as a video camera. 12 is CP
U is a unit for processing image data and controlling the entire terminal 1a via the bus 18 in accordance with a control program stored in the ROM 14. Also, the CPU 12 has a RAM 13
By accessing the predetermined area of, the pixel value at the arbitrary coordinates of the arbitrary frame of the moving image data stored in the RAM 13 can be read.

Reference numeral 15 is an encoding / decoding circuit, 16 is a display, 17 is an interface, and the communication line 101
Mediate data communication with. The above components are connected to each other by a bus 18. Further, the other terminals b2, c3, ... Have substantially the same configuration as the terminal a1. 3A and 3B are flowcharts showing an example of the moving image editing procedure of the present embodiment. FIG. 3A shows the procedure of the transmitting side terminal, and FIG. 3B shows the procedure of the receiving side terminal.

In FIG. 1A, the transmission side terminal of this embodiment reads the moving image data into the RAM 13 via the input port 11 in step S11. Then, in step S12, the transmitting terminal accepts designation of a plurality of effects such as superimpose and transition effects (fade out, wipe, etc.).

Then, the transmitting side terminal performs step S13.
Then, the effect designation data (command data) accepted in step S12 is transmitted via the interface 17 to the communication line 1
Send to 01. Then, the transmitting terminal, step S14
Then, the moving image data is encoded by the encoding / decoding circuit 15, and the encoded data is sent to the communication line 101 via the interface 17 in step S15. The moving image data may be encoded in advance, the effect designation data may be added, and the data may be transmitted to the communication line 101.

That is, in this embodiment, the effect designation data and the encoded unedited moving image data are transmitted. Next, in FIG. 3B, the receiving side terminal receives the effect designation data through the interface 17 and stores it in the RAM 13 in step S16. Subsequently, the receiving terminal receives the coded data of the moving image data by the interface 17 and stores it in the RAM 13 in step S17.

Subsequently, the receiving side terminal makes a step S18.
Then, the encoding / decoding circuit 15 decodes the code data stored in the RAM 13, and the moving image data obtained by the decoding is decoded.
Store in RAM13. Subsequently, in step S19, the receiving side terminal edits the moving image data stored in the RAM 13 based on the effect designation data stored in the RAM 13.

Editing is performed as follows, for example. now,
It is assumed that the rate of the received moving image is 5 frames per second and the displayable rate of the display 16 is 20 frames per second. At this time, if a command for fading out is received, the frames of the moving image of 5 frames per second are interpolated and
After creating moving image data of 20 frames, fade-out processing is performed (see FIG. 6A). Here, as the interpolation processing, the simplest is to simply repeat the same screen for four frames.

According to the conventional method, the display image is as shown in FIG. 6B. In addition to fade-out, transition effects such as wipe can be performed in the same manner as described above. Then, the receiving terminal
In step S20, the moving image is displayed on the display 16.

That is, in this embodiment, after the effect designation data and the moving image data are received at the receiving side terminal,
The moving image data obtained by decoding the coded data is edited based on the effect designation data. In the above conventional example,
For example, when trying to superimpose characters or the like on a moving image, image data in which line drawing data (that is, characters) is superimposed is encoded, and a known moving image code mainly for compressing natural images is used. Format (eg MPEG
Encoding method), if the image quality is improved, the coded data cannot be efficiently compressed and the code data is increased. On the other hand, if the compression rate is increased, the characters become difficult to read and the image quality is significantly deteriorated. Also, in the conventional example,
When moving image data is transmitted at a low frame rate of about 10 frames, if the moving image is faded out, the changes in the faded out image have to be expressed at a low frame rate, and only a jerky and awkward effect can be obtained.

On the other hand, according to this embodiment, since the superimposing of characters is performed immediately before the image is displayed on the receiving side terminal, the compression efficiency is not deteriorated and the image quality is not deteriorated. Further, according to the present embodiment, when fading out a moving image, the receiving side terminal,
By performing the above-described interpolation processing, it is possible to easily raise the frame rate only in the changed portion of the image to be faded out than in other portions, so that a smooth fade-out can be obtained without degrading the transmission efficiency of image data. You can

In the above description and drawings, an example in which the encoding / decoding circuit 15 encodes / decodes a moving image has been described, but the present embodiment is not limited to this. The CPU 12 may implement encoding / decoding of a moving image by software processing. In this case, the processing speed is reduced, but the cost of the terminal can be reduced.

Further, in the above description and the drawings, the example in which the moving image data input by the input port 11 or the code data received by the interface 17 is stored as it is in the RAM 13 has been described. However, the present invention is not limited to this, and for example, a sufficiently high-speed encoding / decoding circuit is provided to perform encoding in real time while inputting moving image data, or decoding in real time while receiving encoded data to be stored in the RAM 13. May be stored. In this case, the processing speed can be improved, and in the former case, the memory capacity of the RAM 13 can be reduced.

Further, in the above description and the drawings, the example in which the RAM is used for storing data has been described, but the present embodiment is not limited to this, and for example, when high-speed processing is not required. An external storage device such as a hard disk may be used. In this case, moving image data having a large amount of data can be processed at low cost. As described above, according to the present embodiment, by editing moving image data at the receiving terminal, high-quality moving image transmission can be performed without causing deterioration in compression efficiency and image quality due to encoding / decoding. realizable.

The terminal of this embodiment may be a system composed of a plurality of devices, for example, a system such as a video camera or a host computer, or a device composed of one device, such as a moving image memory. It may be a host computer. Also. The processing of this embodiment may be realized by supplying a program stored in a medium to a system or an apparatus.

As described above, according to the first embodiment of the present invention, it is possible to provide a moving image processing method for transmitting edit information of moving image data, encoding the moving image data, and efficiently transmitting the encoded moving image data. Further, according to the first embodiment of the present invention, it is possible to provide the moving image processing method for editing the moving image data obtained by decoding the received code data based on the received editing information.

[0029]

[Second Embodiment] This embodiment uses the moving image processing method of the first embodiment described above to superimpose text on an image in synchronization with the moving image, such as movie subtitles or karaoke lyrics with moving images. It is applied to the technique of posing. In the present embodiment, moving image data and text data such as subtitles are held separately, and when reproducing, text such as subtitles is expanded into a bitmap and superimposed on the image,
The image quality deterioration at the time of image compression / expansion due to the insertion of text such as subtitles is prevented, and at the same time, the operability problem described above is solved.

The second embodiment of the present invention will be described in detail below with reference to the drawings. FIG. 4 is a block diagram of the moving image processing system in this embodiment. Reference numeral 111 is a subtitle text data memory for storing subtitle text data, and 1
12 is a moving image data memory for storing moving image data, 1
13 is a moving picture compression circuit, 114 is a multiplexer, 11
5 is a storage / transmission system, 116 is a demultiplexer, 117
Is a moving picture expansion circuit, 118 is a subtitle superimposer, 1
Reference numeral 19 is a display, and 120 and 121 are signal lines.

The moving image data stored in the moving image data memory 112 is compressed by the moving image compression circuit 113, and is combined with the text data stored in the subtitle text data memory 111 by the multiplexer 114 to be stored / transmitted by the storage / transmission system 115. Sent to. Here, the storage / transmission system is a package medium such as tape or CD-ROM, or a communication system such as ISDN or TV broadcasting, through which data is delivered to the user.

The data received by the user is separated into compressed image data and text data by the demultiplexer 116. The image data is sent to the moving image expansion circuit 117 through the signal line 120. The text data is sent to the subtitle superimposer 118 through the signal line 121,
Here, it is developed into a bitmap text image and superimposed on the moving image expanded by the moving image expansion circuit 17. The moving image with subtitles thus obtained is displayed on the display device 119.

An image forming apparatus (laser beam printer or the like) may be used in place of the display to make a hard copy of a moving image frame by frame.
This means that the subtitle super imposer is on the user side. Then, the user sets the display / non-display of subtitles, the display color, the display position, the size of characters, etc. by the operation panel 122, and the control circuit 123 makes the superimpose by the subtitle superimposer 118 based on the settings. To control.

In the above-described conventional transmission method, since the subtitle superimposer is on the transmission side (the creator of the moving image), the subtitles are always displayed at a fixed position. On the other hand, according to the present embodiment, the user gives an instruction to the subtitle superimposer to display / hide the subtitle, display color,
The display position and the size of characters can be freely set, and operability can be greatly improved. Further, according to this embodiment, it is possible to improve the relationship between the image quality and the compression efficiency as described above.

Further, the moving picture compression circuit 113 shown in FIG.
An MPEG encoder, which is an international standard moving image compression method, can be used as the. A “user data area” is defined in the MPEG bit stream, and arbitrary data can be written for each picture based on MPEG. Therefore, in the multiplexer 114, text is written in this “user data area”. The operation of writing data is performed. The demultiplexer 116 reads the text data from the “user data area” of the MPEG bit stream and sends it to the subtitle superimposer 118. An MPEG decoder is used for the moving picture expansion circuit 117.

According to the present embodiment, there is an advantage that code data conforming to MPEG which is a standard moving image compression method can be generated. When the data is played in a normal MPEG decoder instead of the playback system described in this embodiment, the data in the user data area is ignored, the subtitles cannot be displayed, and the moving image can be played back correctly. Be drunk

As described above, the second aspect of the present invention
According to the embodiment, it is possible to avoid image quality deterioration caused by superimposing text on an image, and a user can arbitrarily select a subtitle display method.

[0038]

[Third Embodiment] The third embodiment of the present invention will now be described in detail with reference to the drawings. FIG. 5 is a block diagram of the moving image processing system in this embodiment. 131,13
2, 133 are caption text data memories for storing caption text data, 134 is a moving image data memory, 135 is a moving image compression circuit, 136 is a multiplexer, 137 is an accumulation and transmission system, 138 is a demultiplexer, 139 is a moving image. A decompression circuit, 140 is a subtitle superimposer, 141 is a display, and 142 is a changeover switch.

Subtitle text data memories 131, 132,
133 contains different types of text data. For example, the first subtitle text data memory 13
1 for Japanese subtitles, 2nd subtitle text data memory 13
2 has English subtitles, 3rd subtitle text data memory 133
Shall have Chinese subtitles. These three types of text data are image data compressed from the moving image data memory 134 through the moving image compression circuit 135, and
The data is superposed by the multiplexer 136 and accumulated and transmitted 13
Sent to 7.

On the receiving side, the data is sent to the demultiplexer 13
8, the compressed image data and three types of text data are separated. The compressed image data is decompressed by the moving image decompression circuit 139 and sent to the subtitle superimposer 160. The switch 142 selects any one of the text data. In this embodiment, the user can select a desired subtitle from three languages. The text data selected here is developed into a bitmap by the superimposer 140, superimposed on the moving image, and displayed on the display 141.

With the configuration described above, the user can select and display any one of a plurality of types of subtitles. In addition, although there are three types of subtitle text here, it is not limited to this, and any number of lines can be inserted as long as the capacity and transfer capability of the storage and transmission systems allow. Needless to say, it is not necessary to prepare multiple languages, and texts of different contents in the same language may be inserted.

Instead of expanding the text into a bitmap and superimposing the text on the image, the text may be read aloud by a voice synthesizer, mixed with the voice accompanying the moving image (or replaced with the voice) and reproduced. The present invention may be applied to a system including a plurality of devices or an apparatus including a single device. Also,
It goes without saying that the present invention can also be applied to the case where it is achieved by supplying a program to a system or an apparatus.

[0043]

As described above, according to the present invention, the encoded digital moving image data transmitted through the common transmission path and the command data for instructing the editing processing to be applied to the digital moving image data are received. Then, it is possible to provide a moving image processing method and apparatus for decoding encoded digital moving image data and forming an image in accordance with the decoded digital moving image data that has been decoded and edited using command data. This has the effect of preventing a reduction in the compression rate and deterioration of the image quality. In addition, the transition effect can be effectively realized with good image quality, which is effective. It also has the effect of enhancing the superimpose function.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of a system for processing a moving image by a moving image processing method according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a conventional moving image editing procedure.

FIG. 3 is a flowchart showing a moving image editing procedure according to the first embodiment of the present invention.

FIG. 4 is a block diagram showing a configuration example of a second embodiment according to the present invention.

FIG. 5 is a block diagram showing a configuration example of a third embodiment according to the present invention.

FIG. 6A is a diagram illustrating an example of fade-out according to the first embodiment.

FIG. 6B is a diagram illustrating fade-out according to a conventional example.

[Explanation of symbols]

 11 Input Port 12 CPU 15 Encoding / Decoding Circuit 16 Display 17 Interface 111 Subtitle Text Data Memory 112 Moving Image Data Memory 113 Moving Image Compressing Circuit 114 Multiplexer 115 Storage / Transmission System 116 Demultiplexer 117 Moving Image Decompression Circuit 118 Moving Image Decompression Circuit 118 Subtitle Superin Poser 119 Display 131-133 Subtitle text data memory 134 Moving picture data memory 135 Moving picture compression circuit 136 Multiplexer 137 Storage and transmission system 138 Demultiplexer 139 Moving picture decompression circuit 140 Subtitle superimposer 141 Display 142 142 Changeover switch

Claims (8)

[Claims] 1. Receiving means for receiving encoded digital moving image data and command data for instructing an editing process to be applied to the digital moving image data, and decoding the encoded digital moving image data. Decoding means, processing means for editing the digital moving image data decoded by the decoding means using the command data, and image forming means for forming an image in accordance with the digital moving image data processed by the processing means. A moving image processing apparatus, comprising: the digital moving image data and the command data being transmitted via a common transmission path. 2. The moving image processing apparatus according to claim 1, wherein a frame rate of the moving image formed by the image forming unit is higher than a frame rate of the moving image received by the receiving unit. 3. The moving image processing apparatus according to claim 2, wherein the processing means performs frame interpolation processing. 4. The moving image processing apparatus according to claim 3, wherein the processing means executes a transition effect on the moving image subjected to frame interpolation. 5. The moving image processing apparatus according to claim 1, wherein the processing means executes a superimpose. 6. The moving image processing apparatus according to claim 5, wherein the command data includes a plurality of types of text data. 7. The moving image processing apparatus according to claim 6, further comprising selection means for selecting one of the plurality of types of text data to be superimposed. 8. A receiving step of receiving encoded digital moving image data and command data for instructing an editing process to be performed on the digital moving image data, and decoding the encoded digital moving image data. A decoding step, a processing step of editing the digital moving image data decoded in the decoding step using the command data, and an image forming step of forming an image according to the digital moving image data processed in the processing step. The moving image processing method, wherein the digital moving image data and the command data are transmitted via a common transmission path.