JP2004266850A - Motion picture treatment apparatus and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that if motion pictures with caption is compressed in a compression system based on quantization in a frequency region where white of high luminance is used in many cases in order to heighten perceptivity in caption indication of motion pictures, picture quality of a total picture deteriorates compared with a case where there is no caption since mosquito noises tend to occur around characters, and bits concentrate to character parts. <P>SOLUTION: The motion picture data are compressed in a motion picture compression circuit 113, composed with caption text data in a multiplexer 114, and sent to a user through a storage/transmission system 115. Received data by the user are separated to compressed picture data and caption data by a demultiplexer 116. The picture data are sent to a motion picture expansion circuit 117 through a signal line 120. The text data are sent to a caption superimposer 118 through a signal line 121, deployed to a bitmap text picture there, and superimposed on the motion picture expanded by a motion picture expansion circuit 117. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to a moving image processing apparatus and method, and more particularly to a moving image processing for editing a moving image such as superimposition.

  Conventionally, when editing, compressing, and transmitting digital moving image data, it is common to edit the moving image data and then compress the processed moving image data. Here, "editing" means, for example, superimposing a character or figure on a moving image, or applying a transition effect described below to the moving image.

  The transition effect is a process of fading out a moving image that is temporally backward while fading out a moving image that is temporally forward, for example, in order to smoothly connect two temporally consecutive moving image sequences. That is. In this case, in the range specified in the transition period, an image in a special state in which two moving image sequences overlap appears.

  The edited moving image data thus obtained is encoded by various proposed moving image encoding methods and transmitted to the receiving side through a communication line. On the receiving side, the received encoded data is decoded and displayed.

  FIG. 2 is a flowchart showing a conventional moving image editing procedure. FIG. 2 (a) shows the procedure of the transmitting terminal, and FIG. 2 (b) shows the procedure of the receiving terminal.

  In FIG. 7A, in step S1, the transmitting terminal inputs moving image data and stores it in a memory. Subsequently, the transmitting terminal receives the designation of an effect such as a transition effect in step S2, and edits the image data stored in the memory based on the designation received in step S2 in step S3.

  Next, in FIG. 6B, the receiving terminal receives the code data in step S6. Subsequently, in step S7, the receiving terminal decodes the received code data and stores the decoded moving image data in the memory. Subsequently, the receiving terminal displays a moving image on the display in step S8.

JP-A-4-103271 JP-A-60-79883 JP-A-2-123881 JP-A-1-286682 JP-A-1-232573 JP-A-4-256296

  Usually, for displaying subtitles of moving images such as television and movies, high brightness white is often used to enhance visibility, but compression methods such as MPEG based on quantization in the frequency domain When a moving image with a subtitle is compressed, mosquito noise is likely to be generated around the character, and the bits are concentrated in the character portion, so that the image quality of the entire image is deteriorated as compared with the case without the subtitle.

  There are also many problems in terms of user operability. For example, the caption position is usually fixed at the lower part of the screen, and the user cannot arbitrarily switch the display position. Even when the caption is unnecessary, it cannot be switched to non-display.

  Furthermore, there is a problem that it is not possible to watch a moving image such as a movie while switching between Japanese and English subtitles. If two types of subtitles are required, the user needs to obtain two moving image data that differ only in the subtitle part, while the producer also depends on the sales area (for example, Japanese and English) I had to work on different versions, which was wasteful.

  SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problem, and to prevent deterioration in image quality due to superimposition of text data on moving image data and to enable setting of superimposition conditions of text data on a receiving side. I do.

  The present invention has the following configuration as one means for achieving the above object.

  According to the present invention, moving image data in which the text data is not superimposed is subjected to frequency domain conversion and compression encoded video data, and information data in which text data is multiplexed is received from an external device, Separating the encoded video data and the text data from the received information data, decoding the separated encoded video data, and superimposing the separated text data on the decoded video data; Receiving the setting of the superimposition condition of the text data, superimposing the text data on the moving image data based on the superimposition condition, and outputting the moving image data on which the text data is superimposed. .

  According to the present invention, by superimposing text data on moving image data on the receiving side, it is possible to prevent deterioration in image quality caused by superimposing text data on moving image data on the transmitting side, and to reduce the text data on the receiving side. It is possible to set the superimposition condition.

  Hereinafter, moving image processing according to an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of a system that processes a moving image by the moving image processing method according to the first embodiment of the present invention. The moving image transmission system shown in FIG. 1 includes a plurality of terminals a1, b2, c3,... Connected by a communication line, and the terminal a1 has the following configuration.

  The moving image transmission system shown in FIG. 1 includes a plurality of terminals a1, b2, c3,... Connected by a communication line, and the terminal a1 has the following configuration.

  Reference numeral 11 denotes an input port which receives moving image data from an image input device 19 such as a video camera. Reference numeral 12 denotes a CPU that executes processing of image data and control of the entire terminal 1a via the bus 18 according to a control program stored in the ROM 14. Further, the CPU 12 can read a pixel value at an arbitrary coordinate of an arbitrary frame of the moving image data stored in the RAM 13 by accessing a predetermined area of the RAM 13.

  Reference numeral 15 denotes an encoding / decoding circuit, 16 denotes a display, and 17 denotes an interface, which mediates data communication with the communication line 101. The above components are interconnected by a bus 18. The other terminals b2, c3,... Have substantially the same configuration as the terminal a1.

  FIG. 3 is a flowchart illustrating an example of a moving image editing procedure according to the present embodiment. FIG. 3 (a) illustrates the procedure of the transmitting terminal, and FIG.

  In FIG. 1A, the transmitting terminal of the present embodiment reads moving image data into the RAM 13 via the input port 11 in step S11. Subsequently, in step S12, the transmitting terminal accepts designation of a plurality of effects such as superimposition and transition effects (fade-out, wipe, etc.).

  Subsequently, in step S13, the transmitting terminal transmits the effect designation data (command data) received in step S12 to the communication line 101 via the interface 17. Subsequently, the transmitting terminal encodes the moving image data by the encoding / decoding circuit 15 in step S14, and sends the encoded data to the communication line 101 via the interface 17 in step S15. Note that the moving image data may be encoded in advance, added with the effect designation data, and transmitted to the communication line 101.

  That is, in the present embodiment, the effect designation data and the encoded unedited moving image data are transmitted.

  Next, in FIG. 3 (b), the receiving terminal receives the effect designation data through the interface 17 and stores it in the RAM 13 in step S16. Subsequently, the receiving terminal receives the encoded data of the moving image data through the interface 17 and stores it in the RAM 13 in step S17. Subsequently, the receiving terminal uses the encoding / decoding circuit 15 to decode the code data stored in the RAM 13 and store the decoded moving image data in the RAM 13 in step S18. Subsequently, the receiving terminal edits the moving image data stored in the RAM 13 based on the effect specifying data stored in the RAM 13 in step S19.

  Editing is performed, for example, as follows. Now, assume that the rate of the received moving image is 5 frames per second, and the displayable rate of the display 16 is 20 frames per second. At this time, when a command for performing fade-out is received, interpolating between frames of a moving image of 5 frames per second, and generating moving image data of 20 frames per second, a fade-out process is performed (see FIG. 6A). Here, the simplest interpolation processing may be to simply repeat the same screen for four frames.

  According to the conventional method, the display image is as shown in FIG. 6B. In addition to the fade-out, for example, a transition effect such as a wipe can be performed in the same manner as described above.

  Subsequently, the receiving terminal displays a moving image on the display 16 in step S20.

  That is, in the present embodiment, after receiving the effect specifying data and the moving image data, the receiving terminal edits the moving image data obtained by decoding the code data based on the effect specifying data.

  In the above-mentioned conventional example, for example, when attempting to superimpose a character or the like on a moving image, image data on which line drawing data (that is, a character) is superimposed is coded. According to the known moving picture coding method (for example, the MPEG coding method), it is not possible to efficiently compress the image in order to improve the image quality, and the code data is increased. On the other hand, if the compression rate is increased, the character becomes difficult to read. Remarkable deterioration of image quality, such as In addition, in the conventional example, when moving image data is transmitted at a low frame rate of about 10 frames per second, if the moving image is faded out, the change of the image to be faded out has to be expressed at a low frame rate, and it is jerky. Only an awkward effect was obtained.

  On the other hand, according to the present embodiment, the superimposition of characters is performed immediately before the image is displayed on the receiving terminal, so that the compression efficiency does not decrease and the image quality does not deteriorate. Further, according to the present embodiment, when performing the fade-out of the moving image, the receiving terminal performs the above-described interpolation processing, so that only the changed portion of the image to be faded out has a higher frame rate than the other portions. Therefore, a smooth fade-out can be obtained without deteriorating the transmission efficiency of the image data.

  In the above description and the drawings, an example in which encoding / decoding of a moving image is performed by the encoding / decoding circuit 15 has been described, but the present embodiment is not limited to this. Thus, encoding / decoding of a moving image may be realized by software processing. In this case, the processing speed is reduced, but the cost of the terminal can be reduced.

  Further, in the above description and drawings, an example has been described in which the moving image data input through the input port 11 or the code data received through the interface 17 is stored in the RAM 13 as it is, but this embodiment is not limited to this. For example, a sufficiently high-speed encoding / decoding circuit is provided, and encoding is performed in real time while moving image data is input, or decoded in real time while receiving encoded data, and stored in the RAM 13. May be. In this case, the processing speed can be improved, and in the former case, the memory capacity of the RAM 13 can be reduced.

  Further, in the above description and drawings, an example is described in which a RAM is used for storing data, but the present embodiment is not limited to this. For example, when high-speed processing is not required, a hard disk or the like may be used. May be used. In this case, a large amount of moving image data can be processed at low cost.

  As described above, according to the present embodiment, by editing moving image data at the receiving terminal, high-quality moving image transmission can be performed without causing a decrease in compression efficiency or image quality due to encoding / decoding. realizable.

  The terminal according to the present embodiment may be a system including a plurality of devices, for example, a system such as a video camera and a host computer, or a device including a single device, for example, a host computer that stores moving images. It may be. Also. The processing of the present embodiment may be realized by supplying a program stored in a medium to a system or an apparatus.

  As described above, according to the first embodiment of the present invention, it is possible to provide a moving image processing method that transmits editing information of moving image data, encodes the moving image data, and transmits the moving image data efficiently. Further, according to the first embodiment of the present invention, it is possible to provide a moving image processing method for editing moving image data obtained by decoding received code data based on received editing information.

  In this embodiment, the moving image processing method of the first embodiment described above is applied to a technique of superimposing text on an image in synchronization with a moving image, such as a movie subtitle or a karaoke lyrics with a moving image. Things.

  In the present embodiment, text data such as subtitles is separately stored, and text such as subtitles is inserted into a bitmap during reproduction and superimposed on an image, thereby inserting text such as subtitles. This prevents image quality degradation at the time of image compression / decompression due to this, and also solves the above-described operability problem.

  Hereinafter, a second embodiment according to the present invention will be described in detail with reference to the drawings.

  FIG. 4 is a configuration diagram of a moving image processing system in the present embodiment. 111 is a subtitle text data memory for storing subtitle text data, 112 is a moving image data memory for storing moving image data, 113 is a moving image compression circuit, 114 is a multiplexer, 115 is a storage / transmission system, 116 is a demultiplexer, 117 Is a moving picture decompression circuit, 118 is a subtitle superimposer, 119 is a display, and 120 and 121 are signal lines.

  The moving image data stored in the moving image data memory 112 is compressed by the moving image compression circuit 113, combined with the text data stored in the subtitle text data memory 111 by the multiplexer 114, and sent to the storage / transmission system 115. . Here, the storage / transmission system is a package system such as a tape or a CD-ROM, or a communication system such as an ISDN or a TV broadcast, through which data is delivered to a user.

  The data received by the user is separated by the demultiplexer 116 into compressed image data and text data. The image data is sent to the moving image expansion circuit 117 through the signal line 120. The text data is sent to the subtitle superimposer 118 via the signal line 121, where it is developed into a bitmap text image and superimposed on the moving image expanded by the moving image expansion circuit 117. The subtitled moving image thus obtained is displayed on the display 119.

  Note that a moving image may be hard-copied for each frame by using an image forming apparatus (laser beam printer or the like) instead of the display. It should be noted that in this embodiment, the subtitle superimposer is That is. Then, the user sets the display / non-display of the subtitles, the display color, the display position, the size of the characters, and the like by using the operation panel 122, and based on the settings, the control circuit 123 controls the superimposition by the subtitle superimposer 118. Control.

  In the above-described conventional transmission method, the caption is always displayed at a fixed position because the caption superimposer is on the transmitting side (the creator of the moving image). On the other hand, according to the present embodiment, the user can freely set subtitle display / non-display, display color, display position, character size, and the like by giving an instruction to the subtitle superimposer. Can be greatly improved. Further, according to the present embodiment, the relationship between the image quality and the compression efficiency as described above can be improved.

  Further, as the moving image compression circuit 113 shown in FIG. 4, an MPEG encoder which is an international standard moving image compression method can be used. Since a "user data area" is defined in the MPEG bit stream, and arbitrary data can be written for each picture based on the MPEG, the multiplexer 114 writes a text in the "user data area". An operation of writing data is performed. The demultiplexer 116 reads out text data from the “user data area” of the MPEG bit stream and sends it to the subtitle superimposer 118. An MPEG decoder is used for the moving image decompression circuit 117.

  According to the present embodiment, there is an advantage that code data compliant with MPEG which is a standard moving image compression method can be generated.

  If the data is played back by a normal MPEG decoder instead of the playback system described in this embodiment, the data in the user data area is ignored, and the subtitles cannot be displayed. Dripping.

  As described above, according to the second embodiment of the present invention, it is possible to avoid image quality deterioration due to superimposition of text on an image, and to allow a user to specify a subtitle display method. Can be selected.

  Hereinafter, a third embodiment according to the present invention will be described in detail with reference to the drawings.

  FIG. 5 is a configuration diagram of the moving image processing system in the present embodiment. 131, 132, and 133 are subtitle text data memories for storing subtitle text data, 134 is a moving image data memory, 135 is a moving image compression circuit, 136 is a multiplexer, 137 is a storage and transmission system, 138 is a demultiplexer, and 139 is A moving image decompression circuit, 140 is a subtitle superimposer, 141 is a display, and 142 is a changeover switch.

  The caption text data memories 131, 132, and 133 contain different types of text data. For example, the first subtitle text data memory 131 contains Japanese subtitles, the second subtitle text data memory 132 contains English subtitles, and the third subtitle text data memory 133 contains Chinese subtitles. These three types of text data are superimposed on the image data compressed from the moving image data memory 134 via the moving image compression circuit 135 by the multiplexer 136 and sent to the storage and transmission system 137.

  On the receiving side, the data passes through a demultiplexer 138 and is separated into compressed image data and three types of text data. The compressed image data is expanded by the moving image expansion circuit 139 and sent to the subtitle superimposer 160. The switch 142 selects any one of the text data. In this embodiment, the user can select a desired one from trilingual captions. The text data selected here is developed into a bitmap by the superimposer 140, superimposed on a moving image, and displayed on the display 141.

  With the above-described configuration, the user can select and display any one of a plurality of types of subtitles.

  Also, here, the type of subtitle text is three, but it is needless to say that the present invention is not limited to this. In addition, it is needless to say that there is no need to prepare a multilingual language, and that texts having different contents in the same language may be inserted.

  Instead of developing a text into a bitmap and superimposing the text on an image, the text may be read out by a voice synthesizer, and mixed with (or replaced with) voice accompanying a moving image and reproduced.

  The present invention may be applied to a system including a plurality of devices or to an apparatus including a single device. Also. It is needless to say that the present embodiment can be applied to a case where the present invention is achieved by supplying a program to a system or an apparatus.

Block diagram showing a configuration example of a system for processing a moving image by the moving image processing of the first embodiment according to the present invention, Flowchart showing a conventional moving image editing procedure, Flow chart showing a moving image editing procedure of the first embodiment according to the present invention, Block diagram showing a configuration example of a second embodiment according to the present invention, Block diagram showing a configuration example of a third embodiment according to the present invention, FIG. 7 is a diagram illustrating an example of a fade-out according to the first embodiment. FIG. 9 is a diagram illustrating fade-out according to a conventional example.

Claims (2)

A receiving means for receiving, from an external device, moving image encoded data obtained by compressing and encoding the moving image data in which the text data is not superimposed by frequency domain and text data multiplexed,
Separating means for separating the moving image encoded data and the text data from the information data received by the receiving means,
Decoding means for decoding the moving picture encoded data separated by the separation means,
Setting means for setting a superimposition condition of text data when superimposing the text data separated by the separation means on the moving image data decoded by the decoding means;
Superimposing means for superimposing the text data on the moving image data based on the superimposing condition;
Output means for outputting moving image data on which the text data is superimposed. Moving image data in which text data on which text data is not superimposed is subjected to frequency domain conversion and compression-encoded moving image encoded data, and information data in which text data is multiplexed is received from an external device,
Separating the moving image encoded data and the text data from the received information data,
Decoding the separated moving image encoded data,
When superimposing the separated text data on the decoded moving image data, accepting the setting of the superimposition condition of the text data,
Superimposing the text data on the moving image data based on the superimposition condition,
A moving image processing method, comprising outputting moving image data on which the text data is superimposed.

Images