KR101059779B1 - Character generator and subtitle and graphic input method - Google Patents

Abstract

A user interface for generating subtitles and graphics and storing them in a memory as a data structure; A rendering engine for generating an image based on the data structure; A video decoder for converting a file into an image; A mixer for synthesizing the image generated by the rendering engine and the image converted by the video decoder; A video encoder for converting the image and video synthesized by the mixer into a file; And a thread manager for exchanging data between the tasks by the user interface, the rendering engine, the video decoder, the mixer, and the video encoder, and for ordering the tasks. And a graphic input method.

Character Generator, Render Engine, Mixer, File, Subtitles, Graphics

Description

CHARACTER GENERATOR AND METHOD FOR MAKING CAPTIONS AND GRAPHICS USING THIS}

The present invention relates to a character generator and a caption and graphic input method using the same. More particularly, the present invention includes a rendering engine capable of directly writing captions and graphics into a file output from a camera. Omitted, it is possible to perform the input operation of the subtitles and graphics without a comprehensive editing room, and to the character generator and the subtitles and graphics input method using the same so that the entire process from the first output of the camera to the final transmission can be carried out only by the file.

1 schematically illustrates the workflow of analog content production according to the prior art.

The shape of the photographed image using the conventional camera 10 was a tape 11. The tape 11 was linearly edited using two expensive VCRs 12, and then the edited tape 13 was sent to the general editing room 14 to add subtitles and graphics. The comprehensive editing room 14 is equipped with equipment such as a character generator, a switcher, a VCR, etc. These devices are expensive, and there is a burden of cost because there is always a manpower for operating the comprehensive editing room. The final tape 15 to which subtitles and graphics were added in the general editing room 14 was broadcasted in the delivery main control room VCR 16.

2 schematically illustrates an internal configuration of a general editing room according to the prior art.

Comprehensive editing room is equipped with a lot of equipment for video editing and operating personnel resides. Actually, the internal structure of the comprehensive editing room is very complicated, but the functional part will be described with reference to FIG. 2.

Referring to FIG. 2, the comprehensive editing room includes a VCR 20, a character generator 21, a switcher 22, and a VCR 23. Others include routers (not shown) that change the path of signals coming and going, distributor amplifiers (not shown), and multiple monitors (not shown) to view intermediate processes.

The VCR 20 is equipment for reproducing and recording a broadcast tape. The character generator 21 is a device for adding subtitles and graphics to the image. The switcher 22 is a device that receives several video signals and combines them into one.

In the comprehensive editing room, the VCR 20 largely plays back the tape, combines the video signal resulting from the playback and the graphic signal from the character generator 21 through a device called a switcher 22, and outputs the tape from the VCR 23 again. Record with.

Figure 3 schematically shows a character generator structure according to the prior art.

The character generator according to the prior art is based on a PC. The video signal input / output card, which is a hardware board for inputting and outputting video video signals (broadcasting and high-quality video uses a video signal standard called SDI (Serial Digital Interface)), is used as a PC. It is constructed by mounting on. The configuration of the conventional character generator is shown in FIG.

The user edits graphics, captions, and animations using the user interface 30. The edited materials are then stored as a data structure 31 on the PC memory 33. The rendering engine 32 generates an image based on the data structure stored in the PC memory 33. Most character generators use Microsoft's GDI or GDI + software as the rendering engine.

The part shown by the dotted line is the structure of the video signal input / output card 34. The video signal input / output card 34 includes a converter (AD converter) 35 for converting an analog video signal into a digital video signal, a frame buffer 36, and a converter for converting a digital video signal into an analog video signal. 37). The video signal input / output card 34 is a hardware card mounted in an expansion slot of a PC. In this video signal input / output card 34, the converter 35 converts an image coming from an external SDI signal into a digital video signal and stores it in a memory called an internal frame buffer 36. The image generated by the rendering engine 32 and the image received by the image signal input / output card 34 are synthesized in the frame buffer 36, and the synthesized result is converted into an analog image signal by the converter 37 and converted into an SDI signal. Is output.

4 schematically illustrates a workflow of digital content production according to the prior art.

The new camera 40 exports the output in the form of a file 41. This moving image file 41 is edited by NLE (None Linear Editor, 42). However, inputting subtitles and graphics using the NLE 42 is very inconvenient, and requires a further rendering process. For reference, it takes about 5 ~ 6 hours to render 1 hour HD video. Therefore, inputting subtitles and graphics into the NLE 42 is very inefficient, and therefore a comprehensive editing room must be used. By the way, the comprehensive editing room facility has not evolved compared with the conventional analog content production environment. Therefore, files that have passed through the NLE 42 for work in the comprehensive editing room must be converted back to tape (43). However, there is a problem that it takes at least one hour to convert a one hour file to a tape. The tape 44 converted as described above is converted into a file again after subtitles and graphics are added in a comprehensive editing room 45 equipped with a conventional character generator, switcher and the like (46). At this time, it takes at least one hour for the tape to be converted back into a file. The file 47 thus converted is broadcasted through the delivery main control room file server 48.

As such, the digital content production work has been digitized, but it is difficult to digitize the whole work because the files must be converted to tape for work in the general editing room, and the tapes are converted back into files after the work in the general editing room. to be. Therefore, there is an inefficient problem in terms of time and cost.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a rendering engine capable of directly writing captions and graphics to a file that is a camera output. It is possible to provide subtitles and graphic input without the comprehensive editing room, and to provide a character generator that can simplify the content production network because the entire process from the first output of the camera to the final transmission is done only by file.

Another object of the present invention is that the rendering engine directly inputs subtitles or graphics into a file that is the output of the camera, eliminating the need to convert the file to tape, and enables caption and graphic input without a comprehensive editing room. It is to provide subtitle and graphic input method using character generator that can simplify content production transmission network because the whole process from the first output to the final transmission is done by file only.

The character generator according to the present invention for achieving the above object comprises: a user interface for generating subtitles and graphics and storing them in a memory as a data structure; A rendering engine for generating an image based on the data structure; A video decoder for converting a file into an image; A mixer for synthesizing the image generated by the rendering engine and the image converted by the video decoder; A video encoder for converting the image and video synthesized by the mixer into a file; And a thread manager for exchanging data between the tasks by the user interface, the rendering engine, the video decoder, the mixer, and the video encoder and for ordering the tasks.

The rendering engine generates a base path of the image, deforms the base path, rasterizes the modified path, and renders a raster image generated by the rasterization to produce a final image.

The mixer synthesizes the image with the image through alpha blending using transparency values of the image generated by the rendering engine.

The file format is MXF, HDV, MPEG2, MPEG4, MOV, H.264, AVI, WMV, MPG, M2T, TS, TP, TRP, D10, GXF, AAF, CIF, QCIF, QQCIF, DVCPRO, DVCPROHD, DVD , Blue-ray (Blue-ray) and AVCHD is characterized in that any one.

The character generator according to the present invention further includes a video signal input / output card, wherein the video signal input / output card comprises: a first converter for converting an analog video signal into a digital video signal; A frame buffer for synthesizing the digital video signal and the image generated by the rendering engine; And a second converter converting the digital video signal and the image synthesized by the frame buffer into an analog video signal.

Subtitles and graphics input method using the character generator according to the present invention for achieving the above another object comprises the steps of generating a caption and graphics by the user interface in the memory as a data structure; A rendering engine generating an image based on the data structure; The video decoder converting the file into an image; A mixer synthesizing the image generated by the rendering engine and the image converted by the video decoder; And converting, by the video encoder, the image and the image synthesized by the mixer into a file.

Generating an image by the rendering engine includes: generating a basic path of an image; Deforming the base path to produce a modified path; Rasterizing the modified path to produce a raster image; And generating a final image by rendering the raster image.

The compositing step is characterized by using alpha blending using transparency values of the image generated by the rendering engine.

The file format is MXF, HDV, MPEG2, MPEG4, MOV, H.264, AVI, WMV, MPG, M2T, TS, TP, TRP, D10, GXF, AAF, CIF, QCIF, QQCIF, DVCPRO, DVCPROHD, DVD , Blue-ray and AVCHD.

A caption and graphic input method using a character generator according to the present invention includes the steps of: converting an analog video signal into a digital video signal by a first converter; A frame buffer synthesizing the digital video signal and an image generated by the rendering engine; And converting, by the second converter, the digital video signal and the image synthesized by the frame buffer into an analog video signal.

As described above, the character generator according to the present invention includes a rendering engine that can directly write subtitles, graphics, and the like into a file output from a camera, thereby reducing work time by eliminating the conversion time between the file and the tape. You can add subtitles and graphics without a comprehensive editing room, reducing the cost of building an expensive comprehensive editing room.

In addition, since the entire process from the camera's first output to the final transmission is done only with files, the content production network can be simplified, the entire production process can be configured with an IP network, and the content production can be simplified. do.

In addition, the user can perform all the editing process from the camera's output file to the final transmission file, greatly reducing the cost.

Hereinafter, the present invention will be described in more detail with reference to Examples, but the scope of the present invention is not limited to the following Examples.

5a and 5b schematically show one embodiment of the structure of the character generator according to the present invention.

The character generator according to the present invention is processed all the software in general PC without additional hardware. In addition, unlike conventional character generators that receive video signals using hardware, subtitles and graphics may be directly input to a video file on a PC hard disk.

5A and 5B, the character generator according to the present invention includes a user interface 50, a rendering engine 51, a video decoder 52, a mixer 53, a video encoder 54, a thread manager, 55) and a video signal input / output card 56.

The user generates graphics, subtitles, and animations using the user interface 50 and stores them in a PC memory as a data structure.

The rendering engine 51 takes full advantage of the function of the CPU and performs 256 × 256 sampling on one pixel to generate a high quality image at high speed. To create a picture, you first need to free memory to draw it. The HD image is composed of 1920 pixels horizontally and 1080 pixels vertically, and each pixel is composed of RGBA values. Each RGBA consists of 8 bits, so one pixel requires 32 bits (4 bytes) of space. The calculation requires 1920 × 1080 × 4 = 8,294,400 bytes for a single image. Free up this amount of space in memory, and set the render engine to draw here.

Meanwhile, the video decoder 52 converts the video file into an image, and the converted image is synthesized with the image generated by the rendering engine 51 in the software mixer MIXER 53.

The mixer 53 performs a synthesis process called "alpha blending" using the transparency value of the image generated by the rendering engine. To represent the color of a single pixel on a computer, we use red, blue, and green color values and alpha values representing transparency. Each value ranges from 0 to 255. This is called RGBA. If you draw two pixels at the same location, the transparency alpha value is used to calculate the blend of the two colors. Each value is obtained by the following equation.

If two pixels are (r1, g1, b1, a1) and (r2, g2, b2, a2):

R = (((r1-r2) * a1 + (r2 << 8)) >> 8)

G = (((g1-g2) * a1 + (g2 << 8)) >> 8)

B = (((b1-b2) * a1 + (b2 << 8)) >> 8)

A = ((a1 + a2)-((a1 * a2 + 255) >> 8))

By the above equation, the caption and graphic image are combined with the video image. For example, if the subtitles are translucent, the subtitles are synthesized semi-transparently for the video image.

The synthesized result is then converted into a file by video encoder 54.

At this time, the thread manager 55 exchanges data between tasks by the user interface 50, the rendering engine 51, the video decoder 52, the mixer 53, and the video encoder 54. And order the tasks. In other words, when the tasks operate in a multithreaded environment, the thread manager 55 operates without collisions, and efficiently arranges task sequences. For example, when editing captions or figures, rendering operations or sending operations are automatically delayed to prevent data conflicts. Also, if only one thread is working exclusively for a long time, no other work can be done. To solve this, limit the time that one thread can run exclusively, wait for a while, and then wait. If there is a thread, it executes the waiting task, otherwise it continues the task.

The character generator according to the present embodiment includes a video signal processing function of the character generator according to the prior art, in addition to a function of directly inputting captions and graphics into a file.

The video signal input / output card 56 includes a converter 57 for converting an analog video signal into a digital video signal, a frame buffer 58 for synthesizing the digital video signal and an image generated by the rendering engine 51, and the frame. And a converter 59 for converting the digital video signal and the image synthesized by the buffer 58 into an analog video signal. In the present embodiment, an example of the character generator including the video signal input / output card 56 has been described. However, in the alternative embodiment, the video signal input / output card 56 may be omitted.

5C schematically illustrates a process in which a rendering engine generates an image.

Referring to FIG. 5C, all subtitles and graphics start at vector data 90. For example, a quadrangle is represented by the position, width, and height of the upper right vertex, and in the case of a letter, it is composed of the position values of the points constituting the letter. If you connect these points with a line, you get the path (91) corresponding to the outline. When the outline value is deformed using various kinds of deformation algorithms, the deformed path 92, which is the second deformation outline, is obtained. Filling the interior with this value yields a raster image (93). Applying various colors or textures to the image completes the rendered image 94 which is the final image.

Thus, according to the present invention, unlike the conventional character generator, it is possible to directly input the subtitles and graphics in the video file, thereby reducing the work time by eliminating the conversion time between the file and the tape, even without a comprehensive editing room Additional work can be done to reduce the cost of building an expensive, comprehensive editing room. In addition, the entire process from the first output of the camera to the final transmission is done only by file, so the content production network can be simplified, the entire production process can be composed of an IP network, and the content production work can be simplified. Promote.

Figure 6 schematically illustrates the MXF rendering function of the character generator according to the present invention.

The character generator in this embodiment includes all the functions of the character generator according to the prior art, and further has an MXF rendering function as a unique function. The MXF rendering feature writes subtitles and graphics directly into the MXF file itself.

The video format in this embodiment is MXF, but the present invention is not limited thereto. Alternative embodiments include HDV, MPEG2, MPEG4, MOV, H.264, AVI, WMV, MPG, M2T, TS, TP, TRP, D10, GXF, AAF, CIF, QCIF, QQCIF, DVCPRO, DVCPROHD, DVD, Blue-ray or AVCHD is possible.

Each block in FIG. 6 is a software module.

First, when the MXF file 60 designated as an input is selected, the MXF demultiplexer 61 separates the audio data and the video data. The separated data is restored to the raw data using the audio decoder 62 and the video decoder 63. The audio is directly input to the audio encoder 64 without any additional processing. After the video data is restored to the raw data, it passes through the rendering engine 65 of the character generator according to the present invention. In this process, subtitles and graphics are synthesized in the original image. The synthesized image is input to the video encoder 66 and compressed. Compressed audio / video data are combined in an MXF multiplexer 67 to form an MXF file 68.

7 schematically illustrates a flow of a process of synthesizing captions and graphics in a video file using a character generator according to the present invention.

Graphic data 70 consisting of coordinate values and numbers is made using a user interface. The rendering engine 71 generates an image using the graphic data 70. The generated image is stored in a high speed memory structure called DIB (Device Independant Bitmap) 72. This is called 'graphic'.

The video file 73 is a computer file containing a video. The decoder 74 converts the file into an image to generate a video image. The generated video image is stored in the DIB mixer 75. This is called 'video'.

Compose the graphic image on the video image. In this case, alpha blending (76) is performed using the transparency value of the graphic. An image is created by combining video and graphics. The composite image is input to the encoder 77. The composite image is displayed (78) in the user interface for preview. Encoder 77 encodes the composite image into a video file 79.

8 schematically illustrates a workflow of digital content production using a character generator according to the present invention.

First, the output of the camera 80 comes out as a video file 81. The moving image file 81 is edited by the NLE 82, and the edited file 83 is input to the character generator 84 according to the present invention. The character generator 84 according to the present invention inputs subtitles and graphics into the edited file 83 while playing the file. Subtitles and graphic input of the character generator 84 according to the present invention is performed in real time. Therefore, no additional rendering or conversion work is required. The video file 85 passed through the character generator 84 according to the present invention is directly transmitted to the file-based delivery main control room file server 86 via the IP network.

As described above, according to the present invention, the conversion between the file and the tape becomes unnecessary, and the operation of adding subtitles and graphics can be performed without the comprehensive editing room, thereby reducing the cost of constructing an expensive comprehensive editing room. In addition, since the entire process from the camera's first output to the final transmission is done only with files, the content production network can be simplified, the entire production process can be configured with an IP network, and the content production can be simplified. do. In addition, the user can perform all the editing process from the camera's output file to the final transmission file, greatly reducing the cost.

While specific embodiments of the present invention have been described above, the present invention is not limited to the above-described specific embodiments, and the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Anyone of ordinary skill in the art can make various modifications, as well as such modifications are within the scope of the claims.

1 schematically illustrates the workflow of analog content production according to the prior art.

2 schematically illustrates an internal configuration of a general editing room according to the prior art.

Figure 3 schematically shows a character generator structure according to the prior art.

4 schematically illustrates a workflow of digital content production according to the prior art.

5a and 5b schematically show one embodiment of the structure of the character generator according to the present invention.

5C schematically illustrates a process in which a rendering engine generates an image.

Figure 6 schematically illustrates the MXF rendering function of the character generator according to the present invention.

7 schematically illustrates a flow of a process of synthesizing captions and graphics in a video file using a character generator according to the present invention.

8 schematically illustrates a workflow of digital content production using a character generator according to the present invention.

Claims (10)

A user interface for generating subtitles and graphics and storing them in a memory as a data structure; A rendering engine for generating an image based on the data structure; A video decoder for converting a file into an image; A mixer for synthesizing the image generated by the rendering engine and the image converted by the video decoder; A video encoder for converting the image and video synthesized by the mixer into a file; And And a thread manager for exchanging data between the tasks by the user interface, the rendering engine, the video decoder, the mixer, and the video encoder and for ordering the tasks. The rendering engine of claim 1, wherein the rendering engine generates a basic path of an image, deforms the basic path, rasterizes the modified path, and renders a raster image generated by the rasterization to generate a final image. Character generator characterized in that. The character generator of claim 1, wherein the mixer synthesizes the image and the image through alpha blending using transparency values of the image generated by the rendering engine. The format of claim 1, wherein the file format is MXF, HDV, MPEG2, MPEG4, MOV, H.264, AVI, WMV, MPG, M2T, TS, TP, TRP, D10, GXF, AAF, CIF, QCIF, QQCIF Character generator, characterized in that any one of, DVCPRO, DVCPROHD, DVD, Blu-ray and AVCHD. The video signal input / output card according to any one of claims 1 to 4, further comprising a video signal input / output card, A first converter converting the analog video signal into a digital video signal; A frame buffer for synthesizing the digital video signal and the image generated by the rendering engine; And And a second converter for converting the digital video signal and the image synthesized by the frame buffer into an analog video signal. Generating, by the user interface, subtitles and graphics and storing them in memory as data structures; A rendering engine generating an image based on the data structure; The video decoder converting the file into an image; A mixer synthesizing the image generated by the rendering engine and the image converted by the video decoder; And Caption and graphic input method using a character generator comprising a; video encoder converts the image and video synthesized by the mixer to a file. The method of claim 6, wherein generating the image by the rendering engine comprises: Generating a basic path of an image; Deforming the base path to produce a modified path; Rasterizing the modified path to produce a raster image; And Caption and graphic input method using a character generator, comprising the step of rendering a raster image to generate a final image. The method of claim 6, wherein the synthesizing comprises: Caption and graphic input method using a character generator using alpha blending using the transparency value of the image generated by the rendering engine. The file format of claim 6, wherein the file format is MXF, HDV, MPEG2, MPEG4, MOV, H.264, AVI, WMV, MPG, M2T, TS, TP, TRP, D10, GXF, AAF, CIF, QCIF, QQCIF Subtitle and graphic input method using a character generator, characterized in that any one of, DVCPRO, DVCPROHD, DVD, Blu-ray and AVCHD. The method of claim 6, further comprising: converting, by the first converter, the analog video signal into a digital video signal; A frame buffer synthesizing the digital video signal and an image generated by the rendering engine; And And a second converter converting the digital video signal and the image synthesized by the frame buffer into an analog video signal.

Images