JP2015060539A - Content generation device and content generation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a content simply in real time.SOLUTION: Provided is a content generation device for generating a video content having one or a plurality of CG objects, the content generation device having: script charge assignment means for assigning, on the basis of the type and number of CG objects included in the video content to be generated, the number of video generation engines to generate the video content from a script, and assigning, for each of the assigned video generation engines, a script to be taken charge of; script generation means for generating the script for generating the video content; and content generation means for generating, by the assigned video generation engines and using the generated script, a video content to be taken charge of by each, and, while accepting video contents generated by other video generation engines as input video, synthesizing the input video and the video contents generated from the script.

Description

  The present invention relates to a content generation device and a content generation program, and more particularly, to a content generation device and a content generation program for generating content easily and in real time according to the type and number of CG (Computer Graphics) objects.

  Conventionally, video content such as a television program has been produced using CG, voice synthesis, or the like. In such video content, it may be necessary to generate in real time video content in which various CG objects such as CG characters (CG avatars), desks, and accessories appear. However, for example, the number of CG objects that can be displayed in real time on one computer (information processing apparatus) is limited due to the influence of the data capacity (for example, the number of polygons, etc.) and the load on the CPU (Central Processing Unit).

Conventionally, in order to reduce the load on the computer, screen division, time division, etc. are performed, the divided images are generated by different devices, and the generated images are spatially or temporally synthesized and output to the display unit. (For example, refer to Patent Documents 1 and 2.)
In the method shown in Patent Document 1, in an image display method for displaying an animation by moving a hero character or the like (cast) according to a scenario, the character in the image is divided into a plurality of blocks, and each block is independently Performs image processing such as compression and decompression.

  The technique shown in Patent Document 2 is a message display method for displaying a file transmitted from the server side through the network on the client side, dividing the display screen on the client side into a plurality of frames, and HTML based on the frame setting. When a (Hyper Text Markup Language) file is displayed, a message is displayed with reference to an HTML file corresponding to another frame.

JP 2000-148133 A Japanese Patent Laid-Open No. 2003-186792

  However, in the conventional screen division method, since it is necessary to generate video of different partial areas in each computer, the processing becomes complicated. Further, in the conventional time division method, since it is necessary to individually determine each script transmitted from the server side and calculate a necessary time, the processing becomes complicated.

  Furthermore, if the timing of the output video of each script deviates even slightly, the boundary between screen divisions can be seen, and the smoothness of the animation movement is impaired. Further, when the number of video generation units (video generation engines) is changed in accordance with the number of divisions, a change in the respective video generation processing and a change in the synthesis processing occur, so that a flexible configuration change is difficult.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to generate content easily and in real time according to the type and number of CG objects.

  In order to solve the above problems, the present invention employs means for solving the problems having the following features.

  The content generation device according to one aspect of the present invention is a content generation device that generates video content having one or more CG objects, based on the type and number of CG objects included in the video content to be generated, from the script. Assigning the number of video generation engines for generating video content and assigning a script in charge for each assigned video generation engine; script generating means for generating the script for generating the video content; Using the script generated by the script generation means, the video generation engine assigned by the script charge assignment means generates the video content each is in charge of, and is generated by another video generation engine. Video content as an input video image, and having a content generation means for synthesizing the video content generated from the the input video script.

  A content generation program according to an aspect of the present invention is a content generation program for causing a computer to function as each unit included in the above-described content generation apparatus.

  According to the present invention, content can be generated easily and in real time.

It is a figure which shows an example of a function structure of the content generation apparatus in this embodiment. It is a flowchart which shows an example of the process in a content generation apparatus. It is a figure for demonstrating the example of determination of the video production | generation engine number. It is a figure for demonstrating the specific example of content generation. FIG. 6 is a diagram (part 1) illustrating an example of a script output to a video generation engine. It is FIG. (2) which shows an example of the script output to a video generation engine. FIG. 11 is a third diagram illustrating an example of a script output to the video generation engine. It is a figure for demonstrating the shift | offset | difference of a composite image. It is a figure which shows an example of the setting file for script transmission. It is a figure which shows the structural example of a content generation system. It is a figure which shows the example of application of a content generation system.

  Hereinafter, a preferred embodiment of a content generation device and a content generation program according to the present invention having the above-described features will be described in detail with reference to the drawings. In the following description, a broadcast program is used as an example of content. However, the example of content is not limited to this, and can be applied to various video contents such as movies and TV commercials. In the content generation according to the present embodiment, for example, TVML (TV program making language) can be used, but the present invention is not limited to this.

  TVML is a scripting language that includes the concept and know-how of TV program production such as camera and lighting in the language system, and can describe video and audio separately for material data and scripts (production contents, etc.). . By writing a script on the production side, software operating on a PC (Personal Computer) or the like reads this as a script (TVML script), acquires the material data described in the script, Body movement, facial expressions, etc. can be given.

<Functional configuration example of content generation device>
FIG. 1 is a diagram illustrating an example of a functional configuration of a content generation apparatus according to the present embodiment. 1 includes an input unit 11, an output unit 12, a storage unit 13, a script charge assignment unit 14, a script generation unit 15, a delay adjustment unit 16, a content generation unit 17, The communication unit 18 and the control unit 19 are included. The content generation unit 17 includes one or a plurality of video generation engines 20-1 to 20-n (hereinafter referred to as “video generation engines 20” as necessary).

  The content generation device 10 is a PC, a server, or the like, but is not limited thereto, and may be a smartphone, a tablet terminal, a television device having a video generation function / communication function, a game device, a camera, a mobile phone, or the like. .

  The input unit 11 receives an input such as an instruction from a user or the like who uses the content generation device 10. The input unit 11 receives, for example, each input such as a script charge assignment instruction, a script generation instruction, a delay adjustment instruction, a content generation instruction, and a transmission / reception instruction for content generation in the present embodiment. The input unit 11 is, for example, a keyboard, a pointing device such as a mouse, a voice input device such as a microphone, or the like.

  The output unit 12 displays the instruction content or setting content input from the input unit 11 on a screen or the like, or outputs an execution result or the like executed based on the instruction content or setting content. The output means 12 is, for example, a screen display device such as a display or a sound output device such as a speaker. Note that the input unit 11 and the output unit 12 described above may have an input / output integrated configuration such as a touch panel, for example.

  The storage means 13 stores various information necessary for content generation in the present embodiment. For example, the storage unit 13 stores information on a broadcast program (content) to be generated (for example, a program script, a studio set, the type (for example, complexity of the CG object) and number of CG objects appearing in the program, and delay information). Script, contents, material data, program production engine, basic operation data for each CG object, etc. are stored.

  Note that the CG object in the present embodiment is, for example, a CG character (human or animal), a building (for example, a building or a house), a tree, a flower, a chair, a desk, furniture, a PC, a home appliance, a vehicle, a ship, the sea, the sun, Although it is various objects, such as an airplane, it is not limited to these. The complexity of the CG object is set by, for example, the number of polygons of the entire object, and becomes more complex as the number of polygons increases. Note that the complexity criterion is not limited to this, and may be set based on, for example, the expression of the CG object, the operation content, or the like.

  The storage unit 13 can also acquire the various data described above from an external device connected to a communication network such as the Internet or a LAN (Local Area Network) via the communication unit 18, for example. Various data stored in the storage unit 13 is read and written as necessary.

  The script assignment assigning unit 14 assigns a script assignment to one or a plurality of video generation engines 20-1 to 20-n in the content generation unit 17. For example, the script charge assignment unit 14 measures the performance of the video generation engine 20 in the content generation unit 17. The performance measurement means that the video generation engine 20 used at the time of program generation measures, for example, the relationship between the complexity of CG characters (for example, the number of polygons) that appear in the program and the number of characters that can be displayed in real time. It is not limited to.

  Further, the script charge assigning means 14 determines the type of CG object and the number of CG objects at the time of generating the program based on the above-mentioned performance measurement result and the intention and effect of the program obtained from the script, etc. The number of video generation engines 20 is allocated according to the above. For example, when it is determined that two video generation engines 20 are necessary based on the number of CG objects, the video generation engine 20-1 and the video generation engine 20-2 are allocated as the content generation unit 17, and which engine is assigned to each engine. Such a video is generated (script assignment), and how to compose (composition order) and the like are set. The script staff assignment unit 14 sets the video output of the video generation engine 20-1 to be used as the input video of the video generation engine 20-2. The set information is managed as parameter information.

  The script generation unit 15 generates a script corresponding to each of the video generation engines 20 assigned by the script charge assignment unit 14 in order to generate content. For example, the script generation means 15 may generate a script for an initial scene using a preset studio set or a CG character, and the CG character includes parameter information such as basic speech and operation content. If so, a corresponding program progress (for subsequent scene) script may be generated based on the parameter information.

  The delay adjusting unit 16 adjusts the delay time in the video generation between the video generation engines so that the content generated by the video generation engines 20 is output as a video signal in synchronization. For example, the delay adjustment unit 16 uses the initial scene script of the program generated by the script generation unit 15 or a preset script, and from the result of the temporary generation of the video by each video generation engine 20, The video error is acquired, and the delay time of each video generation engine 20 is acquired.

  The delay adjusting unit 16 adjusts which video generation engine 20 the script is transmitted to at which timing so that the content generated from each video generation engine 20 is synchronously output based on the acquired delay time. Get time.

  The delay adjustment information obtained by the delay adjustment unit 16 is output to the script generation unit 15. Thereby, the script generation means 15 can adjust the transmission timing of the script sent to each video generation engine 20 based on the delay adjustment information obtained by the delay adjustment means 16, and synchronizes each video at the time of video synthesis. be able to.

  The content generation unit 17 generates video content based on script, parameter information, material data (for example, image or video), and the like. In addition, the content generation unit 17 synthesizes a video acquired from another video generation engine, a video acquired from the outside via the communication unit 18 and the like, and a video generated inside the engine. For example, the content generation unit 17 includes a plurality of video generation engines 20-1 to 20-n, and the video generated by the previous video generation engine in accordance with the order assigned by the script charge assignment unit 14, It is possible to synthesize the video generated by itself and send the synthesized video to the video generation engine at the next stage. By synthesizing videos generated by a plurality of video generation engines 20 in stages, many CG objects can be displayed simultaneously on one screen.

  Each of the video generation engines 20-1 to 20-n generates a video including a CG object by a script in real time, and synthesizes a video input from another video generation engine and an internally generated video in real time. To do. For example, each of the video generation engines 20-1 to 20-n extracts the script assigned by the script assignment assigning unit 14 from the script file input by each, and generates content for the extracted script. To do.

  Each of the video generation engines 20-1 to 20-n is configured by a GPU (Graphics Processing Unit) or the like, but is not limited thereto, and may be, for example, a VPU (Visual Processing Unit) or a CPU. . That is, the content generation unit 17 generates a program by distributing video generation processing using a multiprocessor or the like.

  Also, each of the video generation engines 20-1 to 20-n interprets a script input by each of the video generation engines 20-1 to 20-n, and generates a video corresponding to the script when the script is in charge of video generation. Also, each of the video generation engines 20-1 to 20-n does not generate a video for the script when the input script is not a script in charge.

  The content generation unit 17 generates content based on the script to be processed for each video generation engine 20 assigned by the script charge assignment unit 14 and information such as the order in which the video is to be combined.

  Further, the content generation unit 17 may have a function as a TVML player that reads, for example, a TVML script, material data, etc., and generates an animation video in real time while sequentially interpreting the script.

  The communication means 18 is a communication interface for transmitting / receiving data to / from an external device via a communication network or the like. Communication by the communication means 18 may be wired or wireless, or a combination thereof.

  The control unit 19 controls the entire functional configuration of the content generation apparatus 10 in the present embodiment. Specifically, for example, the control unit 19 assigns a script in charge by the script assigning unit 14, generates a script by the script generation unit 15, adjusts a delay in each video generation engine 20 in the delay adjustment unit 16, Various controls such as content generation in the content generation unit 17 and data transmission / reception in the communication unit 18 are performed.

  Note that the control performed by the control unit 19 is not limited to this. For example, the start and end of content generation processing can be controlled, and error control (for example, message output when an error occurs) can be performed. .

  In the present embodiment, the above-described content generation apparatus 10 can easily and in real time generate video content in which many CG objects appear using one or a plurality of video generation engines.

<Example of Processing (Content Generation Processing) in Content Generation Device 10>
Next, an example of processing in the content generation apparatus 10 described above will be described using a flowchart. FIG. 2 is a flowchart illustrating an example of processing in the content generation apparatus. In the following description, a CG character is used as an example of a CG object that appears in a program (content) to be generated. However, the present invention is not limited to this. In the content generation process shown in the example of FIG. The assigning unit 14 measures the performance of video generation in the content generation unit 17 (S01). In the process of S01, for example, the relationship between the number of characters that can be displayed in real time is measured for each type of CG character stored in advance in the storage unit 13 or the like with respect to the video generation engine 20.

  Next, the script charge assigning means 14 determines the character and the number of characters to appear in the generated program (content) based on the performance measurement result obtained in the process of S01, a preset program script, and the like (S02). ).

  Next, the script charge assignment unit 14 determines the number of characters and the number of video generation engines 20 corresponding to the number of characters, and assigns a script for each video generation engine (S03). For example, when displaying 40 female CG characters in red clothes in a program to be generated, it is determined from a performance measurement result that up to 20 can be displayed in real time by one video generation engine. In this case, the script charge assignment unit 14 assigns two video generation engines.

  Each video generation engine 20 is identified by various address information such as a preset ID, network address, host name, port number, and MAC (Media Access Control) address. However, the present invention is not limited to this. Further, the script charge assignment unit 14 acquires, for example, the video generated by the video generation engine 20-1 as an input of the video generation engine 20-2, and synthesizes it with the video generated by the video generation engine 20-2. An order or the like can also be set.

  Next, the script generation unit 15 generates an initial scene script corresponding to each video generation engine, for example (S04). In the process of S04, the script generation unit 15 generates a script file for the video generation engine 20-1 (for example, “Char40test_h1.h” when generating the video using the two video generation engines 20-1 and 20-2, for example. tvml ") and a script file (for example," Char40test_h2.tvml ") for the video generation engine 20-2.

  Next, the delay adjusting unit 16 adjusts the delay time in each video generation engine 20 so that there is no delay (error) between the two videos when the video generated in each video generation engine 20 is synthesized. (S05). In the process of S05, for example, when the video generation engine 20-2 synthesizes the output video of the video generation engine 20-1 and the video generated from the script by the video generation engine 20-2, the delay adjustment unit 16 The same script is output to the two video generation engines 20-1 and 20-2 described above to temporarily generate a video, and further, the delay time is determined from the shift amount of the two videos synthesized by the video generation engine 20-2. Get (delay amount). The same script may be, for example, an initial scene script obtained in S04 or a script that performs a preset process (for example, camera work), but is not limited thereto. .

  In addition, when there is a video shift, the delay adjusting unit 16 again shifts the timing of the script output to the video generation engine 20-1 and the video generation engine 20-2 based on the shift amount. The same script is output, and the video temporarily generated by each video generation engine is acquired. By performing this operation until there is no video shift, a delay time (delay amount) is finally acquired at which there is no video shift during synthesis.

  As an example of the script for acquiring the delay time, for example, a script “camera: movement (name = CameraA, y = 1.96, z = 6, pan =) -21.00, tilt = 8.00, angle = 46.00, speed = 0.2) ”,“ camera: movement (name = CameraA, y = 1.96, z = 6, pan = 21.00, tilt = 8.00, vangle = 46.00, speed = 0.2) ”can be used, but the present invention is not limited to this.

  Next, the script generation unit 15 outputs a script to each video generation engine 20 at a timing based on the delay time adjusted by the process of S05 (S06). Next, the content generation unit 17 generates content with each video generation engine 20 to which the script is input (S07). In the processing of S07, the content generation unit 17 generates content corresponding to the script when the input script includes a script that the user is in charge of. Further, the content generation unit 17 does not generate content when the input script does not include a script that the content generation unit 17 is responsible for.

  Next, the content generation means 17 combines the content generated by the plurality of video generation engines 20 to generate a final program (S08). In the processing of S08, the content generation unit 17 performs the video generation engine when, for example, the output of the video content generated by the video generation engine 20-1 is input as the background of the video content generated by the video generation engine 20-2. The video content generated by the video generation engine 20-2 is synthesized on the video content (background) generated in 20-1. In the present embodiment, since the generation time is delayed by the delay adjusting unit 16 in consideration of the generation time of each video generation engine 20, in the process of S08, synchronized video composition can be realized. In the present embodiment, content may be combined using three or more video generation engines 20. Next, the output unit 12 outputs the combined program generated by the content generation unit 17 by the output unit 12 or the like (S09).

  Here, the control means 19 of the content generation apparatus 10 determines whether or not to generate a script of the subsequent scene (program progression script), for example (S10). In the processing of S10, the control means 19 can make a determination based on, for example, a script corresponding to the program being produced, but is not limited to this.

  When it is necessary to generate a script for the subsequent scene (YES in S10), the script generation unit 15 generates a script for the subsequent scene corresponding to each video generation engine 20 (S11), and returns to the process of S06, and returns to the predetermined process. The script is output at the timing. For example, in the process of S11 described above, the script generation unit 15 generates the same script (Char40test_h3.tvml) along with the progress of the program, and based on the delay time acquired in the process of S05, the video generation engine 20- 1 and 20-2 are transmitted at adjusted timings.

  Further, in the process of S10, when the script of the subsequent scene is not generated (NO in S10), the control unit 19 ends the content generation process.

<Determination Example of Number of Video Generation Engines in Script Assigning Unit 14>
Next, an example of determining the number of video generation engines in the script person assigning unit 14 will be described. FIG. 3 is a diagram for explaining an example of determining the number of video generation engines. In the following description, a CG character is used as an example of a CG object.

  In general, display of a CG character having skin that deforms smoothly according to body movement requires a lot of calculation processing. Therefore, in this embodiment, the script charge assignment unit 14 measures the performance of the video generation engine 20 in the content generation unit 17 and measures the number of CG objects that can be displayed in real time for each type of CG object.

  For example, the script charge assignment unit 14 performs real-time without causing a delay in the operation of the video generation engine 20 due to processing load or the like according to parameters such as the number of polygons (complexity) set in advance for each CG character. Determine the number of characters that can be generated.

  In the example shown in FIG. 3, CG characters 30-1 to 30-3 that can appear in a program are set in advance, and the number of polygons is set for each CG character. Note that the types of CG characters 30-1 to 30-3 and the number of polygons are not limited thereto.

  The script charge assigning unit 14 sets the number of characters that can be generated by one video generation engine in accordance with the number of polygons used by the CG characters 30-1 to 30-3. Note that the number of characters and the number of characters to appear in the content can be set by, for example, a script of a program to be generated.

  For example, if 40 CG characters 30-2 (female in red clothes) are to be displayed in real time, it can be seen from the performance measurement results shown in FIG. Therefore, the script charge assignment unit 14 assigns two video generation engines to display the 40 CG characters 30-2, and generates a script corresponding to each video generation engine assigned to the script generation unit 15.

<Specific example of content generation>
Next, a specific example of content generation in the above-described embodiment will be described. FIG. 4 is a diagram for explaining a specific example of content generation. 5 to 7 are diagrams (part 1 to part 3) illustrating examples of scripts output to the video generation engine. In the examples of FIGS. 5 to 7, line numbers are shown on the left side of the TVML script for convenience of explanation.

  In the example of FIG. 4, an example in which a program is generated using two video generation engines 20-1 and 20-2 in the content generation unit 17 is shown. The video generation engines 20-1 and 20-2 read the script file generated by the script generation means 15, reproduce the scripts that they are responsible for in order, and output the video. In addition, the video generation engines 20-1 and 20-2 can also input camerawork data and the like that change at high speed in real time as parameters.

  In addition, since the video generation engines 20-1 and 20-2 can synthesize the video step by step, many CG characters can easily appear simultaneously on one screen.

  In the example of FIG. 4, the video generation engine 20-1 is in charge of generating a background set and a character group behind the screen (camera position) using a TVML script (for example, Char40test_h1.tvml) as shown in FIG. doing. The script in this embodiment can be used for video content production in general such as a television program, for example, in addition to chatting and operation of a CG character, lighting, camera, set, prop, superimpose, video, BGM, etc. All functions necessary for TV program studio production can be described.

  In the TVML script shown in the example of FIG. 5, in addition to the settings of the studio set, lighting, character, camera, etc., the display position coordinates (20) of each CG character (20 bodies) on the screen (54) to (62). Some are omitted). In addition, the action content (partially omitted) for each CG character (20 characters) is set in the (66) to (74) lines. In the example of FIG. 5, Characters 1_1A to 10_1A and Characters 1_2A to 10_2A are CG objects handled by the video generation engine 20-1.

  The video generation engine 20-2 is in charge of generating a character group in front of the screen (camera position) using a script (for example, Char40test_h2.tvml) as shown in FIG.

  In the example of FIG. 6, there is a script that receives the output video 40-1 of the video generation engine 20-1 as a background video (lines (13) to (15)). Further, in the example of FIG. 6, in addition to the setting of lighting, character, camera, etc., the display position coordinates on the screen of each CG character (20 bodies) on the (54) to (62) lines (partially omitted) Is set. In addition, the action content (partially omitted) for each CG character (20 characters) is set in the (68) to (76) lines. In the example of FIG. 6, Characters 1_3A to 10_3A and Characters 1_4A to 10_4A are CG objects in charge of the video generation engine 20-2.

  In the video generation engines 20-1 and 20-2 shown in the example of FIG. 4, for example, scripts for lighting and cameras are processed with respect to the script of the initial scene shown in FIGS. Generate video for Further, as shown in FIG. 4, the video generation engine 20-2 receives the output video from the previous video generation engine 20-1 as a background video, and the video 40-2 generated by the video generation engine 20-2. The final video content 41 is output after being synthesized (overwriting only the character area).

  The scripts shown in FIGS. 5 and 6 can be controlled to be displayed (off: not skipped) or not displayed (on: skipped) by, for example, the “switch” parameter of “skipscript”. In the present embodiment, the video generation engine 20-1 is in charge of the background set and the rear character group. However, the present invention is not limited to this, and the background set and the character group are set as separate video generation engines. It may be divided, and not only the character group but also other CG objects may be displayed.

  Further, in the present embodiment, the script generation unit 15 generates a script file (for example, Char40test_h3.tvml) of the subsequent scene as shown in FIG. Output to 2. The video generation engines 20-1 and 20-2 receive the script file, and execute operation control on a script (for example, a CG character or the like) in charge of the script file.

  In the example of FIG. 7, in the (01) to (04) lines, a script that performs a gesture (motion) in which the characters (characters) 1_1A, 2_2A, 3_3A, and 4_4A shake their hands “bye-bye” is described. Accordingly, the video generation engine 20-1 performs video generation corresponding to the script (Character1_1A, 2_2A) on the (01) and (02) lines in FIG. 7 that it is in charge of, and the (03) and (04) lines This script is not supported. On the other hand, the video generation engine 20-2 does not correspond to the script on the (01) and (02) lines in FIG. 7, but corresponds to the script (Character 3_3A, 4_4A) on the (03) and (04) lines. Generate.

  Note that the script shown in FIG. 7 is output to the video generation engines 20-1 and 20-2 at different timings according to the delay time obtained from the delay adjustment unit 16. Thus, in the present embodiment, even when the same script is output to each video generation engine 20, only the assigned script assigned to each video generation engine 20 can be extracted and processed. Therefore, the script generation unit 15 does not need to generate or transmit an individual script for each video generation engine 20. Therefore, content can be generated easily and in real time.

<Specific example of delay adjustment>
Next, a specific example of delay adjustment in the present embodiment will be described. In the present embodiment, adjustment is performed to eliminate a delay between videos when the videos from the video generation engines 20 are combined. For example, as shown in FIG. 4, a script (camera work script) for changing the position, orientation, angle of view, etc. of the camera is output to the video generation engines 20-1, 20-2. For example, camera work scripts A and B shown in FIG. 4 can be output as the camera work script, but the camera work script is not limited to this. Further, instead of the camera work script, it may be a preset script such as a script for moving a CG character, or the initial scene script described above.

  The video generation engines 20-1 and 20-2 temporarily generate videos in charge of each, and the video generation engine 20-2 synthesizes and outputs the two videos.

  Here, for example, when the camera work script is output to the video generation engines 20-1 and 20-2 at the same timing, the video generation engine 20-1 and the video generation engine 20-2 generate the video almost simultaneously. However, due to the difference in performance of each video generation engine, the video from the video generation engine 20-1 is output to the video generation engine 20-2, the communication time until the video generation engine 20-2 is input, etc. The two images are not synchronized and a delay occurs, resulting in a shift in the overall operation. This is the same even when the CG character moves, for example.

  FIG. 8 is a diagram for explaining the shift of the composite video. FIG. 8A shows a case where there is a video shift, and FIG. 8B shows a case where there is no video shift. As described above, when the shift in processing time in each video generation engine 20 is not taken into account, as shown in FIG. 8A, a phenomenon such as the position of the CG character being shifted due to a change in camera work or the like occurs. .

  Therefore, in this embodiment, in order to eliminate such time lag, the script generation unit 15 shifts the output timing to the video generation engines 20-1 and 20-2 by a predetermined time according to an instruction from the delay adjustment unit 16. Output camera work script. The delay adjustment unit 16 acquires the finally synthesized video from the video generation engine 20-2, for example, and shifts from the video change as shown in FIGS. 8A and 8B (positional shift, motion shift, etc.). ) Is not generated (synchronized). If there is a deviation, the process is repeated until the deviation does not occur while changing the time for shifting.

  Thereby, the delay adjusting means 16 can acquire the delay time from the video temporarily generated by each video generation engine, and adjusts the output timing using the acquired delay time so that no deviation occurs. A script is output to the video generation engine 20. For example, the script generation unit 15 sets the timing of outputting the script to the video generation engine 20-2 later than the timing of outputting the script to the video generation engine 20-1, for example, as shown in FIG. In addition, it is possible to generate a synthesized video in which two videos are synchronized.

  In the present embodiment, the script generation unit 15 transmits a script based on a preset script transmission setting file. The delay time described above is also stored in the script transmission setting file.

  FIG. 9 is a diagram illustrating an example of a script transmission setting file. The script transmission setting file shown in the example of FIG. 9 includes, for example, “set work mode setting section”, “local mode setting section”, “real time parameter input setting section”, “repeat syntax setting section”, and “destination setting section”. However, the present invention is not limited to this.

  For example, as shown in FIG. 9, the script transmission interval based on the delay time is set as “DELAY” parameter as the delay amount (DELAY) parameter together with the host name (HOST) and port number (PORT) of the transmission destination for each video generation engine 20. It is stored in the “destination setting section”.

  In this embodiment, the maximum delay time “SEND_DELAY_MAX” may be set in advance. This setting information is set, for example, as a “network mode setting section” as shown in FIG. By setting the maximum delay time in this way, it is possible to prevent the synthesized video itself from being delayed and maintain the real-time video output.

  In the above-described embodiment, the two video generation engines 20-1 and 20-2 have been described. However, the present invention is not limited to this, and the complexity and number of CG objects appearing in the video, device performance, and the like. Depending on, more than two video generation engines can be used. For example, in the case of adding the video generation engine 20-3 in the above-described embodiment, the video generation engine 20-2 is arranged after the video generation engine 20-2, such as a material or a script that each video generation engine 20-1 to 20-3 is in charge of. Reassign.

<Other embodiments>
The content generation device 10 described above may be configured as a content generation system in which the content generation unit 17 (video generation engine 20) described above is realized by another device (for example, a PC, for example).

  Here, FIG. 10 is a diagram illustrating a configuration example of the content generation system. Note that, in the following configuration, a configuration that performs the same processing as that of the content generation device 10 described above is given the same name and symbol as the configuration described above, and a specific description thereof is omitted here.

  In the example of FIG. 10, the content generation system 50 is configured to include a content generation device 51 and one or a plurality of PCs 52 (PCs 52-1 and 52-2 in the example of FIG. 10). The content generation device 51 and the PC 52 are connected in a state where data can be transmitted and received through a communication network 53 such as the Internet or a LAN.

  The content generation device 51 includes an input unit 11, an output unit 12, a storage unit 13, a script charge assignment unit 14, a script generation unit 15, a delay adjustment unit 16, a communication unit 18, and a control unit 19. It is comprised so that it may have. The PC 52 includes a video generation engine 61 and communication means 62. The PC 52 may be, for example, a server, a television device having a video generation function / communication function, or a mobile terminal such as a smartphone, a tablet terminal, a game device, a camera, or a mobile phone.

  The content generation device 51 in FIG. 10 manages one or more PCs 52 as an information processing device for generating video content in advance. Each PC 52 can be identified by a network address such as an IP address. Each PC 52 has, for example, a video generation engine 61 that generates content using a single processor instead of a multiprocessor. However, the present invention is not limited to this. For example, each PC 52 includes a multiprocessor (a plurality of videos). Generation engine). The video generation engine 61 may have the function of the content generation unit 17 described above or may have a function as a TVML player.

  Similar to the content generation apparatus 10 described above, the content generation apparatus 51 uses the script charge assignment unit 14 to determine the number of PCs 52 to be used, which script is assigned to which PC 52, and in which order information is combined. Assign. The script generation unit 15 generates a script based on the contents assigned by the script assigning unit 14 and shifts the time based on the delay time (delay amount) adjusted by the delay adjustment unit 16. It transmits to PC52 which uses the same script. At this time, the communication unit 18 generates the video content handled by each PC 52, sets the generated video content generated by another PC as an input video, and uses the input video and the video content generated from the script. An instruction to synthesize in a predetermined order is also transmitted. The above instruction may be included in the script to be transmitted, or may be transmitted to each PC 52 in advance.

  The PC 52-1 and PC 52-2 receive the script file transmitted from the content generation device 51 by the communication unit 62, extract the script in charge from the received script file, and generate the video by the video generation engine 61. Do.

  For example, the output video 40-1 obtained by the PC 52-1 is input as a background video to the PC 52-2 at address 2 by the communication means 62 via the communication network 53. In this case, for example, video encoding or decoding processing may be included, and transmission may be performed not by the communication network 53 but by a dedicated line (video line) for transmitting a video signal. The PC 52-2 synthesizes the input background video and the video 40-2 generated by itself from the script, and outputs the final video content 41.

<Application example of content generation system>
FIG. 11 is a diagram illustrating an application example of the content generation system. A content generation system 70 illustrated in FIG. 11 includes the above-described content generation device 51, a television device 71, and a mobile terminal 72. The content generation device 51, the television device 71, and the mobile terminal 72 are connected in a state where data can be transmitted and received through a communication network 53 such as the Internet or a LAN.

  The television apparatus 71 includes the video generation engine 61 and the communication unit 62 described above, and further includes a display unit 81 that displays video content generated by the video generation engine 61 on a screen.

  In addition, the mobile terminal 72 includes the video generation engine 61, the communication unit 62, and the display unit 81 described above, and further, a camera (imaging) that captures content displayed on the screen of the display unit 81 of the television device 71. Means) 82.

  In the content generation system shown in FIG. 11, for example, the display screen of the television apparatus 71 is photographed through a camera 82 built in the mobile terminal 72, and the entire photographed image including the photographed video content is acquired as the main content. Further, the mobile terminal 72 can synthesize the video (sub-content) 83 generated by itself from the script received from the content generation device 51 via the communication network 53 in synchronization with the main content as the background video. Thereby, for example, the display means 81 of the mobile terminal 72 can provide an entertaining video that pops out of the display screen of the television device 71 taken by the character, for example.

  In this case, it is preferable that the content generation apparatus 51 shown in FIG. 11 performs specification of a camera work, a CG generation space (clipping plane), and the like. In addition, the content generation device 51 transmits a control script for the same CG character to the television device 71 and the mobile terminal 72.

  According to this embodiment, many characters can be displayed with a simple configuration. In addition, it is possible to provide entertaining video content in which a CG character jumps out of the screen. The above-described embodiments can be implemented by combining some or all of the embodiments. The script may be a script other than the TVML script.

<Execution program (content generation program)>
Here, the content generation apparatuses 10 and 51 described above include, for example, a CPU, a volatile storage medium such as a RAM (Random Access Memory), a non-volatile storage medium such as a ROM (Read Only Memory), a mouse, a keyboard, and a pointing device. The computer can include an input device such as a display unit for displaying images and data, and an interface for communicating with the outside.

  Accordingly, the functions of the content generation apparatuses 10 and 51 can be realized by causing the CPU to execute a program (content generation program) describing these functions. The program can also be stored and distributed on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  In other words, in the present embodiment, an execution program (content generation program) for causing a computer (hardware) to execute the processing in each configuration described above is generated, and the program is installed in, for example, a general-purpose PC or server. The content generation processing described above can be realized by cooperating the hardware described above and software including a program or the like.

  As described above, according to the present invention, content can be generated easily and in real time according to the complexity and number of CG objects. Specifically, in this embodiment, CG objects such as many characters can appear on one screen with a simple configuration. Further, even if the number of video generation engines increases or decreases, it can be flexibly handled. The number of video generation units required varies depending on the complexity of the CG character and the number of characters to be displayed, but the functions of the video generation units may be the same and can be easily added or deleted. Further, in the present embodiment, the materials handled by the respective video generation units are meaningful groups and do not require precise display timing.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10,51 Content production | generation apparatus 11 Input means 12 Output means 13 Storage means 14 Script assigning means 15 Script generation means 16 Delay adjustment means 17 Content generation means 18,62 Communication means 19 Control means 20,61 Video production engine 30 CG character 40 Video 41 Video content 50, 70 Content generation system 52 PC
53 Communication network 81 Display means 82 Camera (imaging means)

Claims (6)

In a content generation device that generates video content having one or more CG objects,
Based on the type and number of CG objects included in the video content to be generated, the number of video generation engines that generate the video content is assigned from a script, and a script assignment that assigns a script for each assigned video generation engine Means,
Script generating means for generating the script for generating the video content;
Using the script generated by the script generation means, the video generation engine assigned by the script charge assignment means generates the video content each of which is in charge, and the video content generated by another video generation engine And a content generation means for synthesizing the input video and video content generated from the script.   From the video content obtained by the content generation means, a delay time at the time of synthesizing the video content generated for each video generation engine is acquired, and based on the acquired delay time, the script generation means sends each video generation engine The content generation apparatus according to claim 1, further comprising a delay adjustment unit that adjusts a timing at which the script is transmitted. The script generation means outputs the same script to the video generation engine,
The content generation unit extracts only a script assigned to each video generation engine from the script, and generates video content in charge using the extracted script. Content generation device. In a content generation apparatus that generates video content having one or a plurality of CG objects using an information processing apparatus having a video generation engine connected via a communication network.
Script assignment assigning means for assigning the number of information processing devices to which a script is transmitted based on the type and number of CG objects included in the video content to be generated, and assigning a script in charge for each assigned information processing device;
Script generating means for generating the script for generating the video content;
The script generated by the script generation means is transmitted to the information processing apparatus assigned by the script charge assignment means, and the video content assigned to each information processing apparatus is generated, and the video generated by another video generation engine A content generation apparatus comprising: communication means for transmitting content as an input video and an instruction to synthesize the video content generated from the input video and the script.   From the video content obtained by the information processing apparatus assigned by the script charge assignment means, a delay time at the time of synthesis of the video content is acquired, and based on the acquired delay time, the communication means sends the information processing apparatus to the information processing apparatus. The content generation apparatus according to claim 4, further comprising a delay adjustment unit that adjusts a timing for transmitting the script. Computer
A content generation program for causing a content generation apparatus according to any one of claims 1 to 5 to function as each unit.