JP2008217447A - Content generation device and content generation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a content added with the optimum representation to human voice. <P>SOLUTION: This content generation device for generating the content of executing by a representation object a prescribed representation set based on feeling information obtained from the input human voice, has a reading-in means for reading a script of the preliminarily generated content in every unit, a sound recording means for recording the human voice of a user, a feeling estimation means for estimating feeling based on the human voice recorded by the sound recording means, a representation means for extracting preset representation information, corresponding to the feeling information comprising a classification and an intensity of feeling obtained by the feeling estimation means, and a script generation means for substituting the script generated based on the representation information obtained by the representation means, with the script read in by the reading-in means, thus solving the above problem. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a content generation device and a content generation program, and more particularly, to a content generation device and a content generation program for generating content with an optimal performance added to the real voice based on emotion information obtained from real voice data. .

  Conventionally, when video content creators produce video content such as news and sports information-providing programs, a script is created in advance in a predetermined description format before the video is generated, and this is made into dedicated software, etc. There is a mechanism capable of generating a video for performing a motion or camera work of a virtual object such as a computer graphics (CG) character or the like in accordance with a script by inputting it (for example, Patent Document 1). reference.).

  In the technology disclosed in Patent Document 1 described above, a program is produced using TVML (TV program Marking Language). Here, TVML is an object-based description language for producing a television program. TVML describes the video and audio of a TV program divided into materials and scripts (production contents). If a program script is described, software that runs on a personal computer or the like reads it and immediately reads the TV program. It can be viewed (presented) as a program. By using TVML, it is possible to specify the motion and camera work of the CG character each time and control it ad lib.

  Also, in program production using TVML, a script of a program produced by the producer, a program production engine (APE: Automatic Production Engine) described in the script, material data used for the produced program, and the like are used. To produce a program. Here, the above-mentioned program production engine is a CG character such as a program presenter or a performer appearing in a program, or one operation unit in a program, such as “title display”, “zoom-in”, “CG character operation”, etc. These events are predefined, and by using this program production engine, it is possible to efficiently realize program production of a predetermined genre such as news, variety, sports, drama and the like.

  Here, when video content for TV programs, for example, is generated using CG or the like, a real voice is often used instead of a synthesized voice in order to ensure the quality of the CG character who is a performer. In this case, the speech of the performer is recorded in advance, and CG video content with the expression and behavior of the CG character added manually according to these voices is generated. Alternatively, a CG video content to which a facial expression or behavior of a CG character is added is generated in advance, and then a CG video content is generated by a so-called post-recording (after recording) process in which a real voice is assigned while reproducing the CG video.

As a method other than the above, there is a method in which CG video content is once created with synthesized voice using synthesized voice by a voice synthesis application, and then replaced with real voice while reproducing the CG video content ( For example, see Patent Document 2.)
JP 2005-318254 A JP 2004-71013 A

  However, in the case of the above-described prior art, when the real voice is recorded in advance and the voice is heard and then the expression or gesture of the CG character is added, the lines cannot be easily changed. In addition, adding a behavior (production) such as a facial expression or a gesture to a production object such as a character corresponding to a line manually is time-consuming and time-consuming.

  In addition, since the expression and gesture of the CG character have already been determined, it is necessary to assign a real voice so as to match it, but it is not possible to see the CG video content containing the actual speech until the real voice is assigned. In addition, the lines cannot be easily changed, and it is necessary to manually add an effect to the effect object.

  Also, with the technique disclosed in Patent Document 2 described above, which is a prior art, CG video content can be created in advance using synthesized speech, so the entire CG video content can be confirmed in advance after changing the lines. . However, in post-recording processing after CG video content generation using synthesized speech, it is necessary to manually adjust the post-recording sound while listening to the post-recording speech in order to achieve an appropriate facial expression, gesture, etc. that matches the post-recording real voice. There is.

  The present invention has been made in view of the above-described problems, and based on emotion information obtained from real voice data, a content generation apparatus and a content generation program for generating content with an optimal presentation added to the real voice The purpose is to provide.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

  The invention described in claim 1 is generated in advance in a content generating apparatus that generates content in which a predetermined effect set based on emotion information obtained from input real voice data is performed on a production target object. Reading means for reading a script of content for each predetermined unit, recording means for recording a user's real voice, emotion estimation means for estimating an emotion from real voice data recorded by the recording means, and the emotion estimation means Production means for extracting production information set in advance corresponding to emotion information consisting of the type and strength of emotion obtained, and a script generated based on the production information obtained by the production means And a script generation means for replacing the script read by the above.

  According to the first aspect of the present invention, based on the emotion information obtained from the real voice data, it is possible to generate content in which an optimal presentation is added to the real voice. This also makes it possible to quickly produce a program with an appropriate performance.

  The invention described in claim 2 is characterized in that the effect means includes, as the effect information, information related to an expression or an action of the effect object.

  According to the second aspect of the present invention, it is possible to cause the production target to produce a highly accurate production corresponding to the emotion information obtained from the real voice.

  According to a third aspect of the present invention, when the production means sets production information, the production means sets a lower limit value and an upper limit value for the expression or action of the production object for each type and intensity of the emotion. When setting and extracting the effect information, a value is randomly determined between the lower limit value and the upper limit value.

  According to the third aspect of the invention, not only a specific operation but also a general operation can be performed within a certain range. Thereby, it is possible to perform an effect that does not make the user bored.

  According to a fourth aspect of the present invention, the recording means reproduces the pre-generated content for recording the user's real voice and stops the content immediately before recording the real voice. It is characterized by having.

  According to the fourth aspect of the present invention, it is possible to record a real voice for each necessary portion while reproducing a script. In addition, since the user or the like can record the sound while watching the reproduced content, it is possible to realize the input of a real voice with an appropriate emotion along the flow of the content.

  The invention described in claim 5 is for causing a computer to execute content generation processing for generating content in which a predetermined effect set on the basis of emotion information obtained from input real voice data is performed on a production target object. In the content generation program, a reading process for reading a script of content generated in advance for each predetermined unit, a recording process for recording the user's real voice, and emotion estimation from the real voice data recorded by the recording process Based on the emotion estimation process, an effect process for extracting effect information set in advance corresponding to emotion information including the type and strength of the emotion obtained by the emotion estimation process, and the effect information obtained by the effect process A script that replaces the script generated by To execute a process on the computer.

  According to the fifth aspect of the present invention, based on the emotion information obtained from the real voice data, it is possible to generate content in which an optimal presentation is added to the real voice. This also makes it possible to quickly produce a program with an appropriate performance. Furthermore, content generation can be easily realized by installing an execution program in a computer.

  According to the present invention, based on emotion information obtained from real voice data, it is possible to generate content in which an optimal presentation is added to the real voice.

<Outline of the present invention>
In the present invention, for example, in the production of content having a CG video for a television program and the like, when a real voice (live voice) is used by post-recording, facial expressions, gestures, etc. for a production object such as a CG character according to the emotion of the real voice The behavior (production) of is automatically added.

  Specifically, the present invention relates to the field of video content production mainly for television programs and the like. When producing video content for television programs using CG, for example, CG which is a performer of video content. In order to make the character speak, there is a case where the voice reality is improved by using the real voice. In this case, the emotional information of the real voice is used to automatically express the expression and gesture of the CG character that matches the emotion of the real voice. It is possible to add to.

  Hereinafter, a preferred embodiment of a content generation device and a content generation program according to the present invention having the above-described features will be described in detail with reference to the drawings.

  In the present embodiment, a program is used as an example of content. In addition, a CG character appearing in a program is used as an example of the production target. Furthermore, in this embodiment, TVML is used as an example of a script used for program generation or presentation. The present invention is not limited to TVML, and an expression format other than TVML may be used.

<Content generation device: functional configuration example>
FIG. 1 is a diagram illustrating a configuration example of a content generation apparatus. 1 includes an input unit 11, an output unit 12, a storage unit 13, a script generation unit 14, a recording processing unit 15, an emotion estimation unit 16, an effect unit 17, and a reproduction unit. 18, a transmission / reception means 19, and a control means 20.

  The input means 11 accepts various inputs for implementing the apparatus, such as content generation instructions, voice input instructions, voice (real voice), emotion estimation instructions, script generation instructions, playback instructions, etc. from users and producers. . Note that the input unit 11 includes, for example, a keyboard, a pointing device such as a mouse, a voice input device such as a microphone, and the like.

  The output unit 12 displays the instruction content input by the input unit 11, the recorded sound, the content generated based on the content of the instruction, the editing content such as video and audio related to the content, the content of the editing result, etc. Or output voice data such as recorded real voice. The output unit 12 includes a display, a speaker, and the like.

  The storage means 13 is input voice, a plurality of material data consisting of various data such as images and videos for generating contents, sound, text data, generated contents, emotion estimated by the emotion estimation means 16. , The setting contents of the production in the production means 17 (for example, the contents of the CG character production), the script generation result in the script generation means 17 and the like are stored.

  In addition, the storage means 13 stores a TVML script corresponding to a script, an APE script as a program production engine, a TVML script for program setup that can be used when generating a program and a dialogue scene, program generation data (template), and a program. Dialog scene generation data (template), a dialog response dictionary (response script), and the like for realizing a dialog scene with an appearing character are stored.

  Here, the TVML script for program setup mentioned above is what kind of studio set, prop, lighting, and performer as a program (relationship between performers (gender, age, personality, occupation (singer, commentator, comedian, etc.)) The default setting information of the program, such as whether to use audio, etc. The interactive response dictionary stores, for example, expected response data for inquiries from viewers. Accordingly, the storage means 13 is a collection of character information, image information, and other information such as a database, for example, and is structured so that various types of information stored in the inquiry can be searched. It may be configured.

  The storage unit 13 can also acquire the various data described above from the external device connected to the Internet or a communication network represented by a communication line or the like via the transmission / reception unit 19.

  The script generation unit 14 inputs an original TVML script using synthesized speech stored in advance, and creates an effect TVML script describing an effect such as behavior of a CG character included in the original content. Is incorporated immediately before the TVML script corresponding to the instruction to turn the CG character with the synthesized voice. Further, the script generation unit 14 performs a replacement process with the TVML script so that the CG character speaks with a real voice in which the synthesized voice is recorded in advance.

  That is, the script generation unit 14 replaces the production script obtained by the emotion estimation unit 16 with the newly generated script when the production of the CG character is already set. If there is no CG character production, the generated script is added. In this case, a speech composed of voice data recorded by a user or the like is used.

  The recording processing means 15 records a predetermined word or sentence with a voice (such as a real voice) from a user or the like obtained from an input means such as a microphone. The recording processing unit 15 generates file data in a predetermined file format (for example, WAVE, AIFF, MP3, au, WMA, ram, AAC, etc.), and the generated audio data is stored in the storage unit 13 or the script generation unit 14. Etc.

  The emotion estimation means 16 estimates the emotion for the real voice (human voice etc.) recorded by the recording processing means 15. Further, the behavior (effect) of the CG character is set with respect to preset conditions such as an estimation result (emotion type, strength, etc.) in the emotion estimation means 16.

  Here, the emotion estimation means 16 inputs, for example, recorded voice data such as speech, and as a result of emotion estimation, the type of emotion of the voice (for example, normal, anger, joy, sadness, etc.) and its strength (degree) , Level). Note that the emotion type and strength estimation in the emotion estimation means 16 can be analyzed using, for example, a preset emotion estimation engine (for example, ST). A specific estimation method in the emotion estimation means 16 will be described later.

  The rendering unit 17 selects a TVML script that expresses the behavior of a CG character or the like appearing in the content based on the estimation result obtained by the emotion estimation unit 16 from the data group stored in the storage unit 13. To the script generation means 14. Further, the production means 17 inputs the setting using an input form such as a GUI so that the user can easily perform the production setting for each kind of the emotion of the real voice obtained as the estimation result and its strength, for example. Can do. Thereby, it is possible to input an appropriate effect easily and efficiently. Detailed description of the effect setting in the effect means 17 will be described later.

  The reproduction means 18 reads the original TVML script generated in advance, reproduces the video up to immediately before the user or the like records the real voice, and stops at that location. Note that “immediately before recording the real voice” includes, for example, immediately before a portion where the synthesized speech included in the original TVML script is output, but is not limited thereto. As a result, the user or the like can record the real voice for each necessary location using the recording processing means 15 while reproducing the script. In addition, since the user or the like can record the sound while watching the reproduced content, it is possible to realize the input of a real voice with an appropriate emotion along the flow of the content.

  Further, the reproducing means 18 reproduces the program based on the TVML script including the final CG character effect according to the present embodiment obtained from the script generating means 14 and the corresponding voice data composed of the real voice.

  Note that the playback means 18 has a function such as a TVML player. Here, the TVML player is software that can read a script described in TVML and output a program video, audio, or the like in real time. Also, the TVML player generates a studio set with real-time CG, and displays a place where a CG character such as a program presenter or a performer appearing in the CG studio set speaks a line described in the TVML script and performs. It has the function to do.

  In addition, the TVML player also generates video and audio for programs by generating video playback, text fonts, image title display and superimposition, audio data file playback BGM playback, voice narration, etc. in real time. It has the function to do. The playback means 18 can play back an efficient and highly accurate program or dialog scene by using these functions.

  The transmission / reception means 19 is a TVML script based on an external device connected via a communication network represented by the Internet or a communication line, production setting contents, prerecorded real voice data, emotion estimation engine, material data A communication interface for receiving APE, various script groups, and the like. The transmission / reception means 19 can also transmit the TVML script, the program, the recorded voice data, the newly set production content, etc. generated by the present embodiment to other external devices etc. via the communication network. .

  The control unit 20 controls the entire functional configuration in the content generation apparatus 10. Specifically, the control unit 20 performs a recording process of the real voice based on the input information from the user input by the input unit 11, performs an emotion estimation process from the acquired real voice data, or calculates CG from the estimation result. Various control processes such as setting of effects such as the behavior of the character, generation of scripts based on the set effects, and reproduction of the generated scripts are performed.

  Thereby, based on the emotion information obtained from the real voice data, it is possible to generate content in which an optimal presentation is added to the real voice. Specifically, for example, when performing post-recording processing, it is possible to automatically add an effect process that expresses a facial expression, a gesture, or the like of a CG character suitable for a real voice emotion based on emotion information possessed by the real voice. . Therefore, in the content generation process, it is possible to automatically generate a CG video content to which an effect such as an optimum behavior is added without replacing a speech by replacing the speech of the synthesized voice of the CG character with a real voice in the after-recording process. It becomes possible.

<Outline of content processing>
Next, an outline of content processing up to content generation using the content generation apparatus 10 described above will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of an outline of content generation processing.

  In the example shown in FIG. 2, when content that a CG character talks or the like is input as a base TVML script using synthesized speech, the script generation unit 14 performs script reading processing 31 and script replacement / addition processing 32. And do.

  Specifically, the script generation means 14 reads a script, which is a description of a performer speaking in a synthesized voice, as a character string for each predetermined unit (for example, one line or one command), and in the read script If there is a TVML script corresponding to an instruction to turn a CG character with a predetermined synthesized voice, the TVML script up to that point is played back by a TVML player as the playback means 18 in the recording processing means 15. In addition, playback is stopped immediately before the synthesized speech, and a person who speaks the synthesized speech by the recording processing is spoken toward the microphone as the input means 11 to perform the recording processing of the real voice.

<Specific example of recording process>
Here, a specific example of the recording process in the recording processing means 15 will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example for explaining the contents of the recording process. FIG. 3A shows an example of a voice input form for performing a recording process, and FIG. 3B shows an example of a playback screen of the original TVML script.

  The voice input form 40 shown in FIG. 3A is configured to have a menu display area 41, a recording / editing area 42, and a button area 43. Also, in FIG. 3B, a CG character 51 is displayed on the playback screen 50, and is in a stopped state immediately before the speech of the synthesized speech is spoken.

  The menu display area 41 reads the original TVML file or the recorded voice data file, displays the file name of the read TVML file, or adds the file name to the recorded voice data and stores the voice data. Are stored in a predetermined storage area, and various options relating to a predetermined file are set.

  Further, the recording / editing area 42 is for a CG character that produces a speech of “fun” as shown in FIG. 3 (b) with synthesized speech, as shown in FIG. Using the input means 11, the recording button 44 in the recording editing area 42 is pressed and a speech (“fun” in FIG. 3) is input within a predetermined time. At this time, if the recording button 44 is pressed, the CG character on the screen shown in FIG. 3B can also be operated, so that an optimal real voice can be input at an appropriate timing while observing mouth movements and gestures. Can do.

  Furthermore, since the voice output time of the synthesized voice set in advance is displayed in the status display area 45, the user or the like can easily grasp the length of the real voice to be recorded and can adjust the length. it can.

  In the recording / editing area 42, the recorded result is previewed by sound or video, the recording is canceled, the next line (next line) included in the original TVML script is recorded, and a caption (title) is displayed. , Etc.) can be set.

  The button area 43 includes a “stop button” for canceling the recording process, an “all preview button” for previewing all the recorded audio, and a predetermined audio file such as the WAVE file format, for example, the storage means 13 and the like. A “save button” for saving in this area, a “compress and save button” for compressing and saving the audio file, and the like. The button area 43 includes a spare area and various data display areas.

  In the recording process described above, when recording for one line is completed, for example, a dialog message “Recording is completed” is displayed, or after the dubbing of the entire TVML script file is completed, for example, When a dialog message such as “Recording has been completed.” Is displayed, or when the post-recording file is saved, a dialog such as “File saved” may be displayed. Thereby, the user 33 etc. can input the voice with high accuracy while referring to the corresponding screen.

  Next, in FIG. 2, a voice data file (for example, a WAVE file) such as a recorded speech input by the input means 11 such as a microphone is input to the emotion estimation means 16 and analyzed by emotion estimation processing. Type of emotion (normal, anger, joy, sadness) and emotion information such as strength.

<Emotion estimation process>
Here, the emotion estimation process will be described. In the emotion estimation process, for example, from voice data such as input real voice, the intensity, tempo, and inflection of the voice are detected, and the detected intensity, tempo, and pattern of the amount of change in the time axis direction such as intonation, The corresponding emotional state is output based on the emotional state accumulated in advance in association with the pattern.

  In addition, for example, voice recognition such as voice volume, voice waveform, voice pitch, or phoneme is performed, and the result is compared with a preset threshold value corresponding to the above-described voice conditions. It is possible to use a method for estimating the emotion of the person, a method for performing morphological analysis on the content of the utterance, and estimating the emotion of the utterance using a preset emotion dictionary from the result of the speech recognition.

  In the emotion estimation process, for example, an emotion for a real voice can be estimated using a preset emotion estimation engine. Here, for example, ST (Sensitivity Technology) can be used as the emotion estimation engine. ST is a software technology that allows a computer to understand and react to human sensibility information. Specifically, ST's emotion information (eg, anger, joy, sadness, normal, laughter) For example, a continuous response scenario system corresponding to the emotional transition of the speaker can be constructed by adding the excitement etc.) to the word information obtained through voice recognition or the like.

  The above-described emotion estimation method is a well-known technique. For example, Japanese Patent Laid-Open No. 5-12023, Japanese Patent Laid-Open No. 9-22296, Japanese Patent Laid-Open No. 11-1119791, Japanese Patent Laid-Open No. 2002-91482, etc. It is shown.

  In FIG. 2, estimated emotion information (emotion type, emotion strength, etc.) is output based on the emotion information estimated by the emotion estimation process by the emotion estimation means 16 described above. The output emotion information obtains a TVML script for performing behavior corresponding to the emotion information from the CG character effect script group stored in advance in the storage means 13 by the setting process by the effect means 17.

<Direction setting>
Here, the setting method of the production | presentation of the CG character accumulate | stored in the storage means 13, such as CG character production DB, is demonstrated using figures. FIG. 4 is a diagram illustrating an example of the contents of the effect setting. FIG. 5 is a diagram showing an example of the contents of effects set by the effect setting input form corresponding to FIG.

  The effect setting process in the effect means 17 is based on the result obtained by the emotion estimation process in the emotion estimation means 16, with a facial expression and a face angle (facing up, facing down, etc.), nodding, A TVML script that generates a behavior such as shaking the head sideways is set according to the estimation result (for example, emotion type, emotion strength, etc.).

  In the present embodiment, there is an effect setting input form 60 for allowing the user to easily set the effect when setting the effect. The effect setting input form 60 is specifically configured to have a tag display area 61, various setting areas 62, and a button area 63.

  The tag display area 61 includes a type of emotion (normal, angry, joy, sadness, etc.) included in emotion information obtained by the emotion estimation means 16 set in advance, and emotion strength (level 1, level 2, level 3). Etc.) tags are formed so that each setting area 62 can be displayed. In the various setting areas 62, for example, “expression (type, degree)”, “face orientation (vertical, horizontal)”, “nodding (on / off, number of times, speed) for each emotion type and level. ) "," Swing (no / no, number of times, speed, degree of swing) "," user setting (pause name, speed, etc.) ", etc. can be set.

  After the setting, the setting is performed by pressing the “setting button” in the button area 63, and for example, a CG character effect DB according to the type and level of each emotion as shown in FIG. 5 is accumulated. In the button area 63, pressing the “cancel button” cancels the setting so far.

  Further, FIG. 5 shows the contents and the degree of the behavior (effect) added according to each emotion information as an example of the setting contents. Specifically, expressions, expression levels, face positions, actions, etc. are set according to the type and strength of emotion. That is, an effect script is generated based on the set content.

  In FIG. 5, for example, “nodding”, “large nodding”, and “shake the head to the side” can be determined at random by random numbers or not. For example, when the emotion type is “joy”, a predetermined range is set for “expression level” at any level (level 1, level 2, level 3,...) A value can be determined randomly by a random number or the like within the range.

  As described above, the behavior added according to the emotion information is, for example, an expression and its degree, an angle for turning the face up or down, a nod, and an action of shaking the neck sideways, but the present invention is not limited to this. Is not to be done. As for the degree of facial expression, a lower limit value and an upper limit value can be set for each emotion level, and the value can be randomly determined each time. In addition, nodding and shaking motion of the head can be determined randomly each time whether or not to add the motion. As a result, the CG character can be caused to perform a general purpose motion within a certain range as well as a specific motion. Thereby, it is possible to perform an effect that does not make the user bored.

  In addition to the effects shown in FIG. 5, it is possible to set effects objects in a virtual space such as studio lighting, camera work, and switching in a program. As a result, it is possible to provide high performance content.

  In the effect process in FIG. 2, the behavior of the CG character's facial expression, gesture, etc. suitable for emotion is obtained from the CG character effect DB as shown in FIG. The selected TVML script is generated, and is output to the script replacement / addition process 32 of the script generation unit 14.

  Further, in the script replacement / addition process 32, the script is replaced or added based on the CG character presentation information obtained by the presentation means 17. The script replacement / addition process 32 inputs the real voice obtained by the recording process described above, generates a TVML script using the real voice, is reproduced by the TVML player as the reproduction means 18, and the content 34 is output. .

<Script generation example>
Here, an example of a script generated according to the present embodiment will be described with reference to the drawings. FIG. 6 is a diagram illustrating an example of a script generated according to the present embodiment. 6A shows an example of the original TVML script, and FIG. 6B shows an example of the TVML script generated according to the present embodiment. Note that row numbers are assigned to the left side of FIGS. 6A and 6B for convenience.

  In the script generation in the present embodiment, as shown in FIG. 6A, first, as shown in FIG. 6A, a predetermined command (for example, (05 in FIG. ) Line))) is detected, the script up to that point is displayed on the TVML player, and the CG character is stopped.

  Therefore, the user or the like records the same speech as the synthesized speech. The recorded real voice is stored with a unique file name (for example, voice001.wav in the example of FIG. 6).

  Next, emotion estimation based on the recorded real voice is performed, and the type and level of emotion are output (in the example of FIG. 6, the emotion is “happy” and the level is “2”).

  Next, a TVML script that determines the expression and degree of the CG character is generated based on the emotion information. (In the example of FIG. 6, the expression is happy and the degree is 0.6). Further, the behavior of the CG character is determined based on the emotion information, and the TVML script is generated (in the example of FIG. 6, the face angle is turned upward by 7 degrees, and the nodding motion is selected (whether or not is randomly selected). 0.5) in some cases).

  Further, the position of the (05) line of the original script shown in FIG. 6A is replaced and added as shown in the (05) to (08) lines as shown in FIG. 6B. Then, the result is output as a TVML script.

  Note that the above-described content generation is performed up to the last line of the input TVML script, and finally the TVML script using real voice temporarily stored in a memory or the like is output.

  As described above, based on the emotion information obtained from the real voice data, it is possible to generate the content with the optimal presentation added to the real voice based on the emotional information obtained from the real voice data. Specifically, the entire program can be created once while changing the speech using synthesized speech, and a real voice can be recorded at each playback location while the created program is played. In addition, by simply recording the real voice, the behavior of the CG character in accordance with the recorded emotion of the real voice is automatically added, and a program with an appropriate performance can be automatically generated.

<Content generation program>
Here, the content generation device 10 described above can also generate content in the present invention with the dedicated device configuration described above, but generates an execution program for causing a computer to execute the processing in each configuration, for example, A content generation process can be realized by installing a program in a general-purpose personal computer or server.

<Hardware configuration>
Here, an example of a hardware configuration of an executable computer in the present invention will be described with reference to the drawings. FIG. 7 is a diagram illustrating an example of a hardware configuration capable of realizing the content generation processing according to the present invention.

  7 includes an input device 71, an output device 72, a drive device 73, an auxiliary storage device 74, a memory device 75, a CPU (Central Processing Unit) 76 for performing various controls, and a network connection device. 77, which are connected to each other by a system bus B.

  The input device 71 includes a keyboard and a pointing device such as a mouse operated by a user, a voice input device such as a microphone, and the like, and inputs various operation signals such as execution of a program from the user. The output device 72 has a display for displaying various windows and data necessary for operating the computer main body for performing the processing in the present invention, a speaker for outputting sound, and the like, and the program is controlled by a control program of the CPU 76. Execution progress, results, etc. can be displayed or voice output.

  In the present invention, the execution program installed in the computer main body is provided by a recording medium 78 such as a CD-ROM. The recording medium 78 on which the program is recorded can be set in the drive device 73, and the execution program included in the recording medium 78 is installed from the recording medium 78 to the auxiliary storage device 74 via the drive device 73.

  The auxiliary storage device 74 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, etc., and perform input / output as necessary.

  The memory device 75 stores an execution program or the like read from the auxiliary storage device 74 by the CPU 76. The memory device 75 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

  The CPU 76 controls processing of the entire computer, such as various operations and input / output of data with each hardware component, based on a control program such as OS (Operating System) and an execution program stored in the memory device 75. Each processing can be realized. Further, the CPU 76 can acquire various types of information necessary during execution of the program from the auxiliary storage device 74, and the CPU 76 can also store processing results and the like.

  The network connection device 77 obtains an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program or an execution in the present invention The program itself can be provided to other terminals.

  With the hardware configuration as described above, a content generation process can be realized efficiently at a low cost without requiring a special device configuration. Further, the content generation process can be easily realized by installing the program.

<Content generation processing>
Next, a content generation processing procedure by the execution program in the present invention will be described using a flowchart. In the following description of the processing, “program” is used as an example of content, but the present invention is not particularly limited. In addition, the TVML script generated in advance shown below includes a script for outputting synthesized speech.

<Content generation processing procedure>
FIG. 8 is a flowchart showing an example of a content generation processing procedure in the present embodiment. In FIG. 8, first, one line of an existing TVML script is read as a character string (S01).

  Here, it is determined whether or not the processing up to the last line has been completed (S02). If it has not been completed (NO in S02), it is determined whether or not the read line is a predetermined script (S02). S03). Here, the predetermined script is determined based on whether or not it includes a script command (for example, “talk” or the like) that makes a predetermined CG character generated by synthesized speech appear.

  If the script is not a predetermined script in the process of S03 (NO in S03), the read TVML script is added to the buffer (memory) or the like (S04), the process returns to S01, the next script is read, and the subsequent processes I do.

  In the process of S03, if the read script line is a predetermined script (YES in S03), the read TVML script is added to the end of the buffer, and the TVML script existing in the buffer is reproduced by the TVML player. (S05). In the playback process using the above-described TVML script, the playback is stopped immediately before the CG character is beaten with the existing synthesized speech.

  Next, a recording process is performed in which a user or the like speaks the same speech spoken by synthesized speech, and the real voice is recorded by speech input means such as a microphone (S06). In the recording process, the real voice is stored in a file format such as a WAVE file.

  Next, the recorded voice data of the speech is input to, for example, the emotion estimation engine (ST), and the WAVE file is analyzed by ST, and the type of emotion (normal, angry, joyful, sadness) of the voice and the strength thereof are determined. The emotion estimation process to be output is performed (S07).

  Also, based on the emotion estimation result obtained by the processing of S07, the production contents expressing the behavior of the CG character suitable for the emotion, such as facial expressions and gestures, are selected, and a TVML script corresponding to the contents of the selected production is generated. An effect process is performed (S08).

  Next, the TVML script obtained by the effect processing of S08 described above is incorporated immediately before the TVML script corresponding to the instruction to make the CG character struck by the synthesized voice of the original TVML script, and the real CG character recorded from the synthesized voice. Then, the corresponding TVML script is replaced (S09).

  After the process of S09 is completed, the data stored in the reproduction buffer is cleared (S10), and the process returns to S01 to read the interrupted original TVML script from the next line of the interrupted part and the above-described processes. I do.

  In the process of S02, when the process on the last line is completed (YES in S02), the newly generated TVML script is output and the process is terminated (S11).

  As described above, based on the emotion information obtained from the real voice data using the execution program of the present invention, it is possible to easily generate the content with the optimal performance added to the real voice.

  As described above, according to the present invention, based on emotion information obtained from real voice data, it is possible to generate content in which an optimal presentation is added to the real voice. Specifically, the entire program can be created once while changing speech using synthesized speech. In addition, a real voice can be recorded for each playback location while the created program is played back. In addition, by simply recording the real voice, the behavior of the CG character in accordance with the recorded emotion of the real voice is automatically added, and a program with an appropriate performance can be automatically generated.

  It should be noted that the content of the effect in the present invention is not limited to the behavior for the CG character as described above, and may be an effect for the effect object in the virtual space such as studio camera work or illumination.

  In addition, the content generated by the content generation processing in the present invention can be used as a so-called blog TV in which a user uploads a script created by the user to his channel and publishes it on the Internet as a program, It can be widely applied as program generation, community program production, educational tools, and advertising tools.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

It is a figure which shows one structural example of a content production | generation apparatus. It is a figure which shows an example of the outline | summary of a content production | generation process. It is a figure which shows an example for demonstrating the content of a recording process. It is a figure of an example for demonstrating the content of production | presentation setting. It is a figure which shows an example of the content of the effect set by the effect setting input form corresponding to FIG. It is a figure which shows an example of the script produced | generated by this embodiment. It is a figure which shows an example of the hardware constitutions which can implement | achieve the content generation process in this invention. It is a flowchart which shows an example of the content production | generation procedure in this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Content production | generation apparatus 11 Input means 12 Output means 13 Accumulation means 14 Script production means 15 Recording processing means 16 Emotion estimation means 17 Production means 18 Reproduction means 19 Transmission / reception means 20 Control means 31 Script reading process 32 Script substitution and addition process 33 34 Contents 40 Voice input form 41 Menu display area 42 Recording edit area 43, 63 Button area 44 Record button 45 Status display area 50 Playback screen 51 CG character 60 Production setting input form 61 Tag display area 62 Various setting areas 71 Input device 72 Output Device 73 Drive device 74 Auxiliary storage device 75 Memory device 76 CPU
77 Network connection device 78 Recording medium

Claims (5)

In a content generation apparatus that generates content in which a production subject performs a predetermined effect set based on emotion information obtained from input voice data,
Reading means for reading a script of content generated in advance for each predetermined unit;
Recording means for recording the user's real voice;
Emotion estimation means for performing emotion estimation from the real voice data recorded by the recording means;
Production means for extracting production information set in advance corresponding to emotion information consisting of emotion type and strength obtained by the emotion estimation means;
A content generation apparatus comprising script generation means for replacing a script generated based on the presentation information obtained by the presentation means with a script read by the reading means. The production means is
The content generating apparatus according to claim 1, wherein the effect information includes information related to an expression or an action of the effect object. The production means is
When setting the production information, a lower limit value and an upper limit value are set for the expression or motion of the production object for each type and strength of the emotion, and when the production information is extracted, the lower limit is set. 3. The content generation apparatus according to claim 2, wherein a value is randomly determined between a value and the upper limit value.   2. The recording unit according to claim 1, further comprising a reproducing unit that reproduces the previously generated content to record the user's real voice and stops the content immediately before recording the real voice. 4. The content generation device according to any one of 3 above. In a content generation program for causing a computer to execute a content generation process for generating content in which a production subject performs a predetermined effect set based on emotion information obtained from input voice data,
A read process for reading a script of content generated in advance for each predetermined unit;
A recording process for recording the user's real voice;
Emotion estimation processing for estimating emotions from real voice data recorded by the recording processing;
Production processing for extracting production information set in advance corresponding to emotion information consisting of emotion type and strength obtained by the emotion estimation processing;
A content generation program for causing a computer to execute a script generation process for replacing a script generated based on the presentation information obtained by the presentation process with a script read by the reading process.