JP6641045B1 - Content generation system and content generation method - Google Patents

Abstract

A content is generated based on an uttered voice. A content generation system includes a voice acquisition unit that acquires voice information of a speaker, a conversion unit that converts the voice information into text information, an analysis unit that analyzes the text information, and an analysis of the text information. A scenario information generating unit for generating scenario information based on the result, a material image database storing material images, and dividing the scenario information into predetermined editing units, and storing the scenario information in the material image database in the editing units And a moving image generating unit that generates a moving image by linking the material images associated with each of the editing units. And [Selection diagram] FIG.

Description

  The present invention relates to a content generation system and a content generation method that can generate content based on, for example, an uttered voice.

  2. Description of the Related Art Conventionally, various methods have been proposed for easily generating moving images (including animations) and comics.

  For example, in Patent Document 1, when arbitrary moving image data is designated, an image of an important scene is automatically extracted from a plurality of images included in the moving image data, and the extracted images are arranged and converted into a cartoon. It is described that an image effect such as a balloon is automatically arranged on the image.

  Further, for example, in Patent Document 2, a video frame which is a feature of switching video contents from a video signal is extracted as a frame image which is a constituent unit of a comic based on an input video signal and an audio signal corresponding thereto, Detecting a person region of a person appearing in the frame image, generating a speech of the person speech-recognized from the voice signal as character string information, and inserting a balloon with the character string information inserted as speech content of the person, It describes superimposing on the frame image based on the person area.

JP 2014-6912 A JP-A-2003-85572

  As described above, there is a technique for extracting an image from a moving image to generate a comic. However, it is not possible to generate moving images, comics, and other contents based on speech sounds.

  The present invention has been made in view of such a situation, and an object of the present invention is to enable content to be generated based on a speech sound.

  The present invention includes a plurality of means for solving at least a part of the above-described problems, and examples thereof are as follows.

  In order to solve the above problems, a content generation system according to an aspect of the present invention includes a voice acquisition unit that acquires voice information of a speaker, a conversion unit that converts the voice information into text information, and analyzes the text information. A scenario information generating unit that generates scenario information based on the analysis result of the text information, a material image database storing material images, and classifying the scenario information into predetermined editing units. An associating unit that associates the material images stored in the material image database for each unit to generate association information, and a moving image that generates a moving image by linking the material images associated with each edit unit And an image generation unit.

  The analysis unit can specify 5W1H in the content as the analysis result of the text information.

  The analysis unit can specify the emotion of the character in the content.

  The analysis unit can specify an emotion of a character in the content based on the audio information.

  The analysis unit may specify metadata of the speaker based on the audio information.

  The associating unit may generate the association information by associating the material images stored in the material image database with respect to each editing unit based on the metadata of the speaker.

  The associating unit may generate the association information by associating the material images of a predetermined style stored in the material image database for each of the editing units.

  The scenario information generating unit may generate the scenario information in which at least one of a comment, a behavior, a situation, and an emotion regarding a character in the content is arranged in a time series based on the analysis result of the text information. it can.

  The content generation system may include an operation unit that receives an operation input from a user, and a correction unit that corrects at least one of the scenario information and the correspondence information based on the operation input from the user.

  The correction unit may learn a correction result for at least one of the scenario information and the correspondence information, and may update the material image database based on the learning result.

  The correction unit may correct the correspondence information by associating the material image input by the user with the editing unit of the scenario information.

  The content generation system may include a collection unit that collects image data as free material and registers the image data as the material image in the material image database.

  A content generation method according to another aspect of the present invention includes a voice obtaining step of obtaining voice information of a speaker, a conversion step of converting the voice information into text information, an analysis step of analyzing the text information, A scenario information generating step of generating scenario information based on the analysis result of the text information, and dividing the scenario information into predetermined editing units, and associating the material images stored in the material image database for each of the editing units. And a moving image generating step of connecting the material images associated with each editing unit to generate a moving image.

  According to one embodiment of the present invention, it is possible to generate a content based on a speech sound.

  Problems, configurations, and effects other than those described above will be apparent from the following description of the embodiments.

FIG. 1 is a diagram showing a configuration example of a content generation system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration example of a functional block configuring the terminal device. FIG. 3 is a block diagram illustrating a configuration example of a functional block configuring the server device. FIG. 4 is a diagram illustrating an example of the data structure of the scenario information. FIG. 5 is a diagram illustrating an example of the data structure of the material image DB. FIG. 6 is a diagram illustrating an example of the data structure of the correspondence information. FIG. 7 is a flowchart illustrating an example of the content generation process. FIG. 8 is a flowchart illustrating an example of the correction process. FIG. 9 is a diagram illustrating a display example of a content display screen. FIG. 10 is a block diagram illustrating a configuration example of a computer.

  An embodiment according to the present invention will be described below with reference to the drawings. In all the drawings for describing the embodiments, the same members are denoted by the same reference numerals in principle, and the repeated description thereof will be omitted. Also, in the following embodiments, the components (including element steps, etc.) are not necessarily essential, unless otherwise specified or considered to be indispensable in principle. Needless to say. In addition, when saying “consisting of A”, “consisting of A”, “having A”, or “including A”, other elements are excluded unless otherwise specified. Needless to say, it doesn't. Similarly, in the following embodiments, when referring to the shapes, positional relationships, and the like of the components, the shapes are substantially the same unless otherwise specified, and in cases where it is clearly considered in principle not to be so. And the like.

<Overview of Content Generation System According to One Embodiment of the Present Invention>
A content generation system according to an embodiment of the present invention receives a speech voice of a speaker as input and generates a corresponding content such as a moving image and a comic.

  The speaker speaks, for example, a story, a game plan, an experience, memories, or the like, or reads a book such as a novel and speaks. The number of speakers may be one or more.

  FIG. 1 shows a configuration example of a content generation system 10 according to an embodiment of the present invention.

  The content generation system 10 includes a terminal device 20 and a server device 30 connected via a network 11.

  The network 11 includes a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, a public telephone communication network, and the like.

  The terminal device 20 includes a general computer such as a smartphone and a tablet computer.

  The terminal device 20 acquires the uttered voice that is the basis of the content, and transmits the uttered voice to the server device 30 via the network 11 as voice information. In addition, the terminal device 20 receives, reproduces, and displays the content transmitted from the server device 30 via the network 11.

  The server device 30 is arranged on a so-called cloud network. The server device 30 includes a general computer such as a server computer. The server device 30 generates content based on the uttered voice transmitted from the terminal device 20.

<Configuration Example of Functional Blocks Constituting Terminal Device 20>
Next, FIG. 2 illustrates a configuration example of a functional block configuring the terminal device 20. The terminal device 20 includes functional blocks of a control unit 21, a communication unit 22, a voice acquisition unit 23, a voice output unit 24, an operation unit 25, and a display unit 26.

  The control unit 21 is realized, for example, by a CPU (Central Processing Unit) incorporated in a computer or the like executing a predetermined program. The control unit 21 controls the entire operation of the terminal device 20.

  The communication unit 22 is realized by, for example, a communication device built in a computer or the like. The communication unit 22 connects to the server device 30 via the network 11 and communicates various information. For example, the communication unit 22 transmits the speech information of the speaker acquired by the speech acquisition unit 23 to the server device 30. Further, for example, the communication unit 22 receives the content (moving image or comic) transmitted from the server device 30.

  The sound acquisition unit 23 is realized by, for example, a microphone included in an output device built in a computer or the like. The voice acquisition unit 23 acquires the voice of the speaker and outputs the resulting voice information to the communication unit 22. Note that the voice of the speaker acquired by the speech acquisition unit 23 may be recorded and transmitted to the server device 30 after the speech. Alternatively, voice information obtained by pre-recording the voice of the speaker from the outside may be input to the terminal device 20 and transmitted to the server device 30.

  The audio output unit 24 is realized by, for example, a speaker included in an output device built in a computer or the like. The audio output unit 24 outputs, for example, audio of the content transmitted from the server device 30.

  The display unit 26 is realized by, for example, a display included in an output device incorporated in a computer or the like. The display unit 26 displays a screen of the content supplied from the server device 30. Further, the display unit 26 displays a modification screen for modifying the scenario information 331 (FIG. 4) and the correspondence information 333 (FIG. 6) based on the screen information supplied from the server device 30.

<Configuration Example of Functional Blocks Constituting Server Device 30>
Next, FIG. 3 illustrates a configuration example of a functional block configuring the server device 30. The server device 30 includes functional blocks of a control unit 31, a communication unit 32, and a storage unit 33.

  The control unit 31 is realized, for example, by a CPU incorporated in a server computer executing a predetermined program. The control unit 31 controls the entire server device 30. The control unit 31 includes a conversion unit 311, an analysis unit 312, a scenario information generation unit 313, an association unit 314, a moving image generation unit 315, a correction unit 316, a collection unit 317, a cartoon generation unit 318, and a display control unit 319. Has functional blocks. Note that at least a part of each functional block of the control unit 31 may be realized using a learning model obtained by, for example, machine learning or deep learning.

  The conversion unit 311 converts voice information transmitted from the terminal device 20 into text information.

  The analysis unit 312 analyzes the voice information and the text information, and stores the analysis result. Specifically, for example, the analysis unit 312 detects nuances (intonation, strength, tone, speed, etc.) of the uttered voice of the speaker from the voice information, and performs natural language processing on the text information. The analysis of the text information is not limited to natural language processing, and any method can be applied. Then, the analysis unit 312 specifies 5W1H of the content as an analysis result.

  Specifically, for example, based on at least one of the nuances of the uttered voice and the result of the natural language processing of the text information, the title of the content, the emotions of the characters (including the narrator) (emotions, emotions, etc.), the speech (line) , Situation (date and time, season, era, place, etc.), behavior, and metadata (age, gender, country of origin (language used), region of origin (accent of language used, etc.)) are specified.

  Further, the analysis unit 312 specifies the metadata of the speaker based on the nuance of the uttered voice. The metadata of the title, characters, and speakers of the content specified by the analysis unit 312 may be input by the user.

  The scenario information generation unit 313 generates scenario information 331 corresponding to a script of the content generated by the server device 30 based on the analysis result by the analysis unit 312, and stores the scenario information 331 in the storage unit 33.

  FIG. 4 shows an example of the data structure of the scenario information 331. In the case of the figure, a title is given to the scenario information 331, and the metadata of the speaker is arranged as the time series number 0. Further, in the scenario information 331, at least one of the utterance (line) and the behavior of the character is arranged in chronological order with respect to the progress of the content (time series number 1 and later). Further, the scenario information 331 is associated with the situation and emotion when the character makes a statement or behavior. Further, in the scenario information 331, metadata of the characters is arranged. In the scenario information 331, at least one of a comment, a behavior, a situation, and an emotion regarding a character may be arranged in a time series.

  Referring back to FIG. The associating unit 314 converts the generated scenario information 331 into predetermined editing units (for example, scenes, etc.) so as to include at least one utterance and behavior of a character (including a narrator) according to the content thereof. Classify into. Further, the associating unit 314 determines, for each editing unit of the scenario information 331, the material image stored in the material image DB (database) 332 that matches the emotion, situation, behavior, and metadata of the character ( Search for not only exact matches, but also similar ones). Then, the associating unit 314 generates association information 333 in which a material image is associated with each editing unit, and stores the information in the storage unit 33.

  When searching for the material image, a search may be made for a metadata in which the metadata of the character in the material image matches the metadata of the speaker. Specifically, for example, when the country of origin of the speaker is Japan, a material image in which a Japanese character appears and speaks Japanese is detected. Further, a search may be made for a metadata whose style of the material image matches the metadata of the speaker. Specifically, for example, when the speaker is a woman, a material image whose style is directed to a woman is detected. The user may be allowed to select the style of the material image to be searched. When a plurality of material images are searched, the associating unit 314 may select an image to be associated with each editing unit of the scenario information 331 from among them, or the user may select it.

  FIG. 5 shows an example of the data structure of the material image DB 332. The material image DB 332 is generated in advance and stored in the storage unit 33. In the material image DB 332, material image data, material content, parameters, metadata, and style are recorded in association with the material ID. The material ID is an identifier for specifying the material image. The material image data is, for example, a moving image of 3D motion data or 2D motion data, and its data format is arbitrary. The material image data is a still image, and its data format is arbitrary. In the material contents, the contents of the material image data (characters and their behavior, etc.) are recorded. In the parameters, the emotions (emotions, sorrows, etc.) of the characters in the material image data, the situations (time, season, era, place) and the like are quantified and recorded. In the metadata, metadata such as the age of the character in the material image data, gender, country of origin (language used), region of origin (accent of the language used), and the like are recorded. In the style, the style of the material image data (for everyone, for men, for women, for children, for adults, historical drama, western style, oriental style, live action, animation, etc.) is recorded.

  FIG. 6 shows an example of the data structure of the correspondence information 333. In the correspondence information 333, a material ID is recorded in association with the time series number (scene number in FIG. 6) of each editing unit of the scenario information 331.

  Referring back to FIG. The moving image generation unit 315 acquires the material image data corresponding to the material ID from the material image DB 332 based on the generated correspondence information 333, and connects them in chronological order. Further, the moving image generation unit 315 refers to the scenario information 331, and superimposes the utterance (line) of the character as subtitles on the connected material image data, or synthesizes the voice corresponding to the utterance, A moving image as a content including subtitles or synthesized audio is generated and stored in the storage unit 33.

  The correction unit 316 corrects and supplements the scenario information 331 based on a correction instruction from the user transmitted from the terminal device 20. The correction unit 316 corrects the correspondence information 333 based on a correction instruction from the user transmitted from the terminal device 20. When the user modifies the correspondence information 333, the user additionally registers arbitrary material image data (either a moving image or a still image) in the material image DB 332 and associates the material image data with an arbitrary editing unit of the scenario information 331. You may do so. Further, the correction unit 316 learns a correction result for the scenario information 331 and the correspondence information 333, and updates material contents and parameters in the material image DB 332 based on the learning result.

  The collection unit 317 collects image data (moving image data and still image data) disclosed as so-called free materials on the Internet, analyzes the collected image data, and analyzes the material contents, parameters, and meta data. Data and style are set, and registered in the material image DB 332 as material image data.

  The cartoon generation unit 318 generates a cartoon as content based on the moving image generated by the moving image generation unit 315, and stores it in the storage unit 33. Specifically, for example, a manga as content is generated by extracting a representative image from each scene of a moving image and superimposing a speech (line) of a character as a balloon on the representative image.

  The display control unit 319 supplies the generated content to the terminal device 20 according to a request from the terminal device 20, and reproduces and displays the content. Further, the display control unit 319 generates screen information for displaying a correction screen in which the user can correct the scenario information 331 and the correspondence information 333 in response to a request from the terminal device 20. The generated screen information is transmitted to the terminal device 20 by the communication unit 32.

  The communication unit 32 is realized by, for example, a communication device built in the server computer. The communication unit 32 communicates various information with the terminal device 20 connected via the network 11. For example, the communication unit 32 receives audio information transmitted from the terminal device 20. In addition, for example, the communication unit 32 transmits content (moving image or comic) and screen information to the terminal device 20 via the network 11.

  The storage unit 33 is realized by, for example, a storage device built in the server computer. The storage unit 33 stores scenario information 331, a material image DB 332, correspondence information 333, and contents 334. The content 334 stored in the storage unit 33 may be supplied to a predetermined terminal device or the like, in addition to the terminal device 20 that has transmitted the utterance voice on which the content 334 is based.

  Note that some or all of the functional blocks of the server device 30 shown in FIG.

<Content generation processing by the content generation system 10>
Next, content generation processing by the content generation system 10 will be described. FIG. 7 is a flowchart illustrating an example of a content generation process performed by the content generation system 10.

  The content creation process is started, for example, in response to a predetermined start operation on the terminal device 20 by a user (speaker or the like).

  First, the terminal device 20 acquires the uttered speech of the speaker by the speech acquisition unit 23, outputs the speech information obtained as a result to the communication unit 22, and outputs the speech information from the speech acquisition unit 23 by the communication unit 22. , To the server device 30 via the network 11. The server device 30 outputs the voice information from the terminal device 20 to the control unit 31 through the communication unit 32 (Step S1).

  Next, the control unit 31 converts the voice information of the speaker into text information by the conversion unit 311 (step S2).

  Next, the control unit 31 analyzes the voice information and the text information by the analysis unit 312 (step S3). Next, the control unit 31 causes the scenario information generation unit 313 to generate the scenario information 331 based on the analysis result by the analysis unit 312, and stores the scenario information 331 in the storage unit 33 (Step S4).

  Next, the control unit 31 divides the scenario information 331 into predetermined editing units by the associating unit 314 (step S5), and refers to the material image DB 332 to associate the material image with each editing unit of the scenario information 331. The attached information 333 is generated and stored in the storage unit 33 (step S6).

  Next, the control unit 31 acquires the material image data corresponding to the material ID from the material image DB 332 based on the correspondence information 333 by the moving image generating unit 315, and connects them in chronological order. Further, the moving image generation unit 315 refers to the scenario information 331, and superimposes the utterance (line) of the character as subtitles on the connected material image data, or synthesizes the voice corresponding to the utterance, A moving image including captions or synthesized voice is generated and stored in the storage unit 33 as the content 334 (step S7).

  Next, the control unit 31 causes the cartoon generation unit 318 to generate a cartoon based on the moving image and stores it as the content 334 in the storage unit 33 (step S8).

  Thus, the content generation processing by the content generation system 10 ends. Note that the generated content 334 is supplied to a request source and reproduced in response to a request from the terminal device 20 or the like.

  According to the content generation processing described above, it is possible to generate a moving image and a comic as content based on the speech sound of the speaker.

<Correction processing by the content generation system 10>
Next, correction processing by the content generation system 10 will be described. FIG. 8 is a flowchart illustrating an example of a correction process performed by the content generation system 10.

  The correction process is started when a correction request is input from the user to the terminal device 20, and the correction request is transmitted to the server device 30 and input to the control unit 31.

  First, the control unit 31 determines whether the correction request from the user is a request for correction of the scenario information 331 by the correction unit 316 (step S11).

  If the correcting unit 316 determines that the scenario information 331 is to be corrected (YES in step S11), the process proceeds to step S12. Next, the control unit 31 causes the display control unit 319 to generate screen information of a correction screen for correcting the scenario information 331, and the communication unit 32 transmits the generated screen information to the terminal device 20. The terminal device 20 displays a screen for correcting the scenario information 331 based on the transmitted screen information. When the user inputs a correction instruction to the screen using the operation unit 25, the communication unit 22 A correction instruction is transmitted to the server device 30. Then, in the server device 30, the control unit 31 corrects and supplements the scenario information 331 stored in the storage unit 33 by the correction unit 316 in response to a correction instruction from the user (step S12).

  If the modifying unit 316 determines that the modification of the scenario information 331 is not requested (NO in step S11), step S12 is skipped, and the process proceeds to step S13.

  Next, the control unit 31 determines whether or not the correction request from the user requests the correction of the correspondence information 333 by the correction unit 316 (step S13).

  Here, when the correction unit 316 determines that the correction of the correspondence information 333 is requested (YES in step S13), the process proceeds to step S14. Next, the control unit 31 causes the display control unit 319 to generate screen information of a correction screen for correcting the correspondence information 333, and the communication unit 32 transmits the generated screen information to the terminal device 20. The terminal device 20 displays a screen for correcting the correspondence information 333 based on the transmitted screen information. When the user inputs a correction instruction to the screen using the operation unit 25, the communication unit 22 A correction instruction is transmitted to the server device 30. Then, in the server device 30, the control unit 31 causes the correction unit 316 to correct the correspondence information 333 stored in the storage unit 33 in response to a correction instruction from the user (step S14).

  If the correction unit 316 determines that the correction of the correspondence information 333 is not requested (NO in step S13), step S14 is skipped.

  Next, the control unit 31 causes the moving image generation unit 315 to regenerate a moving image as content based on the corrected scenario information 331 and the correspondence information 333 and store the generated moving image in the storage unit 33, and the comic generation unit 318. Thereby, the comic as the content is regenerated based on the regenerated moving image and stored in the storage unit 33 (step S15).

  Next, the control unit 31 learns a correction result for the scenario information 331 and the correspondence information 333 by the correction unit 316, and updates material contents and parameters in the material image DB 332 based on the learning result (step S16). Note that the scenario information generating unit 313 and the associating unit 314 may also learn the correction result for the scenario information 331 and the corresponding information 333 and apply the result to the subsequent processing. Thus, the correction processing by the content generation system 10 is completed.

  According to the modification processing by the content generation system 10, the user can modify the scenario information 331 and the correspondence information 333 and obtain a content reflecting the modification result. Further, the correction result based on the correction instruction from the user is learned and reflected in the material image DB 332, so that a more appropriate material image is searched and content is generated thereafter.

<About the content display screen>
Next, FIG. 9 shows a display example of the content display screen 50 on the terminal device 20 or the like.

  The content display screen 50 is provided with a content reproduction area 51 in which content is displayed. A subtitle 53 can be displayed in the content reproduction area 51. Above the content reproduction area 51, a title 52 of the content is displayed. At the lower side of the content reproduction area 51, an operation button 54 for the user to instruct the start of reproduction of the content, an operation button 55 for instructing the fast rewind of the content, and an operation button 56 for instructing the fast forward of the content Is provided. In addition, the display positions of the title 52, the subtitle 53, and the operation buttons 54 to 56 are not limited to the illustrated example and are arbitrary.

  On the right side of the content reproduction area 51, style buttons 61 to 65 are provided, which indicate the current style of the content and allow the user to instruct the change of the style of the content. In the case of the drawing, the style (for women) button 62 among the style buttons 61 to 65 is highlighted, indicating that the current style of the content is for women.

  In this state, for example, when the user selects the style (for children) button 63, the above-described steps S6 to S8 of the content generation processing (FIG. 7) are executed again, and the content is generated using the material image data for children. Are regenerated and displayed in the content reproduction area 51.

  In addition, the display positions and the number of the style buttons 61 to 65 are not limited to the illustrated example and are arbitrary. Alternatively, only one style button may be displayed, and when the style button is operated, the style options may be displayed in a pop-up display.

<Example of general computer configuration>
As described above, the terminal device 20 and the server device 30 can be realized by a general computer.

  FIG. 10 illustrates a configuration example of a general computer that realizes the terminal device 20 and the server device 30.

  In the computer 100, a CPU 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103 are interconnected by a bus 104.

  The bus 104 is further connected to an input / output interface 105. An input device 106, an output device 107, a storage device 108, a communication device 109, and a drive device 110 are connected to the input / output interface 105.

  The input device 106 includes a keyboard, a mouse, a touch panel, a microphone, and the like, and functions as, for example, the voice acquisition unit 23 and the operation unit 25 (FIG. 2) of the terminal device 20. The output device 107 includes a display, a speaker, and the like, and functions as, for example, the audio output unit 24 and the display unit 26 (FIG. 2) of the terminal device 20.

  The storage device 108 includes a hard disk drive (HDD), a solid state drive (SSD), and the like, and functions as, for example, the storage unit 33 (FIG. 3) of the server device 30. The communication device 109 includes a LAN interface and a Wi-Fi interface, and functions as, for example, the communication unit 22 of the terminal device 20 (FIG. 2) and the communication unit 32 of the server device 30 (FIG. 3). The drive device 110 drives a removable medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer 100 configured as described above, the CPU 101 loads, for example, a program stored in the storage device 108 into the RAM 103 via the input / output interface 105 and the bus 104 and executes the program. The control unit 21 (FIG. 2) of the terminal device 20 and the control unit 31 (FIG. 3) of the server device 30 are realized.

  The program executed by the computer 100 (CPU 101) can be provided by being recorded on a removable medium 111 as a package medium or the like, for example. Further, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer 100, the program can be installed in the storage device 108 via the input / output interface 105 by attaching the removable medium 111 to the drive device 110. Further, the program can be received by the communication device 109 via a wired or wireless transmission medium and installed in the storage device 108. In addition, the program can be installed in the ROM 102 or the storage device 108 in advance.

  Note that the program executed by the computer 100 may be a program in which processing is performed in chronological order according to the order described in this specification, or may be performed at a necessary timing such as in parallel or when a call is made. May be a program that performs the processing in.

  The effects described in the present specification are merely examples and are not limited, and may have other effects.

  The present invention is not limited to the embodiments described above, and includes various modifications. For example, each of the above embodiments has been described in detail in order to explain the present invention in an easily understandable manner, and the present invention is not necessarily limited to a configuration including all the described components. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of one embodiment can be added to the configuration of another embodiment. Further, for a part of the configuration of each embodiment, it is possible to add, delete, or replace another configuration.

  In addition, the above-described configurations, functions, processing units, processing means, and the like may be partially or entirely realized by hardware, for example, by designing an integrated circuit. Each of the above-described configurations, functions, and the like may be implemented by software by a processor interpreting and executing a program that implements each function. Information such as a program, a table, and a file for realizing each function can be stored in a storage device such as a semiconductor memory, an HDD, and an SSD, or a recording medium such as an IC card, an SD card, and a DVD. In addition, control lines and information lines are shown as necessary for the description, and do not necessarily indicate all control lines and information lines on a product. In fact, it can be considered that almost all components are interconnected.

  DESCRIPTION OF SYMBOLS 10 ... Content generation system, 11 ... Network, 20 ... Terminal device, 21 ... Control unit, 22 ... Communication unit, 23 ... Sound acquisition unit, 24 ... Sound output unit , 25 ... operation unit, 26 ... display unit, 30 ... server device, 31 ... control unit, 32 ... communication unit, 33 ... storage unit, 50 ... content display screen , 51: Content playback area, 52: Title, 53: Subtitle, 54 to 56: Operation buttons, 61 to 65: Style buttons, 100: Computer, 101: CPU , 102 ROM, 103 RAM, 104 bus 105 input / output interface 106 input device 107 output device 108 storage device 109 ..Communication devices, 110 ... Dora Device, 111: removable media, 311: conversion unit, 312: analysis unit, 313: scenario information generation unit, 314: association unit, 315: moving image generation unit, 316: correction unit, 317: collection unit, 318: comic generation unit, 319: display control unit, 331: scenario information, 332: material image DB, 333: correspondence Information, 334 ... contents

Claims (13)

A voice acquisition unit that acquires voice information of a speaker;
A conversion unit that converts the audio information into text information,
An analysis unit for analyzing the text information,
A scenario information generating unit that generates scenario information based on the analysis result of the text information,
A material image database in which the material images are stored,
An associating unit that divides the scenario information into predetermined editing units, and generates association information by associating the material images stored in the material image database with each editing unit;
A moving image generation unit that generates a moving image by linking the material images associated with each editing unit,
Equipped with a,
The analysis unit specifies the metadata of the speaker based on the audio information,
The content generation system, wherein the association unit associates the material images stored in the material image database for each editing unit based on the metadata of the speaker to generate the correspondence information. . The content generation system according to claim 1, wherein
The content generation system, wherein the analysis unit specifies 5W1H in the content as the analysis result of the text information. The content generation system according to claim 1 or 2,
The content generation system, wherein the analysis unit specifies an emotion of a character in the content. The content generation system according to claim 3, wherein
The content generation system, wherein the analysis unit specifies an emotion of a character in the content based on the audio information. The content generation system according to claim 1, wherein:
The analysis unit specifies at least one of age, gender, country of origin, language used, region of origin, and accent of language used as metadata of the speaker based on the audio information. Content generation system. A content generation system according to any one of claims 1 to 5 ,
The association unit associates the material image of a predetermined style stored in the material image database for each editing unit based on the metadata of the speaker to generate the association information. Content generation system.   The content generation system according to claim 6, wherein
  In the material image database, as the style, at least one of the material images for everyone, for men, for women, for children, for adults, historical drama, Western style, Eastern style, live-action, and animation Stored
  A content generation system, characterized in that: A content generation system according to any one of claims 1 to 7,
The scenario information generation unit generates the scenario information in which at least one of a comment, a behavior, a situation, and an emotion regarding a character in the content is arranged in time series based on the analysis result of the text information. Characteristic content generation system. It is a content generation system according to any one of claims 1 to 8,
An operation unit for receiving an operation input from a user,
A correction unit that corrects at least one of the scenario information and the correspondence information based on an operation input from the user,
A content generation system comprising: The content generation system according to claim 9, wherein:
The content generation system, wherein the correction unit learns a correction result for at least one of the scenario information and the correspondence information, and updates the material image database based on the learning result. The content generation system according to claim 9, wherein:
The content generation system, wherein the correction unit corrects the correspondence information by associating the material image input by the user with the editing unit of the scenario information. A content generation system according to any one of claims 1 to 11,
A collection unit that collects image data as free material and registers the material image in the material image database as the material image.
A content generation system, comprising: A content generation method by a content generation system,
A voice obtaining step of obtaining voice information of the speaker;
A conversion step of converting the audio information into text information,
An analysis step of analyzing the text information;
A scenario information generating step of generating scenario information based on the analysis result of the text information,
An associating step of dividing the scenario information into predetermined edit units and generating corresponding information by associating material images stored in a material image database for each edit unit;
A moving image generating step of generating a moving image by linking the material images associated with each editing unit,
Only including,
The analyzing step specifies the metadata of the speaker based on the audio information,
The content generation method, wherein the associating step generates the association information by associating the material images stored in the material image database for each of the editing units based on the metadata of the speaker. .

Images