JP2020005309A - Moving image editing server and program - Google Patents

Abstract

To provide a server and a program that make it possible to easily create a moving image content.SOLUTION: There are provided a moving image editing server and a program therefor, the server creating a moving image content to be distributed to a user terminal, and comprising: a template management part which manages a template having a plurality of prescribed cuts given tag information; a raw material read-in part which receives input of image data and character data; a raw material analysis part which recognizes an object in the input image data and outputs similarity to the tag information; an image insertion part which allocates the image data to the template on the basis of the similarity; and a telop insertion part which allocates a telop created from the character data to the template, wherein the image data and telop are allocated to a selected template to create a moving image content.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a server and a program for automatically generating moving image content to be distributed to a user terminal.

Conventionally, a moving image is divided into a plurality of chapters and metadata is added.
For example, Patent Literature 1 discloses a moving image processing apparatus that efficiently searches for a scene image at a desired moment from a moving image having a plurality of chapters. The moving image processing apparatus divides a moving image into a plurality of large blocks every predetermined unit time. A block division unit, a complexity analysis unit that quantifies the complexity of image change of each large block, and a small block division that divides the reproduction time of each large block into a plurality of small blocks by using a numerical value related to complexity There has been proposed a moving image processing apparatus including a unit and a chapter creating unit that creates a chapter by sequentially dividing a plurality of small blocks in a predetermined number along a time series.

JP 2011-130007 A

  Since it takes a great deal of time to create moving image content, it has been required to provide a system that can easily create moving image content.

  In view of the above, an object of the present invention is to provide a server and a program capable of easily creating moving image content.

The moving image editing server of the present invention is a server that creates moving image content for distribution to a user terminal, and includes a template management unit that manages a template in which a plurality of cuts with tag information are defined, A material reading unit that receives input of character data, a material analyzing unit that recognizes an object in the input image data and outputs a similarity with the tag information, and allocates image data to the template based on the similarity A telop insertion unit for allocating a telop created from the character data to the template is provided, and a moving image content is created by allocating the image data and the telop to the selected template.
In the moving image editing server, the telop insertion unit may assign character data to the template based on a degree of similarity with the tag information.
The moving image editing server may further include a template recommendation unit that outputs a recommended template based on the input image data.
In the moving image editing server, the learning model is a learned model that has performed machine learning using learning data, includes a classifier that outputs an annotation word of input image data, and the similarity output by the material analysis unit is It may be characterized by the word similarity between the annotation word output by the classifier and the tag information.

In the moving image editing server, the template is provided with color information, the material analysis unit outputs a color similarity between the image data and the color information, and the image insertion unit includes the word similarity. And / or allocating image data to the template based on the color similarity.
In the moving image editing server, the telop insertion unit may create the telop by summarizing text information included in the character data.
In the moving image editing server, the material reading unit may have a function of recognizing voice in the moving image file and inputting character data.
In the moving image editing server, the material reading unit may include a function of acquiring image data from a database or the Web in addition to the input image data.

  A moving image distribution system of the present invention is a moving image distribution system including the moving image editing server and the moving image distribution server, wherein the moving image distribution server collects content information of a Web page on which a moving image distribution surface is set. A scraper, a content analysis unit that analyzes the content information and analyzes a similarity with the video content created in advance, and a video distribution unit that distributes the video content with a high similarity. . These are also used for distribution of video advertisements.

  The moving image editing server program of the present invention is a moving image editing program for a server that distributes moving image content to a user terminal accessed via the Internet, wherein the server is a template in which a plurality of cuts with tag information are defined. And a material reading unit for receiving input of image data and character data, a material analyzing unit for recognizing an object in the input image data and outputting a similarity with the tag information, And a telop insertion unit that inserts a telop created from the character data based on the image data.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the server and program which enable easy creation of moving image content.

1 is a configuration diagram of a moving image editing system according to an embodiment. FIG. 2 is a configuration diagram of a moving image editing server according to the embodiment. FIG. 4 is a diagram illustrating an example of a screen layout that forms a template. FIG. 3 is a configuration diagram of a composite moving image creating unit. It is a figure explaining the example of a material input screen. FIG. 3 is a diagram illustrating each process for creating moving image content from a material. FIG. 11 is an explanatory diagram of a process of allocating an image / moving image most suitable for each cut. FIG. 11 is an explanatory diagram of a mode in which a plurality of cuts constituting moving image content are displayed as a list on a screen. FIG. 3 is an explanatory diagram of a screen for inserting BGM into moving image content. It is a processing flow of a summary sentence creation function. FIG. 9 is a diagram illustrating a process of subjecting a document to morphological analysis and dividing the document into tokens. It is a figure explaining insertion of a sentence to each cut of a template. It is a lineblock diagram of a video distribution server concerning an example of an embodiment.

<Configuration>
As shown in FIG. 1, the moving image editing system according to the embodiment of the present invention includes a moving image editing server 1, an administrator terminal 2, and a plurality of user terminals 3. Although the example of FIG. 1 illustrates an example in which the moving image editing server 1 is configured by one unit, the moving image editing server 1 can be realized by a plurality of server devices.

  The moving image editing server 1 is constructed by installing moving image editing software and database software on a server device including a processing unit having a CPU, a storage unit having a storage device such as an HDD, and a communication unit having a LAN port. . As shown in FIG. 2, the moving image editing software includes a template managing unit 11, a classifier creating unit 12, and a combined moving image creating unit 13. As shown in FIG. 2, the database software manages a template DB 21, a learning data DB 22, a combined moving image DB 23, and an embedded material DB 24.

  The template management unit 11 manages a plurality of templates stored in the template DB 21. Each template is composed of a plurality of cuts, and a screen layout and a reproduction time are defined for each cut. An image file may be assigned to the template, and the image file to be assigned to the template can be selected from the embedded material DB 24. FIG. 3 is an example of a screen layout of cuts constituting a template. In the figure, the edited article material (text sentence) is inserted into the telop field 31, the selected image material is inserted into the image field 32, and the logo material is inserted into the logo field 33. Each template is provided with style information, color information, and tag information. The color information and the tag information are used by a template recommendation unit 134 described later.

  The classifier creating unit 12 acquires the learning data from the learning data DB 22 and performs machine learning to create a classifier 133 described later, which is a learned model. The creation of the classifier 133 by the classifier creation unit 12 is performed several times a year, for example. As the learning data, data collected from the Internet or data of the company in which a label is attached may be used, or a labeled data set may be procured and used.

  As shown in FIG. 4, the synthetic moving image creation unit 13 includes a material reading unit 131, a material analysis unit 132, a classifier 133, a template recommendation unit 134, an image insertion unit 135, a telop insertion unit 136, and music. And an insertion portion 137.

The material reading unit 131 displays a material input screen page on a Web browser of the user terminal 3. The material reading unit 131 includes an image data input unit that receives input of image data, and a character data input unit that receives input of character data.
FIG. 5 is a diagram illustrating an example of a material input screen output by the material reading unit 131. By dragging and dropping image data (image / movie) into the material submission field 51 or the logo submission field 52, the user can submit an image / video material. The number of image / movie materials to be submitted can be increased or decreased by clicking the + button or the-button.
Character data (article material) can be entered by inputting text in the title field 53 and the body field 54. After submitting each material, when the recommended format button 55 is clicked, the recommended format is output in a recommended format display column. In the example of FIG. 5, only one recommended format is displayed. However, a specification in which a plurality of recommended formats are displayed in order of recommendation may be adopted. Unlike the embodiment, a specification in which a user manually selects an arbitrary format from a plurality of formats may be adopted. In addition, a specification may be adopted in which a text sentence is input by inputting a text sentence by recognizing a voice in a moving image file and converting the text into a text. For example, a voice recognition service provided as an external API may be used. Is also good. Further, a specification may be adopted in which article materials, image / moving image materials are acquired from the URL input by the user, and are stored in the embedded material DB 24.

The material analysis unit 132 calculates the degree of relevance between the information extracted from the input material and the tag information assigned to the format. Information extraction from the material is performed by the classifier 133.
The classifier 133 is a trained model using a convolutional neural network, and extracts specific information (annotation word) when a moving image or an image is input. The first classifier according to the embodiment outputs a word (for example, stylish or simple) representing a classification of a moving image or an image. The second classifier according to the embodiment outputs words (for example, seafood, grilled meat, people, and furniture) representing an object appearing in a moving image or an image.

The template recommendation unit 134 recommends the format with the highest relevance to the user based on the output of the material analysis unit 132.
The image insertion unit 135 is a learned model using a convolutional neural network, and calculates similarity between information extracted from the input material and words, colors, and brightness assigned to each cut, and calculates the highest similarity. Assign a high image / movie to each cut. Here, the similarity may be determined and assigned not only to the image / video material input by the user but also to the image / video material stored in the embedded material DB 24 in advance.
To determine the word similarity, a learned model in which a word vector is learned is prepared, and the vector similarity is used to determine the word similarity by a method such as cosine similarity or Word Mover's Distance. The determination of the color similarity is performed, for example, by calculating the Euclidean distance in CIELAB coordinates. More specifically, the color difference from the color to be compared is calculated for all the pixels of the image, and a pixel having a small total value is determined to be close to the color difference. For moving images, this calculation is performed on images sampled from frame images of moving images. The determination of the lightness is performed, for example, by converting the image to gray scale and comparing the RMS contrast values of the pixels. For a moving image, this calculation is performed on an image sampled from a frame image of the moving image, as in the case of color determination.
The telop insertion unit 136 inserts a telop (text sentence) into each cut of the selected template. Details of the summary sentence creation function by the telop insertion unit 136 will be described later.
The music insertion unit 137 inserts music that becomes BGM into the moving image content.

The administrator terminal 2 and the user terminal 3 are computers each including an input unit, a display unit, a processing unit, a storage unit, and a communication unit, and include, for example, Web sites such as a smartphone, a tablet terminal (tablet PC), a notebook personal computer, and a desktop personal computer. It is a computer with a browser.
The administrator uses the administrator terminal 2 to change the settings of the moving image editing server 1 and operate and manage the database.
The user can access the moving image editing server 1 through the user terminal 3 and browse the created moving image content.

As shown in FIG. 6, the moving image editing system of the present invention includes a material analyzing step of analyzing a material input by a user, a format selecting step of selecting a format of a moving image to be created, and a telop and image data (still image) in the selected format. (Moving image) is executed.
In the material analysis step, the material analysis unit 132 analyzes materials such as articles, images (still images), and moving images, and the template recommendation unit 134 selects a format highly relevant to the input material and recommends it to the user. For example, if the tag information extracted from the uploaded material is “person”, “nail”, or “soap”, it is determined that it is close to “cosmetics” or “female”, and the format A with the cute style is attached. Is recommended. Further, the recommended format may be selected based on the color information extracted from the material, or the recommended format may be selected using both the color information and the tag information.

  In the format selection step, the user selects the format of the moving image used for the moving image configuration. The user may use the format most recommended by the template recommendation unit 134 as it is, may select a favorite format from a plurality of recommended formats, or may select his or her own format from unrecommended formats. You may choose your favorite format. Each cut that makes up the format has a decorated frame placed on it and an animation effect (such as split, fade, slide in / out, spin, turn, etc. on PowerPoint) on the inserted image. It may be set.

In the moving image composing step, the telop insertion unit 136 creates a summary sentence extracted from the material and assigns it to each cut, and the image insertion unit 135 assigns an image / moving image extracted from the material to each cut. As shown in FIG. 7, each cut constituting the format is associated with brightness information, color information, and tag information such as a person, a car, a business, and a natural scenery. The image insertion unit 135 determines the most suitable image / moving image for each cut based on the similarity between the word, the color, and the brightness, and assigns the cut image / moving image to each cut. In the embodiment, the image insertion unit 135 assigns the image / video material input by the user to each cut. However, a specification may be adopted in which a prepared image / video material is allocated to each cut.
When the image / moving image assignment is completed, a plurality of cuts constituting the moving image content can be displayed on the screen as shown in FIG. In each cut, information on the reproduction time (number of seconds) of each cut is displayed together with the displayed image / moving image and telop. The user can correct the telop by clicking the text button or the text field, and can replace the image by clicking the image button.

  BGM can be inserted into the created moving image content by the music insertion unit 14. When the BGM button is clicked on the screen in FIG. 8, a list of BGM that can be inserted is displayed as shown in FIG. Click the play button to the right of the song name to start playing the song. By clicking the check box on the left side of the song name and clicking the preview button, it becomes possible to reproduce a preview of the moving image content into which BGM has been inserted.

(Summary sentence creation function)
The summary sentence creation function of the telop insertion unit 136 will be described with reference to FIGS.
STEP 91: Paragraph Division / Document Division The telop insertion unit 136 divides the document input in the body field 54 into paragraphs, and divides the document in each paragraph into documents. Also, a sentence (for example, 80 characters or more) which is too long to be read if it is displayed as a telop of a moving image in one scene is further divided into a plurality of sentences at a portion satisfying a condition of a specific part of speech or notation.
STEP 92: Morphological Analysis of Document The telop insertion unit 136 performs a morphological analysis on each sentence, and extracts a token that is the minimum unit for syntactic analysis. As shown in FIG. 11, a part of speech is assigned to each token.

STEP 93: Deletion of Unnecessary Word / Unnecessary Paragraph The telop insertion unit 136 deletes a sentence and a paragraph defined as invalid from a predefined invalid sentence determination rule. For example, a line starting with a specific symbol such as “■” or “▼”, a paragraph surrounded by the specific symbol, a paragraph describing a URL, a mail address, an address or a telephone number, etc. are deleted.
STEP94: Deletion of Stop Words and the Like The telop insertion unit 136 converts specific tokens such as “ni”, “kara”, “this”, “san” and other words that are not so important as words (stop words) and particles from the token. delete.
STEP 95: Token Bigram Creation A token bigram is obtained by connecting a plurality of tokens satisfying a specific condition (for example, a predefined part of speech condition). For example, "2014" (noun, proper noun, general) and "June" (noun, proper noun, general) are connected to form "June 2014", or "Verdi" (proper noun) and "support" (Ordinary nouns) to form "Verdi sponsorship".

STEP 96: Extraction of Important Sentences Tokens and token viagrams, which are characteristic words, are extracted from an index for evaluating the importance of words such as TF-IDF based on the tokens and token bigrams, and sentence segmentation is performed from the above-described word similarity determination. Then, an important sentence is extracted from each segment to form a summary.
STEP 97: Applying to a template The summary (important sentence) is subjected to syntax analysis, and is separated into a phrase and a syntax tree. In the above-mentioned template, the number of characters that can be inserted into each cut is defined, and the sentence is divided based on the modification relation between the phrases so that a natural section as a sentence is included in each template, and is applied to the template. FIG. 12 shows an example of inserting a sentence into each cut of the template.
The summary sentence creation function described above can support not only Japanese but also multiple languages such as English.

<Video distribution server>
A moving image distribution server having a function of distributing the created moving image content may be added. The transmission of the Web page 6 in which the moving image distribution script is incorporated uses the external Web server 5.
As shown in FIG. 13, the video distribution server 4 includes a Web scraper, an article text extraction unit, an article text analysis unit, a video analysis unit, a video distribution unit, a hash value database, and a video database. ing. The article body analysis unit and the moving image analysis unit may be referred to as a content analysis unit. Note that the moving image distribution server 4 can be realized by a plurality of server devices.
The web scraper collects content information of a web page on which a distribution surface of a moving image is set.
The article body extracting unit is software for extracting a part corresponding to the body of the article from the collected content information, and can be configured by a tool such as Readability.
The article body analysis unit performs a morphological analysis on the extracted article body, extracts important keywords by a method such as TF-IDF, and converts the body hash value generated using a hash value generation MinHash or b-bit Minwise Hashing into a hash value. It is stored in the database together with the URL of the Web page.

The video analysis unit performs a morphological analysis on the telop of the video in the collected content information, extracts important keywords by a method such as TF-IDF, generates a telop hash value in the same manner as the article body analysis unit, and generates a hash value. Save to database. The moving image analysis unit calculates the similarity from the generated telop hash value and the stored article hash value, and stores the similarity.
When the script embedded in the Web page 6 requests acquisition of a moving image, the moving image distribution unit obtains, from the moving image database, a moving image that has a high degree of similarity and is currently available for distribution from the URL of the received Web page 6 And distribute the video. More specifically, the request URL transmitted when the Web page 6 requests a moving image includes the URL of the Web page 6 as a parameter, and the moving image distribution server 4 determines the URL of the Web page 6 from the received parameter. The extracted moving image is selected based on the analysis result by the moving image analysis unit, and is distributed to the Web page 6.
The moving image database can store the moving image created by the moving image editing server 1 described above. According to the moving image distribution server 4 of the embodiment, for example, by distributing a moving image advertisement having a high similarity to the article body of the Web page 6, it is also possible to efficiently perform a targeting advertisement that matches the interest of the user. It is.

According to the moving image editing system of the embodiment described above, it is possible to easily create moving image contents without having to prepare moving image editing software, a moving image server, an editor with specialized skills, and the like. . For example, it can be used in the following situations.
1) Animation of product information sold at EC shops 2) Distribution of press release information, CSR information, etc. in animation 3) Animation of manuals on usage and operation flow 4) Creation of creatives that can be used as video advertisements

  The preferred embodiment of the present invention has been described above, but the technical scope of the present invention is not limited to the description of the above embodiment. Various changes and improvements can be made to the above-described embodiment, and the forms with such changes or improvements are also included in the technical scope of the present invention.

1 Video Editing Server 2 Administrator Terminal 3 User Terminal 4 Video Distribution Server 5 External Web Server 11 Template Management Unit 12 Classifier Creation Unit 13 Synthetic Animation Creation Unit 21 Template DB
22 Learning data DB
23 Synthetic Video DB
24 Embedding Material DB

Claims (10)

A server that creates video content for distribution to a user terminal,
A template management unit that manages a template in which a plurality of cuts with tag information are defined,
A material reading unit for receiving input of image data and character data,
A material analysis unit that recognizes an object in the input image data and outputs a similarity to the tag information;
An image insertion unit that assigns image data to the template based on the similarity;
A telop insertion unit for allocating a telop created from the character data to the template,
And a moving image editing server that creates moving image contents by allocating image data and telop to a selected template.   The moving picture editing server according to claim 1, wherein the telop insertion unit assigns character data to the template based on a similarity with the tag information.   The moving image editing server according to claim 1, further comprising a template recommendation unit that outputs a template recommended based on the input image data. It is a trained model that has performed machine learning using learning data, and has a classifier that outputs annotation words of input image data,
The moving image editing server according to any one of claims 1 to 3, wherein the similarity output by the material analysis unit is a word similarity between the annotation word output by the classifier and the tag information. The template is provided with color information,
The material analysis unit outputs a color similarity between the image data and the color information,
The moving image editing server according to claim 4, wherein the image insertion unit assigns image data to the template based on the word similarity and / or the color similarity.   The moving picture editing server according to any one of claims 1 to 5, wherein the telop insertion unit summarizes text information included in the character data to create the telop.   7. The moving image editing server according to claim 1, wherein the material reading unit has a function of recognizing voice in a moving image file and inputting character data.   The moving image editing server according to any one of claims 1 to 7, wherein the material reading unit has a function of acquiring image data from a database or the Web in addition to the input image data. A video distribution system comprising the video editing server according to any one of claims 1 to 7, and a video distribution server,
The video distribution server
A web scraper for collecting content information of a web page to which a video distribution surface is set,
A content analysis unit that analyzes the content information and analyzes a degree of similarity with the previously created moving image content,
A moving image advertisement distribution unit that distributes the moving image content having a high degree of similarity. In a moving image editing program for a server that distributes moving image contents to a user terminal accessed via the Internet,
Said server,
A template management unit that manages a template in which a plurality of cuts with tag information are defined,
Material reading unit that accepts input of image data and character data,
A material analysis unit that recognizes an object in the input image data and outputs a similarity with the tag information;
An image insertion unit that assigns image data to the template based on the similarity, and
A moving image editing server program functioning as a telop inserting unit for inserting a telop created from the character data.

Images