JP2020065307A - Server, program, and moving image distribution system - Google Patents

Abstract

To provide a server, a program, and a moving image distribution system capable of easily creating a moving image content.SOLUTION: The server comprises: a template management unit which manages a template composed of a plurality of cuts to which tag information is attached; a material reading unit which accepts input of image data; a material analysis unit which recognizes an object in the input image data and outputs similarity with the tag information; and an image insertion unit which allocates the image data to the template based on the similarity.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a server and a program for automatically generating moving image content to be distributed to a user terminal, and a moving image distribution system.

Conventionally, a moving image is divided into a plurality of chapters and metadata is added.
For example, Patent Document 1 discloses a moving image processing apparatus that efficiently searches a moving image having a plurality of chapters for a scene image at a desired moment, and divides the moving image into a plurality of large blocks at predetermined unit time intervals. A block division unit, a complexity analysis unit that digitizes the complexity of image change of each large block, and a small block division that divides the playback time of each large block into a plurality of small blocks according to the numerical value of the complexity. There has been proposed a moving image processing apparatus including a section and a chapter creating section that creates a chapter by sequentially dividing a plurality of small blocks into a predetermined number in a time series.

JP, 2011-130007, A

  Since it takes a lot of time and effort to create moving image content, it has been demanded to provide a system that can easily create moving image content.

  Therefore, it is an object of the present invention to provide a server, a program, and a moving image distribution system that can easily create moving image contents.

In the server of the present invention, a template management unit that manages a template composed of a plurality of cuts to which tag information is attached, a material reading unit that receives input of image data, and an object in the input image data A server comprising: a material analysis unit that recognizes and outputs a degree of similarity to the tag information; and an image insertion unit that allocates image data to the template based on the degree of similarity.
In the server, the model is a learned model that has been machine-learned using learning data, and a classifier that outputs the annotation word of the input image data is provided, and the similarity output by the material analysis unit is It may be characterized in that it is the word similarity between the annotation word output by and the tag information.

In the server, color information is attached to the template, the material analysis unit outputs a color similarity between the image data and the color information, and the image insertion unit outputs the word similarity and / or the word similarity. Alternatively, image data may be assigned to the template based on the color similarity.
In the above server, the material reading unit may be provided with a function of acquiring image data from a database or a Web in addition to the input image data.

  A moving picture distribution system of the present invention is a moving picture distribution system including the above-mentioned server that creates moving picture contents, and a moving picture distribution server, wherein the moving picture distribution server displays content information of a Web page in which a moving picture distribution plane is set. A web scraper to collect, a content analysis unit that analyzes the content information and analyzes the degree of similarity with the video content created in advance, and a video distribution unit that distributes the video content with high similarity. Characterize. These are also used for distribution of video advertisements.

  A server program according to the present invention includes a template management unit that manages a template including a plurality of cuts to which tag information is attached, a material reading unit that receives input of image data, and input image data. It is characterized by functioning as a material analysis unit that recognizes an object inside and outputs a degree of similarity with the tag information, and an image insertion unit that allocates image data to the template based on the degree of similarity.

  According to the present invention, it is possible to provide a server, a program, and a moving image distribution system that can easily create moving image contents.

It is a block diagram of the moving image editing system which concerns on the example of embodiment. It is a block diagram of the video editing server which concerns on the example of embodiment. It is a figure explaining the example of a screen layout which comprises a template. It is a block diagram of a synthetic | combination moving image production | generation part. It is a figure explaining the example of a material input screen. It is a figure explaining each process for creating moving image contents from a material. It is explanatory drawing of the process which allocates the image and moving image most suitable for each cut. It is explanatory drawing of the aspect which displays the several cuts which comprise moving image content as a list on a screen. It is explanatory drawing of the screen which inserts BGM in moving image content. It is a processing flow of a summary sentence creation function. FIG. 6 is a diagram illustrating a process of subjecting a document to morphological analysis and dividing the document into tokens. It is a figure explaining insertion of a sentence to each cut of a template. It is a block diagram of the moving image distribution server which concerns on the example of embodiment.

<Structure>
As shown in FIG. 1, the moving image editing system of the present invention according to the embodiment includes a moving image editing server 1, an administrator terminal 2, and a plurality of user terminals 3. In the example of FIG. 1, the moving image editing server 1 is configured by one unit, but the moving image editing server 1 can be realized by a plurality of server devices.

  The moving image editing server 1 is constructed by installing moving image editing software and database software in a server device including a processing unit having a CPU, a storage unit having a storage device such as an HDD, and a communication unit having a LAN port. . As shown in FIG. 2, the moving image editing software includes a template management unit 11, a classifier creating unit 12, and a composite moving image creating unit 13. As shown in FIG. 2, the database software manages a template DB 21, a learning data DB 22, a synthetic moving image DB 23, and an embedded material DB 24.

  The template management unit 11 manages a plurality of templates stored in the template DB 21. Each template is composed of a plurality of cuts, and the screen layout and the reproduction time are defined for each cut. An image file may be assigned to the template, and the image file to be assigned to the template can be selected from the embedded material DB 24. FIG. 3 is an example of a screen layout of cuts that form a template. In the figure, the edited article material (text sentence) is inserted in the telop field 31, the selected image material is inserted in the image field 32, and the logo material is inserted in the logo field 33. Style information, color information, and tag information are attached to each template. The color information and the tag information are used by the template recommendation unit 134 described later.

  The classifier creating unit 12 creates learning classifier 133, which is a learned model, by acquiring learning data from the learning data DB 22 and performing machine learning. The classifier 133 is created by the classifier creating unit 12 several times a year, for example. As the learning data, data collected from the Internet or data labeled in-house may be used, or a labeled data set may be procured and used.

  As shown in FIG. 4, the composite moving image creating unit 13 includes a material reading unit 131, a material analyzing unit 132, a classifier 133, a template recommending unit 134, an image inserting unit 135, a telop inserting unit 136, and music. And an insertion portion 137.

The material reading unit 131 displays the material input screen page on the web browser of the user terminal 3. The material reading unit 131 includes an image data input unit that receives image data input and a character data input unit that receives character data input.
FIG. 5 is a diagram illustrating an example of a material input screen output by the material reading unit 131. The user can submit the image / moving image material by dragging and dropping the image data (image / moving image) to the material uploading field 51 or the logo uploading field 52. The number of image / moving image materials to be submitted can be increased or decreased by clicking the + button or the-button.
The character data (article material) can be submitted by inputting a text sentence in the title field 53 and the body field 54. When the recommended format button 55 is clicked after each material is submitted, the recommended format is output in the recommended format display field. In the example of FIG. 5, only one recommended format is displayed, but a specification in which a plurality of recommended formats are displayed in the order of the degree of recommendation may be adopted. Unlike the embodiment, a specification may be adopted in which the user manually selects an arbitrary format from a plurality of formats. In addition, a specification for inputting a text sentence may be adopted by converting the voice in a video file into a voice by converting the voice into a text, for example, using a voice recognition service provided as an external API. Good. Further, a specification may be adopted in which article materials and image / moving picture materials are acquired from the URL input by the user and stored in the embedded material DB 24.

The material analysis unit 132 calculates the degree of association between the information extracted from the input material and the tag information assigned to the format. Information extraction from the material is performed by the classifier 133.
The classifier 133 is a learned model that uses a convolutional neural network, and extracts specific information (annotation word) when a moving image or an image is input. The first classifier of the example embodiment outputs a word (e.g., stylish, simple) representing a classification of a moving image or an image. The second classifier in the example embodiment outputs a word (for example, seafood, yakiniku, a person, furniture) that represents an object that appears in a moving image or an image.

The template recommending unit 134 recommends the format having the highest degree of association to the user based on the output of the material analyzing unit 132.
The image inserting unit 135 is a trained model using a convolutional neural network, calculates the information extracted from the input material, the word assigned to each cut, the color and the lightness similarity, and calculates the similarity. Assign high images and videos to each cut. Here, not only the image / moving image material input by the user, but also the image / moving image material previously stored in the embedded material DB 24 may be determined and assigned.
The word similarity is determined by preparing a learned model in which a word vector is learned, and using the vector, the word similarity is determined by a method such as cosine similarity or Word Mover's Distance. The color similarity is determined by calculating the Euclidean distance in CIELAB coordinates, for example. Specifically, in all the pixels of the image, the color difference from the color to be compared is calculated, and the one with the smaller total value is determined to have the closer color difference. For moving images, this calculation is performed on images sampled from frame images of moving images. The determination of the brightness is performed by converting the image to gray scale and comparing the RMS contrast values of the pixels, for example. For a moving image, this calculation is performed on an image sampled from a frame image of the moving image, similarly to the determination of color.
The telop insertion unit 136 inserts a telop (text sentence) into each cut of the selected template. Details of the abstract sentence creating function of the telop insertion unit 136 will be described later.
The music insertion unit 137 inserts music that becomes BGM into the moving image content.

The administrator terminal 2 and the user terminal 3 are computers including an input unit, a display unit, a processing unit, a storage unit, and a communication unit. For example, a Web site such as a smartphone, a tablet terminal (tablet PC), a laptop computer, or a desktop personal computer. A computer with a browser.
The administrator uses the administrator terminal 2 to change the settings of the video editing server 1 and manage the operation of the database.
The user can access the moving image editing server 1 using the user terminal 3 and browse the created moving image content.

As shown in FIG. 6, the moving image editing system of the present invention includes a material analyzing step of analyzing a material input by a user, a format selecting step of selecting a format of a moving image to be created, a telop and image data (still image) in the selected format. -Perform the video composition process of assigning video).
In the material analysis process, the material analysis unit 132 analyzes materials such as articles, images (still images), and moving images, and the template recommendation unit 134 selects a format highly relevant to the input material and recommends it to the user. For example, if the tag information extracted from the uploaded material is “person”, “nail”, or “soap”, it is determined that it is close to “cosmetics” or “female”, and the format A with a cute style is added. Is recommended. Further, the recommended format may be selected based on the color information extracted from the material, or the recommended format may be selected using both the color information and the tag information.

  In the format selection step, the user selects the format of the moving image used for the moving image configuration. The user may adopt the format most recommended by the template recommending unit 134 as it is, select a favorite format from a plurality of recommended formats, or select his / her own format from among the formats not recommended. You may select any format you like. Each cut that makes up the format has a decorated frame, and also has animated effects on the inserted image (eg effects like split, fade, slide in / out, spin, turn in PowerPoint). It may be set.

In the moving image composition process, the telop insertion unit 136 creates a summary sentence extracted from the material and assigns it to each cut, and the image insertion unit 135 performs processing to assign the image / moving image extracted from the material to each cut. As shown in FIG. 7, lightness information, color information, and tag information such as people, cars, businesses, and natural scenery are associated with each cut forming the format. The image inserting unit 135 determines the most suitable image / moving image for each cut based on the similarity of the word, the color, and the brightness, and assigns it to each cut. In the embodiment, the image inserting unit 135 assigns the image / moving image material input by the user to each cut, but it is also possible to adopt a specification to assign the image / moving image material prepared in advance to each cut.
When the allocation of the images / moving images is completed, a plurality of cuts forming the moving image content can be displayed as a list on the screen as shown in FIG. For each cut, information on the reproduction time (seconds) of each cut is displayed along with the displayed image / moving picture and telop. The user can correct the telop by clicking the text button or the text field, and can replace the image by clicking the image button.

  BGM can be inserted into the created moving image content by the music inserting unit 14. When the BGM button is clicked on the screen of FIG. 8, a list of insertable BGMs is displayed as shown in FIG. Click the play button to the right of the song title to start playing the song. By clicking the check box to the left of the song title and clicking the preview button, it is possible to play back a preview of the video content in which BGM has been inserted.

(Summary sentence creation function)
The summary sentence creating function of the telop insertion unit 136 will be described with reference to FIGS.
STEP 91: Paragraph division / document division The telop insertion unit 136 divides the document input in the body field 54 into paragraphs, and divides the documents in each paragraph into documents. In addition, a sentence (for example, 80 characters or more) that is too long to be displayed as a telop of a moving image and deteriorates readability is further divided into a plurality of sentences at a portion satisfying a condition such as a specific part of speech or notation.
STEP 92: Morphological Analysis of Document The telop insertion unit 136 subjects each sentence to morphological analysis, and extracts a token that is the minimum unit of syntactic analysis. As shown in FIG. 11, a part of speech is attached to each token.

STEP 93: Delete Unnecessary Words / Unnecessary Paragraphs The telop insertion unit 136 deletes sentences and paragraphs defined as invalid according to a predefined invalid sentence determination rule. For example, a line starting from a specific symbol such as “■” or “▼”, a paragraph surrounded by the specific symbol, a paragraph in which a URL, a mail address, an address / telephone number, etc. are described is deleted.
STEP94: Delete stop words, etc. The telop insertion unit 136 uses words such as "ni", "kara", "kore", and "san" that are not so important words (stop words) and specific parts of speech such as particles. delete.
STEP95: Creation of token bigram A token bigram is obtained by connecting a plurality of tokens satisfying a specific condition (for example, a predefined part-of-speech condition). For example, “2014” (noun, proper noun, general) and “June” (noun, proper noun, general) are connected to be “June 2014”, or “Verdi” (proper noun) and “sponsor” Connect (ordinary nouns) and say "Sponsor Verdi".

STEP 96: Extraction of Important Sentences Tokens and token viagrams, which are characteristic words, are extracted from an index that evaluates the importance of words such as TF-IDF based on tokens and token bigrams, and sentence segmentation is performed based on the aforementioned word similarity determination. The summary is made by extracting important sentences from each segment.
STEP97: Fitting to template The summary (important sentences) is subjected to syntactic analysis and divided into clauses and syntactic trees. In the above template, the number of characters that can be inserted in each cut is defined. Based on the modification relation between clauses, sentences are divided so that a natural section as a sentence fits in each template and the template is applied. FIG. 12 shows an example of inserting a sentence into each cut of the template.
The above-described summary sentence creating function can support not only Japanese but also multiple languages such as English.

<Video distribution server>
A video distribution server having a function of distributing the created video content may be added. The external Web server 5 is used to transmit the Web page 6 in which the moving image distribution script is incorporated.
As shown in FIG. 13, the moving image distribution server 4 includes a Web scraper, an article body extracting unit, an article body analyzing unit, a moving image analyzing unit, a moving image delivering unit, a hash value database, and a moving image database. ing. The article text analysis unit and the moving image analysis unit may be referred to as a content analysis unit. The moving picture distribution server 4 can be realized by a plurality of server devices.
The web scraper collects content information of web pages for which the distribution side of moving images is set.
The article body extraction unit is software that extracts a portion corresponding to the body of the article from the collected content information, and can be configured by a tool such as Readability.
The article text analysis unit subjects the extracted article text to morphological analysis, extracts important keywords by a method such as TF-IDF, and creates a hash value of the text hash value generated using hash value generation MinHash or b-bit Minwise Hashing. Save in the database with the URL of the web page.

The moving picture analysis unit subjects the moving picture telops in the collected content information to morphological analysis, extracts important keywords by a method such as TF-IDF, generates a telop hash value by the same method as the article text analysis unit, and calculates a hash value. Save to database. The moving image analysis unit obtains the similarity from the generated telop hash value and the stored article hash value, and stores the similarity.
When the script incorporated in the web page 6 requests the acquisition of a video, the video distribution unit acquires from the video database a video that has a high degree of similarity from the URL of the received web page 6 and is currently available for distribution. And deliver the video. More specifically, the request URL transmitted when the web page 6 requests a video includes the URL of the web page 6 as a parameter, and the video distribution server 4 determines the URL of the web page 6 from the received parameter. A video that is taken out and has a high degree of similarity is selected based on the analysis result by the video analysis unit, and is delivered to the Web page 6.
The moving image created by the moving image editing server 1 can be stored in the moving image database. According to the video distribution server 4 of the embodiment, for example, by distributing a video advertisement having a high degree of similarity to the article body of the Web page 6, it is possible to efficiently perform a targeting advertisement that is of interest to the user. Is.

According to the video editing system of the embodiment described above, it is possible to easily create video content without having to prepare video editing software, a video server, an editor with specialized technology, etc. . For example, it is expected to be used in the following situations.
1) Animation of product information sold at EC shops 2) Video distribution of press release information, CSR information, etc. 3) Animation of manuals such as usage and operation flow 4) Creation of creatives that can be used as video advertisements

  Although the preferred embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the description of the above embodiments. Various modifications and improvements can be added to the above-described embodiment, and such modifications and improvements are also included in the technical scope of the present invention.

1 Video Editing Server 2 Administrator Terminal 3 User Terminal 4 Video Distribution Server 5 External Web Server 11 Template Management Unit 12 Classifier Creation Unit 13 Synthetic Video Creation Unit 21 Template DB
22 Learning data DB
23 Synthetic video DB
24 Embedded Material DB

Claims (6)

A server,
A template management unit that manages a template composed of a plurality of cuts with tag information,
A material reading unit that accepts input of image data,
A material analysis unit that recognizes an object in the input image data and outputs a degree of similarity with the tag information,
An image insertion unit that allocates image data to the template based on the similarity.
A server characterized by that. It is a trained model that has been machine-learned using the training data, equipped with a classifier that outputs the annotation words of the input image data,
The server according to claim 1, wherein the similarity output by the material analysis unit is a word similarity between the annotation word output by the classifier and the tag information. Color information is attached to the template,
The material analysis unit outputs the color similarity between the image data and the color information,
The server according to claim 1, wherein the image inserting unit allocates image data to the template based on the word similarity and / or the color similarity.   4. The server according to claim 1, wherein the material reading unit has a function of acquiring image data from a database or the Web in addition to the input image data. A video distribution system comprising the server according to any one of claims 1 to 4 for creating video content, and a video distribution server,
The video distribution server
A web scraper that collects the content information of the web page for which the distribution side of the video is set,
A content analysis unit that analyzes the content information and analyzes the degree of similarity with the video content created in advance;
A moving image distribution system, comprising: a moving image advertisement distribution unit that distributes the moving image content having a high degree of similarity. In the server program,
The server
A template management unit that manages a template composed of a plurality of cuts with tag information,
A material reading unit that accepts input of image data,
A material analysis unit that recognizes an object in the input image data and outputs a degree of similarity with the tag information,
A server program that functions as an image insertion unit that allocates image data to the template based on the degree of similarity.

Images