JP2020096373A - Server, program, and video distribution system - Google Patents

Abstract

To provide a server, a program, and a video distribution system that make it easy to create a video content.SOLUTION: A server includes a template management unit that manages a template composed of multiple cuts, a material reading unit that recognizes a material including voice and accepts the input as character data, and a telop insertion unit that allocates a text sentence created from the character data to the template.SELECTED DRAWING: Figure 2

Description

The present invention relates to a server and a program for automatically generating moving image content to be distributed to a user terminal, and a moving image distribution system.

Conventionally, a moving image is divided into a plurality of chapters and metadata is added.
For example, Patent Document 1 discloses a moving image processing apparatus that efficiently searches a moving image having a plurality of chapters for a scene image at a desired moment, and divides the moving image into a plurality of large blocks at predetermined unit time intervals. A block division unit, a complexity analysis unit that digitizes the complexity of image change of each large block, and a small block division that divides the playback time of each large block into a plurality of small blocks according to the numerical value of the complexity. There has been proposed a moving image processing apparatus including a section and a chapter creating section that creates a chapter by sequentially dividing a plurality of small blocks into a predetermined number in a time series.

JP, 2011-130007, A

Since it takes a lot of time and effort to create moving image content, it has been demanded to provide a system that can easily create moving image content.

Therefore, it is an object of the present invention to provide a server, a program, and a moving image distribution system that can easily create moving image contents.

The server of the present invention is a template management unit that manages a template composed of a plurality of cuts, a material reading unit that recognizes a material including a voice by voice and receives an input as character data, and is created from the character data. A telop insertion unit for allocating a text sentence to the template, the server.

In the above server, the sound may be sound in a moving image file.

In the above server, the voice recognition utilizes a voice recognition service provided as an external API.

A server program according to the present invention includes a template management unit that manages a template including a plurality of cuts, a material reading unit that recognizes a material including a voice and receives an input as character data, and the character. It is characterized in that it is made to function as a telop insertion section for allocating a text sentence created from data to the template.

According to the present invention, it is possible to provide a server, a program, and a moving image distribution system that can easily create moving image contents.

It is a block diagram of the moving image editing system which concerns on an example of embodiment. It is a block diagram of the video editing server which concerns on the example of embodiment. It is a figure explaining the example of a screen layout which comprises a template. It is a block diagram of a synthetic|combination moving image production|generation part. It is a figure explaining the example of a material input screen. It is a figure explaining each process for creating moving image contents from a material. It is explanatory drawing of the process which allocates the image and moving image most suitable for each cut. It is explanatory drawing of the aspect which carries out the list display of several cuts which comprise moving image content on a screen. It is explanatory drawing of the screen which inserts BGM in moving image content. It is a processing flow of a summary sentence creation function. It is a figure explaining the process which subjects a document to morphological analysis and divides it into tokens. It is a figure explaining insertion of a sentence to each cut of a template. It is a block diagram of the video delivery server which concerns on the example of embodiment.

<Structure>
As shown in FIG. 1, the moving picture editing system of the present invention according to the embodiment includes a moving picture editing server 1, an administrator terminal 2, and a plurality of user terminals 3. In the example of FIG. 1, the moving image editing server 1 is configured by one unit, but the moving image editing server 1 can be realized by a plurality of server devices.

The moving image editing server 1 is constructed by installing moving image editing software and database software in a server device including a processing unit having a CPU, a storage unit having a storage device such as an HDD, and a communication unit having a LAN port. .. As shown in FIG. 2, the moving image editing software includes a template managing unit 11, a classifier creating unit 12, and a composite moving image creating unit 13. As shown in FIG. 2, the database software manages a template DB 21, a learning data DB 22, a synthetic moving image DB 23, and an embedded material DB 24.

The template management unit 11 manages a plurality of templates stored in the template DB 21. Each template is composed of a plurality of cuts, and the screen layout and the reproduction time are defined for each cut. An image file may be allocated to the template, and the image file to be allocated to the template can be selected from the embedded material DB 24. FIG. 3 is an example of a screen layout of cuts that form a template. In the figure, the edited article material (text sentence) is inserted in the telop field 31, the selected image material is inserted in the image field 32, and the logo material is inserted in the logo field 33. Style information, color information, and tag information are attached to each template. The color information and the tag information are used by the template recommendation unit 134 described later.

The classifier creating unit 12 creates the classifier 133 described later, which is a learned model, by acquiring learning data from the learning data DB 22 and performing machine learning. The classifier 133 is created by the classifier creating unit 12 several times a year, for example. As the learning data, data collected from the Internet or data labeled in-house may be used, or a labeled data set may be procured and used.

As shown in FIG. 4, the composite moving image creating unit 13 includes a material reading unit 131, a material analyzing unit 132, a classifier 133, a template recommending unit 134, an image inserting unit 135, a telop inserting unit 136, and music. And an insertion portion 137.

The material reading unit 131 displays the material input screen page on the web browser of the user terminal 3. The material reading unit 131 includes an image data input unit that accepts input of image data and a character data input unit that accepts input of character data.
FIG. 5 is a diagram illustrating an example of the material input screen output by the material reading unit 131. The user can submit the image/moving image material by dragging and dropping the image data (image/moving image) to the material submission field 51 or the logo submission field 52. The number of image/moving image materials to be submitted can be increased or decreased by clicking the + button or the-button.
The character data (article material) can be submitted by inputting a text sentence in the title field 53 and the body field 54. When the recommended format button 55 is clicked after each material is submitted, the recommended format is output in the recommended format display field. In the example of FIG. 5, only one recommended format is displayed, but a specification in which a plurality of recommended formats are displayed in order of the degree of recommendation may be adopted. Unlike the example of the embodiment, a specification may be adopted in which the user manually selects an arbitrary format from a plurality of formats. In addition, a specification for inputting a text sentence by recognizing voice in a video file and converting the voice into a text may be adopted. For example, a voice recognition service provided as an external API is used. Good. Further, a specification may be adopted in which article materials, image/moving picture materials are acquired from the URL input by the user and stored in the embedded material DB 24.

The material analysis unit 132 calculates the degree of association between the information extracted from the input material and the tag information assigned to the format. Information extraction from the material is performed by the classifier 133.
The classifier 133 is a learned model that uses a convolutional neural network, and extracts specific information (annotation word) when a moving image or image is input. The first classifier of the example embodiment outputs a word (e.g., stylish, simple) representing a classification of a moving image or an image. The second classifier of the example embodiment outputs a word (for example, seafood, yakiniku, a person, furniture) that represents an object that appears in a moving image or an image.

The template recommendation unit 134 recommends the format having the highest degree of association to the user based on the output of the material analysis unit 132.
The image inserting unit 135 is a trained model using a convolutional neural network, calculates the similarity between the information extracted from the input material and the word, color, and lightness assigned to each cut, and determines the highest similarity. Assign high images and videos to each cut. Here, not only the image/moving image material input by the user but also the image/moving image material previously stored in the embedded material DB 24 may be determined and assigned.
To determine the degree of similarity between words, a learned model prepared by learning a word vector is prepared, and the vector is used to determine the degree of similarity between words by methods such as cosine similarity and Word Mover's Distance. The color similarity is determined by calculating the Euclidean distance in CIELAB coordinates, for example. Specifically, in all the pixels of the image, the color difference from the color to be compared is calculated, and the one having the smaller total value is determined to have the closer color difference. For moving images, this calculation is performed on images sampled from the frame images of the moving image. The brightness is determined by converting the image to grayscale and comparing the RMS contrast values of the pixels, for example. For a moving image, this calculation is performed on an image sampled from a frame image of the moving image, similarly to the determination of color.
The telop insertion unit 136 inserts a telop (text sentence) into each cut of the selected template. The details of the abstract sentence creating function by the telop insertion unit 136 will be described later.
The music insertion unit 137 inserts music that becomes BGM into the moving image content.

The administrator terminal 2 and the user terminal 3 are computers including an input unit, a display unit, a processing unit, a storage unit, and a communication unit, and for example, a Web such as a smartphone, a tablet terminal (tablet PC), a laptop computer, or a desktop personal computer. A computer with a browser.
The administrator uses the administrator terminal 2 to change the settings of the video editing server 1 and manage the database.
The user can access the moving image editing server 1 through the user terminal 3 and browse the created moving image content.

As shown in FIG. 6, the moving image editing system of the present invention includes a material analyzing step of analyzing a material input by a user, a format selecting step of selecting a format of a moving image to be created, a telop and image data (still image) in the selected format. -Perform the video composition process of assigning video).
In the material analysis step, the material analysis unit 132 analyzes materials such as articles, images (still images), and moving images, and the template recommendation unit 134 selects a format that is highly relevant to the input material and recommends it to the user. For example, if the tag information extracted from the uploaded material is “person”, “nail”, or “soap”, it is determined that it is close to “cosmetics” or “female”, and the format A with a cute style is added. Is recommended. Also, the recommended format may be selected based on the color information extracted from the material, or the recommended format may be selected using both the color information and the tag information.

In the format selection step, the user selects the format of the moving image used for the moving image configuration. The user may directly adopt the format most recommended by the template recommending unit 134, may select a favorite format from a plurality of recommended formats, or may select his/her own format from among the formats not recommended. You may select any format you like. Each cut that makes up the format has a decorated frame, and also has animated effects on the inserted image (for example, effects like split, fade, slide in/out, spin, turn in PowerPoint). It may be set.

In the moving image composition process, the telop insertion unit 136 creates a summary sentence extracted from the material and assigns it to each cut, and the image insertion unit 135 performs processing to assign the image/moving image extracted from the material to each cut. As shown in FIG. 7, lightness information, color information, and tag information such as people, cars, businesses, and natural scenery are associated with each cut forming the format. The image inserting unit 135 determines an image/moving image most suitable for each cut based on the similarity of the word, the color, and the lightness, and assigns the cut/image to each cut. In the embodiment, the image inserting unit 135 assigns the image/moving image material input by the user to each cut, but a specification that assigns the image/moving image material prepared in advance to each cut may be adopted.
When the image/moving image allocation is completed, a plurality of cuts forming the moving image content can be displayed as a list on the screen as shown in FIG. For each cut, information about the reproduction time (seconds) of each cut is displayed together with the displayed image/moving picture and telop. The user can correct the telop by clicking the text button or the text field, and can replace the image by clicking the image button.

BGM can be inserted into the created moving image content by the music inserting unit 14. When the BGM button is clicked on the screen of FIG. 8, a list of insertable BGMs is displayed as shown in FIG. Click the play button to the right of the song title to start playing the song. By clicking the check box to the left of the song title and clicking the preview button, it is possible to play back a preview of the video content in which BGM has been inserted.

(Summary sentence creation function)
The summary sentence creating function of the telop insertion unit 136 will be described with reference to FIGS.
STEP 91: Paragraph division/document division The telop insertion unit 136 divides the document input in the body field 54 into paragraphs, and divides the documents in each paragraph into documents. Further, a sentence (for example, 80 characters or more) that is too long to display as a telop of a moving image and deteriorates readability is further divided into a plurality of sentences at a portion satisfying a condition such as a specific part of speech or notation.
STEP 92: Morphological Analysis of Document The telop insertion unit 136 subjects each sentence to morphological analysis, and extracts a token that is the minimum unit of syntactic analysis. As shown in FIG. 11, a part-of-speech is attached to each token.

STEP 93: Delete Unnecessary Words/Unnecessary Paragraphs The telop insertion unit 136 deletes sentences and paragraphs that are defined as invalid according to the predefined invalid sentence determination rule. For example, a line starting from a specific symbol such as “■” or “▼”, a paragraph surrounded by the specific symbol, a paragraph in which a URL, a mail address, an address/telephone number, etc. are described is deleted.
STEP 94: Deletion of Stop Words, etc. The telop insertion unit 136 uses specific tokens such as “ni”, “kara”, “kore”, “san” and other words that are not so important in meaning (stop words) and particles. delete.
STEP95: Creation of token bigram A token bigram is obtained by connecting a plurality of tokens satisfying a specific condition (for example, a predefined part-of-speech condition). For example, “2014” (noun, proper noun, general) and “June” (noun, proper noun, general) are connected to make “June 2014”, or “Verdi” (proper noun) and “sponsor” Connect (ordinary nouns) and say "sponsored Verdi".

STEP96: Extraction of Important Sentences Tokens and token viagrams, which are characteristic words, are extracted from an index that evaluates the importance of words such as TF-IDF based on tokens and token bigrams, and sentence segmentation is performed based on the aforementioned word similarity determination. The summary is made by extracting important sentences from each segment.
STEP97: Fitting to template The summary (important sentences) is subjected to syntactic analysis and divided into clauses and syntactic trees. Where the number of characters that can be inserted into each cut is defined in the above template, sentences are divided so that a natural interval as a sentence fits into each template from the modification relation between clauses, and the template is applied. FIG. 12 shows an example of inserting a sentence into each cut of the template.
The abstract sentence creating function described above can support not only Japanese but also multiple languages such as English.

<Video distribution server>
A video distribution server having a function of distributing the created video content may be added. The external Web server 5 is used to transmit the Web page 6 in which the moving image distribution script is incorporated.
As shown in FIG. 13, the moving image distribution server 4 includes a web scraper, an article body extracting unit, an article body analyzing unit, a moving image analyzing unit, a moving image delivering unit, a hash value database, and a moving image database. ing. The article body analysis unit and the moving image analysis unit may be referred to as a content analysis unit. The moving image distribution server 4 can be realized by a plurality of server devices.
The web scraper collects the content information of the web page for which the distribution surface of the moving image is set.
The article body extraction unit is software that extracts a portion corresponding to the body of the article from the collected content information, and can be configured by a tool such as Readability.
The article text analysis unit subjects the extracted article text to morphological analysis, extracts important keywords by a method such as TF-IDF, and generates a hash value by using a hash value generation MinHash or b-bit Minwise Hashing. Save in the database with the URL of the web page.

The moving picture analysis unit subjects the moving picture telops in the collected content information to morphological analysis, extracts important keywords by a method such as TF-IDF, generates a telop hash value in the same manner as the article text analysis unit, and calculates a hash value. Save to database. The moving image analysis unit obtains the degree of similarity from the generated telop hash value and the stored article hash value, and stores the similarity.
When the script embedded in the web page 6 requests the acquisition of a video, the video distribution unit acquires from the video database a video that has a high degree of similarity from the URL of the received web page 6 and is currently available for distribution. And deliver the video. More specifically, the request URL transmitted when the web page 6 requests a video includes the URL of the web page 6 as a parameter, and the video distribution server 4 determines the URL of the web page 6 from the received parameter. A video that is taken out and has a high degree of similarity is selected based on the analysis result by the video analysis unit, and is delivered to the Web page 6.
The moving image database can store moving images created by the moving image editing server 1. According to the moving image distribution server 4 of the embodiment, for example, by distributing a moving image advertisement having a high similarity to the article body of the Web page 6, it is possible to efficiently perform a targeting advertisement that is of interest to the user. Is.

According to the moving picture editing system of the embodiment described above, it is possible to easily create moving picture contents without having to prepare the moving picture editing software, the moving picture server, the editors having specialized technology, etc. .. For example, it is expected to be used in the following situations.
1) Animation of product information sold at EC shops 2) Video distribution of press release information, CSR information, etc. 3) Animation of manuals for usage and operation flow 4) Creation of creatives that can be used as video advertisements

Although the preferred embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the description of the above embodiments. Various modifications and improvements can be added to the above-described embodiment, and such modifications and improvements are also included in the technical scope of the present invention.

1 Video Editing Server 2 Administrator Terminal 3 User Terminal 4 Video Distribution Server 5 External Web Server 11 Template Management Unit 12 Classifier Creation Unit 13 Synthetic Video Creation Unit 21 Template DB
22 Learning data DB
23 Synthetic video DB
24 Embedded Material DB

Claims (5)

A server,
A template management unit that manages a template composed of multiple cuts,
A material reading unit that recognizes material including voice and accepts input as character data,
A telop insertion unit for allocating a text sentence created from the character data to the template,
A server characterized by that. The audio is audio in a video file,
The server according to claim 1, wherein: The voice recognition uses an API,
The server according to claim 1, wherein the server is a server. The API is an external API,
The voice recognition uses a voice recognition service provided as the external API,
The server according to claim 3, wherein: In the server program,
The server
A template management unit that manages a template composed of multiple cuts,
A material reading unit that recognizes material including voice and accepts input as character data,
A telop insertion unit that allocates a text sentence created from the character data to the template,
A server program characterized by functioning as a server.

Images