JP6603929B1 - Movie editing server and program - Google Patents

Abstract

Provided is a server and a program capable of easily creating moving image content. In a server for creating a moving image to be distributed to a user terminal, a recommended article extracting unit that extracts a recommended article including a material moving image based on user preference profile information, and a plurality of scenes with scene labels A template creation unit that creates a template in which a material video is defined, and a composite video creation unit that creates a composite video based on the material video and the template, wherein the composite video creation unit converts the material video into a plurality of divided material videos A moving image dividing unit that divides the divided material moving image, a first classifier that adds a scene label to the divided material moving image, and a divided moving image that matches each scene of the template based on the scene label A moving image editing server, and a program for the same. [Selection] Figure 2

Description

  The present invention relates to a server and a program for automatically generating a synthetic moving image to be distributed to a user terminal.

Conventionally, a moving image is divided into a plurality of chapters and metadata is added.
For example, Patent Document 1 discloses a moving image processing apparatus that efficiently searches for a scene image at a desired moment from a moving image having a plurality of chapters. The moving image is divided into a plurality of large blocks every predetermined unit time. A block division unit, a complexity analysis unit that quantifies the complexity of the image change of each large block, and a small block division that divides the playback time of each large block into a plurality of small blocks according to the numerical value related to the complexity There has been proposed a moving image processing apparatus including a section and a chapter creating section that creates chapters by sequentially dividing a plurality of small blocks into a predetermined number along a time series.

JP 2011-130007 A

Since it takes a lot of time and effort to create video content, it has been required to provide a system that can automatically create video content. In addition, even video content created with a great deal of effort is not reproduced until the end unless the content matches the user's preference.
In addition, there is a need to animate existing materials such as in-house manuals and product catalogs, but there is a high threshold for building a video editing environment such as video servers and editors with specialized technology. In the case of outsourcing, there is a problem that the outsourcing cost increases.

  Therefore, an object of the present invention is to provide a server and a program for automatically generating moving image content in order to solve the above-described problems.

  The video editing server of the present invention is a server that creates a video for distribution to a user terminal, and a recommended article extraction unit that extracts a recommended article including a material video based on the user's preference profile information, and a scene label. A template creation unit that creates a template in which a plurality of scenes are defined, and a composite video creation unit that creates a composite video based on the material video and the template, and the composite video creation unit includes a plurality of material video A video dividing unit that divides the video into divided material videos, a first classifier that assigns a scene label to the divided material video, and a split material video that matches each scene of the template based on the scene label and combines them. And a divided moving image combining unit for creating a combined moving image.

In the moving image editing server, the synthesized moving image creating unit may include a second classifier that attaches an object name label representing an object name of an object displayed in the divided material moving image.
In the moving image editing server, each scene of the template is attached with object information defined as a superordinate concept of the object name label, and the divided moving image combining unit adds the scene label and the object name label. On the basis of this, a divided material moving image that matches each scene of the template may be extracted.
In the video editing server, the recommended article extraction unit extracts the recommended article with a recommendation value based on similarity information with an article viewed by a user, and the divided video combining unit has a high recommendation value. A feature may be that the divided material video obtained from the recommended article is preferentially adopted.
In the moving image editing server, the synthesized moving image creating unit may include a telop inserting unit that inserts a telop into the synthesized moving image.
In the moving image editing server, the telop insertion unit may insert a telop created by summarizing character information described in the recommended article into the synthesized moving image.
In the moving image editing server, an allowable time range of each scene is defined in the template, and the divided moving image combining unit extracts a divided material moving image that matches the allowable time range.
In the moving image editing server, the moving image dividing unit has a moving image for learning for generating learning data of the first classifier in addition to a moving image dividing function for material for dividing the material moving image into a plurality of divided material moving images. A division function may be provided.

  The video editing server program according to the present invention is a server video editing program that distributes video content to a user terminal accessed via the Internet. The server includes a recommended article including a material video based on user preference profile information. A recommended article extraction unit that extracts a template, a template creation unit that creates a template in which a plurality of scenes with scene labels are defined, and a composite video creation unit that creates a composite video based on the material video and the template, Functioning, wherein the synthesized moving image creating unit divides the material moving image into a plurality of divided material moving images, a first classifier for assigning a scene label to the divided material moving image, and the template based on the scene label Extract and combine the segmented material videos that match each scene Characterized in that it comprises a divided moving coupling portion to create a picture, a.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the server and program which produce | generate automatically the moving image content according to a user preference. In addition, it is possible to easily animate existing materials such as in-house manuals and product catalogs without building a video editing environment in the company.

1 is a configuration diagram of a moving image editing system according to an embodiment. It is a block diagram of the moving image editing server which concerns on the example of embodiment. It is a figure explaining an example of a template. It is a block diagram of a moving image creation part. It is a figure explaining extraction of the division material animation which matches the conditions of each scene. It is a flow of a moving image editing process. It is explanatory drawing of a moving image division | segmentation process. It is explanatory drawing of a division | segmentation moving image combination process. It is a flow of a summary sentence creation process.

<Configuration>
As shown in FIG. 1, the moving image editing system according to the embodiment of the present invention includes a moving image editing server 1, an administrator terminal 2, and a plurality of user terminals 3. In the example of FIG. 1, the example in which the moving image editing server 1 is configured as one unit has been described. However, the moving image editing server 1 can be realized by a plurality of servers, and is a high-spec device equipped with a GPU. It is preferable from the viewpoint of work efficiency to provide a separate and perform operations such as creating a classifier described later.

  The moving image editing server 1 is constructed by installing moving image editing software and database software in a server device including a processing unit having a CPU, a storage unit having a storage device such as an HDD, and a communication unit having a LAN port. . The video editing software includes a browsing history storage unit 11, a recommended article extraction unit 12, a template creation unit 13, a classifier creation unit 14, and a composite video creation unit 15. The database software manages a browsing record DB 21, a recommended article DB 22, a template DB 23, a learning data DB 24, a synthesized moving image DB 25, and a label DB 26.

  The browsing history storage unit 11 records browsing records of articles on the Web browsed by the user in a logged-in state in the browsing record DB. A unique article ID is assigned to an article on the Web viewed by the user, and is recorded in association with the user ID and the viewing time. Unlike this, a mechanism for recording browsing records as an anonymous user may be adopted.

  The recommended article extraction unit 12 periodically (for example, weekly) performs a task of collecting recommended articles from the Web based on the user's preference profile. In the embodiment, the user's browsing history information is acquired from the browsing record DB 21, an article with a moving image similar to the article browsed by the user is extracted by content-based filtering, and a recommended article is attached with a recommendation value based on similarity information. It is recorded in DB22. Unlike this, the similarity may be determined by collaborative filtering, or hybrid filtering using both content-based filtering and collaborative filtering may be employed. In the recommended article DB 22, dozens or hundreds of recommended articles with high recommendation values are always stored.

  The template creation unit 13 creates a template that defines conditions (label, order, and allowable time range of each scene) attached to the scenes that make up the moving image, and stores them in the template DB 23. The template is composed of a plurality of scenes, and each scene defines object information to be displayed as a divided moving image and characteristics of its movement. In other words, for example, the flow of the moving image content is defined as “moving the building corridor”-> “zoom-out of the building room”-> “appearance”, and there is no sense of incongruity when the scenes are connected. It is important to define the object information and the characteristics of its movement. Template creation by the template creation unit 13 is executed regularly (for example, several times a year) or when necessary. FIG. 3 shows an example of a template composed of four scenes. Scenes 1 and 2 are assigned a “cooking” scene label, scene 3 is assigned a “after cooking” scene label, Is labeled with a scene label of “interior / appearance”. Note that an object name label (to be described later) can be attached to each scene of the template, but it is preferable that many objects can be applied by defining the object information of the superordinate concept of the object name label. In some cases, the allowable number of characters for the telop inserted in each scene is set as a condition.

  The classifier creating unit 14 creates the first classifier 153 and the second classifier 154 that are learned models by acquiring learning data from the learning data DB 24 and performing machine learning. The creation of the first classifier uses a moving image data set, and learns the dependency of time-series features of each scene. The second classifier learns the feature of the object based on the image data set. The classifier that is the learned model by the classifier creating unit 14 is created several times a year, for example. The learning data may be data collected from the Internet or in-house data with a label, or a data set with a label may be procured and used.

As shown in FIG. 4, the synthesized video creating unit 15 includes a recommended article reading unit 151, a video dividing unit 152, a first classifier 153, a second classifier 154, a divided video combining unit 155, and music insertion. Part 156 and a telop insertion part 157.
The recommended article reading unit 151 acquires from the recommended article DB 22 a recommended article that includes a material video that is a material for creating a composite video.

The moving image dividing unit 152 divides the material moving image into frames, determines a dividing position by setting a portion where the color greatly changes from the immediately preceding frame as a section, and creates a divided material moving image (see FIG. 7). More specifically, for example, the color of all the pixels in the image of the frame is digitized, the average color of the colors of all the pixels is calculated, and the image is divided at a location that has changed significantly compared to the average color of the previous frame. To create a split material video. Such a dividing method is useful when a scene having a good visual connection is grouped as one divided moving image, and a divided material moving image connected for a relatively long time is generated.
The moving image dividing unit 152 is also used to create a finely divided moving image used as learning data. More specifically, for example, the color of all the pixels in the image of the frame is digitized, the change in the color saturation of black is determined from the brightness of the pixel, and when it becomes dark to some extent, it is divided to create a divided video for learning data To do.
As described above, the moving image dividing unit 152 according to the present embodiment uses the moving image dividing technique depending on the purpose.

The first classifier 153 is a classifier using a recurrent neural network. When a moving image is input, the first classifier 153 outputs a label as a classification result in which movement is also determined and stores the label in the label DB 26. More specifically, the first classifier 153 is used to attach a word (annotation word) representing a scene state such as during cooking or in-store interior to a divided material moving image as a scene label.
The second classifier 154 is a classifier using a convolutional neural network. When a moving image or image is input, an object name label appearing in the moving image or image is output and stored in the label DB 26. More specifically, the second classifier 154 is used to attach a word (annotation word) representing an object name such as seafood, yakiniku, a person, furniture, or the like to the divided material moving image as an object name label. A divided moving image to be extracted based on the metadata obtained by the second classifier 154 is selected. For example, when it is determined that seafood is included in the divided video extracted from the first video, the split video in which the seafood is reflected is also extracted preferentially from the second and third videos.

The divided moving image combining unit 155 selects a template having the highest affinity for the material moving image, extracts a divided material moving image that matches the conditions of each scene of the selected template, and combines the extracted divided material moving images. FIG. 5 is a diagram for explaining the extraction of the divided material moving images that match the conditions of each scene. Of the divided material moving images divided into 10, the sixth to ninth divided material moving images from the start meet the conditions. Has been extracted.
The divided moving image combining unit 155 extracts divided material moving images in order from the highest recommended value for the plurality of extracted recommended articles. If the segmented material video that matches all the scenes of the template is not extracted by the first recommended article, matching the segmented material video that matches the conditions of the missing scene for the recommended article with the next highest recommendation value By performing the above, a divided material moving image for extracting all scenes required by the template is extracted. At this time, the metadata obtained from the second classifier 154 is used so that a divided moving image extracted from the moving image of each article is selected so that a similar object is reflected in each scene.

The music insertion unit 156 inserts music into the synthesized moving image (no music) combined by the divided moving image combining unit 155.
The telop insertion unit 157 inserts a telop into the combined moving image (no telop) combined by the divided moving image combining unit 155. If the allowable character count range of telops to be inserted into each scene is set as a condition, a telop having the number of characters satisfying the condition is inserted.

The administrator terminal 2 and the user terminal 3 are computers including an input unit, a display unit, a processing unit, a storage unit, and a communication unit. For example, a web such as a smartphone, a tablet terminal (tablet PC), a notebook computer, or a desktop personal computer. A computer with a browser.
The administrator uses the administrator terminal 2 to change the settings of the video editing server 1 and manage the operation of the database.
The user can access the moving image editing server 1 through the user terminal 3 and browse the automatically generated moving image content.

<Operation>
The flow of the moving image editing process will be described with reference to FIG.
The moving image generation agent periodically (for example, once a week) executes the automatic moving image generation process by the composite moving image creating unit 15 (STEP 1). The recommended article reading unit 151 of the synthesized video creating unit 15 reads the recommended articles stored in the recommended article DB 22 in order from the highest recommended value (STEP 2).
The moving image dividing unit 152 of the synthesized moving image creating unit 15 divides the moving image (material moving image) included in the recommended article to be processed into a plurality of moving images (divided material moving image) (STEP 3). The synthesized moving image creating unit 15 labels each divided material moving image with the first classifier 153 and the second classifier 154 (STEP 4). Note that the division position of the material moving image may be stored in the recommended article DB 22 and / or the learning data DB 24. The synthetic moving image creating unit 15 extracts a divided material moving image that matches the label and allowable time of the selected template (STEP 5). For example, in FIG. 8, the third scene condition of the template is “after cooking”, “2 to 4 seconds”, and the fourth scene condition is “interior / appearance” and “2 to 4 seconds”. Two divided material moving images satisfying these conditions are extracted.

  When the divided material moving images are not acquired for all the scenes, the operations of STEP 2 to 5 are repeated for the next recommended article (STEP 6 and STEP 7). For example, in FIG. 8, since the divided material moving images satisfying the first and second scene conditions are not extracted, it is necessary to continuously search for the divided material moving images satisfying the scene conditions for the next recommended article.

  When the divided material moving images are acquired for all the scenes, the divided moving image combining unit 155 of the combined moving image creating unit 15 combines the extracted divided material moving images to generate a combined moving image (STEP 8). The music insertion unit 156 of the synthetic video creation unit 15 inserts a BGM prepared in advance into the synthetic video (STEP 9). The telop insertion unit 157 of the synthetic video creation unit 15 inserts a telop into the synthetic video (STEP 10). The moving image in which music and telop are inserted is stored in the synthesized moving image DB 25 as a completed moving image, and the user can view the completed moving image within the authority of his user ID (STEP 11).

  According to the video editing system of the embodiment described above, it is possible to automatically generate video content that matches the user's preference based on the user's preference profile, and periodically distribute it as a recommended video. . It is also possible to create video advertisements and video press releases, and to animate manuals and product catalogs without having to prepare video editing software, video servers, editors with specialized technology, etc. .

<Modification>
The moving image editing server 1 according to a preferred aspect has a summary document creation function for insertion as a telop. A summary document creation flow will be described with reference to FIG.
STEP 91: Paragraph Division / Document Division The telop insertion unit 157 divides the input document into paragraphs, and divides the documents in each paragraph into documents. In addition, a sentence that is too long to be displayed in one scene as a moving picture telop is further divided into a plurality of sentences at a location that satisfies a specific part of speech or notation.
STEP 92: Document Morphological Analysis The telop insertion unit 136 subjects each sentence to morphological analysis, and extracts a token that is a minimum unit of syntax analysis.
STEP 93: Delete Unnecessary Words / Unnecessary Paragraphs The telop insertion unit 157 deletes sentences and paragraphs that are defined as invalid based on a predefined invalid sentence determination rule. For example, a line starting from a specific symbol such as “■” or “▼”, a line surrounded by the specific symbol, a URL, a mail address, an address / phone number, or the like is deleted.

STEP 94: Delete Stop Words etc. The telop insertion unit 157 adds specific parts of speech such as words (stop words) and particles that are not so important as tokens such as “ni”, “kara”, “this”, “san” from tokens. delete.
STEP 95: Creation of a token bigram A token bigram is obtained by connecting a plurality of tokens satisfying a specific condition (for example, a predefined part of speech condition). For example, “2014” (nouns, proper nouns, general) and “June” (nouns, proper nouns, general) are connected to “June 2014”, or “Verdi” (proper nouns) and “sponsorship” ”(Common nouns) and“ Verdi sponsorship ”.

STEP 96: Extraction of important sentences TF-IDF score based on tokens and token bigrams Tokens and token viagrams as feature words are extracted from an index for evaluating the importance of words, and sentence segmentation is performed from the above word similarity determination. The summary is obtained by extracting important sentences from each segment.
STEP 97: Fitting to template The summary (important sentence) is subjected to parsing, and separated into clauses and syntax trees. In the template described above, the number of characters required for each scene is defined. From the modification relationship between clauses, the sentence is cut so that a natural section as a sentence fits in each template and applied to the template.
The summary sentence creation function described above can be applied not only to Japanese but also to multiple languages such as English.

  The preferred embodiments of the present invention have been described above, but the technical scope of the present invention is not limited to the description of the above embodiments. Various modifications and improvements can be added to the above-described embodiment, and forms with such modifications or improvements are also included in the technical scope of the present invention.

1 video editing server 2 administrator terminal 3 user terminal 11 browsing history storage unit 12 recommended article extraction unit 13 template creation unit 14 classifier creation unit 15 composite video creation unit 21 browsing record DB
22 Recommended Article DB
23 Template DB
24 learning data DB
25 Synthetic Movie DB
26 Label DB
151 Recommended Article Reading Unit 152 Video Dividing Unit 153 First Classifier 154 Second Classifier 155 Divided Video Combining Unit 156 Music Inserting Unit 157 Telop Inserting Unit

Claims (9)

In a server that creates a video for distribution to a user terminal,
A recommended article extraction unit that extracts recommended articles including material videos based on user preference profile information;
A template creation unit for creating a template in which a plurality of scenes with scene labels are defined;
A composite video creation unit that creates a composite video based on the material video and the template,
The synthetic video creation unit is a video dividing unit that divides a material video into a plurality of divided material videos
A first classifier for attaching a scene label to the divided material animation;
A moving image editing server comprising: a divided moving image combining unit that extracts divided material moving images that match each scene of the template based on the scene label and combines them to create a combined moving image. The moving image editing server according to claim 1, wherein the synthetic moving image creating unit includes a second classifier that attaches an object name label representing an object name of an object displayed in the divided material moving image. Each scene of the template is attached with object information defined as a superordinate concept of the object name label,
The moving image editing server according to claim 2, wherein the divided moving image combining unit extracts a divided material moving image that matches each scene of the template based on the scene label and the object name label. The recommended article extraction unit extracts the recommended article with a recommendation value based on similarity information with an article viewed by a user ;
4. The moving image editing server according to claim 1, wherein the divided moving image combining unit preferentially adopts a divided material moving image obtained from a recommended article having a high recommendation value.   5. The moving image editing server according to claim 1, wherein the synthesized moving image creating unit includes a telop inserting unit that inserts a telop into the synthesized moving image.   6. The moving image editing server according to claim 5, wherein the telop insertion unit inserts a telop created by summarizing character information described in the recommended article into the synthesized moving image. In the template, the allowable time range of each scene is defined,
7. The moving image editing server according to claim 1, wherein the divided moving image combining unit extracts a divided material moving image that matches the allowable time range.   The moving image dividing unit has a learning moving image dividing function for creating learning data of the first classifier in addition to a material moving image dividing function for dividing the material moving image into a plurality of divided material moving images. 8. The moving image editing server according to claim 1, wherein In a video editing program for a server that distributes video content to a user terminal accessed via the Internet,
The server,
A recommended article extractor that extracts recommended articles including material videos based on user preference profile information;
A template creation unit for creating a template in which a plurality of scenes with scene labels are defined; and
Functioning as a composite video creation unit that creates a composite video based on the material video and the template;
The synthetic video creation unit is a video dividing unit that divides a material video into a plurality of divided material videos
A first classifier for attaching a scene label to the divided material animation;
A divided moving image combining unit that extracts a divided material moving image that matches each scene of the template based on the scene label and creates a combined moving image by combining the moving images;
A program for a moving image editing server, comprising: