JP2024013024A - Moving image analyzer, moving image analysis system, edition device, learning device, moving image analysis method, and moving image analysis program - Google Patents

Abstract

To allow an improvement of reliability of an analysis of a scene and an object.SOLUTION: A moving image analyzer divides a moving image in each scene from the moving image and genre information indicating genre of the moving image, uses a first learning model for determining a scene name of a classification of the scene to each scene, divides the moving image into each scene, determines the scene name to each scene, sets a label as information related to an object imaged to a video image frame on the basis of the scene name to each video image frame contained in the scene, and outputs the scene name of each scene containing the video image frame and the label in each video image frame.SELECTED DRAWING: Figure 1

Description

The present invention relates to a video analysis device and the like.

When creating a new video, videos created in the past may be used. At this time, the creator of the video searches for a video that he/she wants to use for creating a new video from an archive that stores videos created in the past.

Videos may have metadata attached to them, including information such as the program name and program genre. In this case, the creator of the video searches for the video he or she wants to use based on this metadata. However, if there are many videos stored in the archive, even if you search for videos based on metadata, there is a possibility that many videos will be extracted. Then, the creator visually searches for the scene he or she wants to use from among these many videos. In addition, the creator manually cuts out the scene he or she wants to use, and uses the cut out video to create a new video. Note that a scene refers to a chronologically continuous scene.

As methods related to this, there are methods described in Patent Documents 1 to 2. In these methods, the scenes and objects captured in the video are analyzed.

JP2018-005638A JP2020-077343A

However, for example, in the case of videos used by broadcasting stations, the results of scene and subject analysis require high reliability. If the reliability of the analysis results is low, it will take time for the operator to check and make corrections, increasing the burden on the operator.

SUMMARY OF THE INVENTION In view of the above-mentioned problems, an object of the present invention is to provide a video analysis device and the like that can further improve the reliability of scene and subject analysis.

In one aspect of the present invention, the video analysis device divides the video into scenes based on genre information indicating the genre of the video, and determines, for each of the scenes, a scene name indicating the classification of the scene. a scene dividing section; a label setting section that sets, for each of the video frames included in the scene, a label that is information about a subject imaged in the video frame; an output unit that outputs the scene name and the label of the scene including the scene, and the scene dividing unit divides the video into scenes based on the video and the genre information of the video. , dividing the scenes and determining the scene names using a first learning model that determines the scene names for each of the scenes.

In another aspect of the present invention, the video analysis method divides the video into scenes based on the video and genre information indicating the genre of the video, and indicates a scene classification for each of the scenes. A first learning model that determines a scene name is used to divide the video into the scenes, determine the scene name for each of the scenes, and determine the scene name for each of the video frames included in the scene. , based on the scene name, set a label that is information about the subject imaged in the video frame, and for each video frame, set the scene name of the scene in which the video frame is included, and the label. Output.

In another aspect of the present invention, the video analysis program causes the computer to divide the video into scenes based on genre information indicating the genre of the video, and to classify each of the scenes into scenes. a scene division function that determines a scene name to be displayed; and a label setting that sets a label, which is information about the subject imaged in the video frame, based on the scene name, for each video frame included in the scene. and an output function that outputs, for each of the video frames, the scene name of the scene in which the video frame is included, and the label, and the scene division function is configured to output the video and the label of the video frame. The video is divided into scenes based on the genre information, and the scenes are divided and the scene names are determined using a first learning model that determines the scene names for each of the scenes.

According to the present invention, it is possible to further improve the reliability of scene and subject analysis.

1 is a diagram showing a configuration example of a video analysis device according to a first embodiment of the present invention. It is a figure showing an example of an operation flow of a video analysis device of a first embodiment of the present invention. It is a diagram showing an example of the configuration of a system including a video analysis device according to a second embodiment of the present invention. It is a figure showing an example of composition of a learning device of a second embodiment of the present invention. It is a figure showing an example of composition of a video analysis device of a second embodiment of the present invention. It is a figure showing an example of composition of an editing device of a second embodiment of the present invention. FIG. 7 is a diagram showing an example of a scene edited image according to the second embodiment of the present invention. It is a figure which shows the example of the label edit image of 2nd embodiment of this invention. It is a figure which shows the example of the edit menu image of 2nd embodiment of this invention. It is a figure showing an example of an operation flow of a learning device of a second embodiment of the present invention. It is a figure showing an example of an operation flow of a video analysis device of a second embodiment of the present invention. It is a figure showing an example of an operation flow of an editing device of a second embodiment of the present invention. 1 is a diagram showing an example of a hardware configuration of each embodiment of the present invention. FIG.

[First embodiment]
A first embodiment of the present invention will be described. A specific example of the video analysis device 10 in the first embodiment is a video analysis device 20 in the second embodiment described later.

FIG. 1 shows a configuration example of a video analysis device 10 of this embodiment. The video analysis device 10 of this embodiment includes a scene dividing section 11, a label setting section 12, and an output section 13.

The scene dividing unit 11 divides the video into scenes based on genre information. The genre information indicates the genre of the video. Further, the scene dividing unit 11 determines a scene name for each scene. The scene name indicates the classification of the scene.

The scene dividing unit 11 uses the first learning model to perform the above-described scene division and scene name determination. The first learning model divides the video into scenes based on the video and the genre information of the video, and determines a scene name for each scene.

The label setting unit 12 sets a label for each video frame included in a scene based on the scene name. The label is information regarding the subject imaged in the video frame.

The output unit 13 outputs, for each video frame, the scene name and label of the scene in which the video frame is included.

Next, FIG. 2 shows an example of the operation flow of the video analysis device 10 of this embodiment.

The scene dividing unit 11 divides the video into scenes based on genre information. Furthermore, the scene dividing unit 11 determines a scene name for each scene (step S101).

The label setting unit 12 sets a label for each video frame included in the scene (step S102).

The output unit 13 outputs, for each video frame, the scene name and label of the scene in which the video frame is included (step S103).

As described above, in the first embodiment of the present invention, the video analysis device 10 includes a scene dividing section 11, a label setting section 12, and an output section 13. The scene dividing unit 11 divides the video into scenes based on genre information. Further, the scene dividing unit 11 determines a scene name for each scene. The scene dividing unit 11 uses the first learning model to divide scenes and determine scene names. The first learning model divides the video into scenes based on the video and the genre information of the video, and determines a scene name for each scene. The label setting unit 12 sets a label for each video frame included in a scene based on the scene name. The output unit 13 outputs, for each video frame, the scene name and label of the scene in which the video frame is included.

In this manner, the video analysis device 10 divides scenes and determines scene names based on genre information. Thereby, the video analysis device 10 increases the possibility that a scene will be analyzed as a scene that frequently appears in the genre indicated by the genre information. As a result, the video analysis device 10 can reduce the possibility that a scene with similar video feature amounts will be analyzed as an incorrect scene. Therefore, the reliability of scene analysis can be improved. Furthermore, since the video analysis device 10 sets the label based on the scene name, the reliability of the scene analysis is improved, and the reliability of the subject analysis is also improved. Therefore, it is possible to further improve the reliability of scene and subject analysis.

[Second embodiment]
Next, a video analysis device 20 according to a second embodiment of the present invention will be described. A specific example of the video analysis device 10 in the first embodiment is the video analysis device 20 in the second embodiment.

First, FIG. 3 shows a configuration example of a video analysis system 80 including the video analysis device 20 of this embodiment. The video analysis system 80 includes a learning device 60, a video analysis device 20, and an editing device 30. The video analysis device 20 is connected to the video storage device 40, the learning device 60, and the editing device 30. Further, the learning device 60 is connected to the video storage device 40 and the video analysis device 20. Further, the editing device 30 is connected to the video storage device 40 and the user terminal 50.

The video storage device 40 stores videos. A video includes video information. Further, the video may include audio information. The video storage device 40 also stores metadata. Metadata is data related to videos.

The metadata includes genre information. Genre information is information indicating the genre of the video. The genre is the type of video. Genres include, for example, sports, news/reporting, and variety.

Additionally, the metadata can include a scene name for each video frame included in the video. The scene name is a name indicating the classification of a scene. Scene names are determined for video frames showing the same scene. For example, if the video is a soccer video, scene names include dribbling, goal, and counter.

The metadata can also include labels for each video frame included in the video. The label is information regarding the subject imaged in the video frame. The subject may be a person. One or more labels can be set for one video frame. Furthermore, there may be video frames for which no labels are set.

The label may be, for example, the name of the subject. In this case, the labels are, for example, "soccer player" or "soccer ball." The label may be a person's name. Further, the label may be information indicating the motion of the subject. In this case, the labels are, for example, "kicking the ball," "pitching," "batting," and the like. Furthermore, one or more labels may be set for one subject. For example, for a video frame to which the label "batting" has been set, labels such as "no-step batting method" or "one-leg batting method", which are more detailed information than "batting", may be further set.

Further, the metadata may include, for each label, area information of the subject corresponding to the label. The area information is information indicating in which area of the video frame the subject is imaged.

Note that the video storage device 40 can store videos to be analyzed by the video analysis device 20 and videos used for learning by the learning device 60. Further, the videos to be analyzed may include videos that have not yet been analyzed and videos that have been analyzed. Furthermore, some analyzed videos may have the analysis results included in the metadata edited, while others may not have been edited. Note that the analysis result refers to the scene name and label of each video frame. Further, learning, analysis, and editing will be described later. Furthermore, the analysis result refers to the scene name and label of each video frame. Video metadata that has not yet been analyzed does not include scene names or labels. Metadata of analyzed videos and videos used for training includes scene names and labels.

Further, the video storage device 40 transmits the video requested by the video analysis device 20 and the metadata of the video to the video analysis device 20. Further, the video storage device 40 transmits the video requested by the learning device 60 and the metadata of the video to the learning device 60. Further, the video storage device 40 transmits the video requested by the editing device 30 and the metadata of the video to the editing device 30.

The user terminal 50 is a terminal used by a user who edits metadata. The user terminal 50 is, for example, an information processing device such as a personal computer. The user terminal 50 includes a man-machine interface such as a keyboard, which is an example of an input device, and a display, which is an output device. The user terminal 50 issues instructions to the editing device 30 in response to operations input by the user to the input device. Further, the user terminal 50 causes the display means provided in the user terminal 50 to display an image under the control of the editing device 30.

The learning device 60 generates a learning model for video analysis. Details of the learning device 60 will be described later.

The video analysis device 20 analyzes videos. Details of the video analysis device 20 will be described later.

The editing device 30 edits metadata. Details of the editing device 30 will be described later.

Next, FIG. 4 shows a configuration example of the learning device 60 of this embodiment. The learning device 60 includes a learning information input section 61, a learning information storage section 62, and a model generation section 63.

In this embodiment, a case will be described in which the learning device 60 and the video analysis device 20 are different devices, but the video analysis device 20 may have the functions of the learning device 60.

The learning device 60 of this embodiment generates a first learning model for scene segmentation and a second learning model for label setting. Scene division and label setting will be described later. The process of generating the first learning model and the process of generating the second learning model are mutually different processes. The learning device 60 may be divided into a device that generates a first learning model and a device that generates a second learning model.

The learning information input unit 61 receives learning information and stores it in the learning information storage unit 62. The learning information storage unit 62 stores learning information. The model generation unit 63 generates and outputs a learning model using the learning information.

First, a case where the learning device 60 generates a first learning model will be described. The first learning model is a learning model for scene segmentation. The input to the first learning model is a video and genre information of the video. Furthermore, the output of the first learning model is a scene name for each video frame included in the video. The first learning model divides a video into scenes based on video genre information and determines a scene name for each scene.

In this case, the learning information for generating the first learning model includes a scene name, a video frame corresponding to the scene name, and genre information of a video including the video frame. The video storage device 40 stores learning videos and metadata of the videos. The learning video metadata includes genre information and scene names for each video frame. Therefore, the learning information input unit 61 can obtain the learning information by acquiring the learning video and metadata from the video storage device 40.

Note that among the analyzed videos, a video whose metadata has been edited by the editing device 30 may be used as a learning video. In this way, by using the editing results by the editing device 30 for learning, the accuracy of the learning model can be improved.

Further, the genre information is set in advance for the video by another device (not shown), and metadata including the set genre information is stored in the video storage device 40. Furthermore, if the metadata does not include genre information, the video analysis device 20 or the editing device 30 may set the genre information in the metadata in response to an operation input from the user terminal 50. Furthermore, as will be described later, the editing device 30 can change genre information when editing metadata.

The model generation unit 63 generates a first learning model based on the learning information. Furthermore, the model generation unit 63 transmits the generated first learning model to the scene division unit 21 of the video analysis device 20.

Note that the learning information input unit 61 can acquire a learning video from the video storage device 40 at a predetermined timing. The predetermined timing at this time is, for example, when a new learning video is stored in the video storage device 40 or when metadata is edited by the editing device 30. Furthermore, the model generation unit 63 may generate a new first learning model at a predetermined timing. The predetermined timing at this time is, for example, at regular intervals, when a new learning video is stored in the learning information storage unit 62, or when a learning instruction for instructing learning is input from the user terminal 50. etc.

Next, a case where the learning device 60 generates a second learning model will be described. The second learning model is a learning model for label setting. The second learning model is based on the same scene frame, which is a video frame with the same scene name, the scene name of the same scene frame, and the genre information of the video that includes the same scene frame. Set a label name for each video frame.

In this case, the learning information for generating the second learning model includes a label, a video frame to which the label is set, area information of the label, a scene name of the video frame to which the label is set, and Contains genre information of the video that includes the video frame to which the label is set. The video storage device 40 stores learning videos and metadata of the videos. The learning video metadata includes genre information, scene names and labels for each video frame, and area information for each label. Therefore, the learning information input unit 61 can obtain the learning information by acquiring the learning video and the metadata of the video from the video storage device 40. Note that among the analyzed videos, a video whose metadata has been edited by the editing device 30 may be used as a learning video.

The model generation unit 63 generates a second learning model based on the learning information. Furthermore, the model generation unit 63 transmits the generated second learning model to the label setting unit 22 of the video analysis device 20.

Note that the learning information input unit 61 inputs the learning video at a predetermined timing such as when a new learning video is stored in the video storage device 40 or when metadata is edited by the editing device 30. It can be acquired from the video storage device 40. Furthermore, the model generation unit 63 may generate a new second learning model at a predetermined timing. The predetermined timing is, for example, at a predetermined time interval, when a new learning video is stored in the learning information storage unit 62, or when a learning instruction for instructing learning is input from the user terminal 50. .

Next, FIG. 5 shows a configuration example of the video analysis device 20 of this embodiment. The video analysis device 20 includes a scene dividing section 21, a label setting section 22, and an output section 23. Note that a series of operations performed on a video by the video analysis device 20 is referred to as analysis.

The scene dividing unit 21 acquires a moving image to be analyzed and metadata of the moving image from the moving image storage device 40. Furthermore, the scene dividing unit 21 divides the moving image to be analyzed into scenes based on the genre information. Further, the scene dividing unit 21 determines a scene name for each scene. The scene division unit 21 uses the first learning model generated by the learning device 60 to perform scene division and scene name determination. The scene dividing unit 21 inputs a video to be analyzed and genre information of the video to the first learning model. The first learning model outputs a scene name for each video frame included in the video.

The label setting unit 22 sets a label for each video frame included in the moving image to be analyzed. The label setting unit 22 uses the second learning model generated by the learning device 60 to set a label. Furthermore, the label setting unit 22 uses the second learning model to determine area information for each label. The label setting unit 22 inputs the same scene frame, the scene name of the same scene frame, and the genre information of the video including the same scene frame to the second learning model. The same scene frames are video frames with the same scene name. The second learning model outputs a label for each video frame included in the same scene frame and label area information.

The output unit 23 outputs the scene name determined by the scene dividing unit 21 and the label set by the label setting unit 22. More specifically, the output unit 23 causes the video storage device 40 to store the metadata including the scene name and label for each video frame and the area information for each label.

Next, FIG. 6 shows a configuration example of the editing device 30 of this embodiment. The editing device 30 of this embodiment includes an editing section 34 and a storage section 35. The editing device 30 may be included in the video analysis device 20.

The editing unit 34 acquires a video to be edited and metadata of the video from the video storage device 40 in response to a video acquisition instruction from the user terminal 50. Furthermore, the editing unit 34 causes the storage unit 35 to store the acquired moving image and metadata. The video acquisition instruction is an instruction to acquire a video to be edited. The video acquisition instruction includes identification information of the video to be edited, such as a file name.

The editing unit 34 edits the metadata of the video to be edited in accordance with the scene editing instruction. The scene editing instruction is an instruction regarding editing the scene name for each video frame. Furthermore, the editing unit 34 edits the metadata of the video to be edited in accordance with the label editing instruction. The label editing instruction is an instruction regarding editing the label for each video frame. Scene editing instructions and label editing instructions are input from the user terminal 50 to the editing section 34. The editing unit 34 also stores the edited metadata in the video storage device 40.

Furthermore, the editing unit 34 displays the scene edited image 110 on the user terminal 50 in response to the scene edited image display instruction. The scene edited image display instruction is an instruction to display the scene edited image 110. The scene editing image 110 is an image for editing a scene name. Furthermore, the editing unit 34 generates a scene edited image 110 to be displayed on the user terminal 50 based on the acquired video and metadata.

FIG. 7 shows an example of the scene edited image 110.

The scene edited image 110 includes an original moving image display image 111. The original video display image 111 displays the video to be edited. The video displayed in the original video display image 111 can be played back or stopped by operation input from the user terminal 50. The editing unit 34 displays the video to be edited on the original video display image 111. Further, the editing unit 34 starts or stops playing the video in response to an operation input from the user terminal 50.

The scene edited image 110 also includes a scene display image 112. The scene display image 112 shows how the video to be edited is divided into scenes. In the scene display image 112, the horizontal axis indicates the playback time from the starting point of the video. In the scene display image 112, video frames to which the same scene name is given may be shown in the same color. The metadata includes scene names for each video frame. The editing unit 34 can determine that video frames to which the same scene name is given are the same scene, based on the metadata of the video to be edited. The editing unit 34 causes the scene display image 112 to display boundaries between video frames to which mutually different scene names are given.

Furthermore, the scene edited image 110 includes a cutout image 113. The cutout image 113 is an image (still image) of the video frame indicated by the cursor displayed on the scene display image 112. The cursor displayed on the scene display image 112 is moved according to an operation input from the user terminal 50. The editing unit 34 displays the image of the video frame indicated by the cursor displayed on the scene display image 112 as a cutout image 113 in response to an operation input from the user terminal 50. Note that the time code displayed above the cutout image 113 represents the start position and end position of the scene that includes the video frame indicated by the cursor.

Furthermore, the scene edited image 110 includes a genre image 114. The genre image 114 includes genre information of the video to be edited. The editing unit 34 displays genre information included in the metadata of the video to be edited on the genre image 114.

The scene edited image 110 also includes a scene name edited image 115. The scene name editing image 115 is an image for editing a scene name. The scene name editing image 115 displays the scene name of the scene indicated by the cursor displayed on the scene display image 112. Furthermore, in the scene name editing image 115, when "edit" is selected by operation input from the user terminal 50, the scene name can be edited. When a character string is input into the scene name editing image 115 through an operation input from the user terminal 50, a scene editing instruction is input into the editing section 34. The editing unit 34 changes the scene name of the video frame included in the scene by editing the metadata stored in the storage unit 35 according to the input scene editing instruction, and edits the edited metadata. is stored in the user terminal 50.

The scene editing image 110 also includes a scene list image 116. The scene list image 116 displays a list of scenes. The scene list image 116 includes scene names of scenes included in the video to be edited. Further, in the scene list image 116, a thumbnail of the corresponding scene is displayed for each scene name. Further, the scene list image 116 may include a time code for each scene. The time code indicates where each scene is included in the video, depending on the playback time from the start point of the video.

The scene list image 116 may include a scene name input area 117 and a scene search image 118. The scene name input area 117 is an area for inputting a scene name. Further, the scene search image 118 is an image for searching for a scene with the scene name input in the scene name input area 117. When the scene search image 118 is selected, the scene with the scene name input in the scene name input area 117 is displayed on the scene list image 116.

Furthermore, the thumbnails displayed in the scene list image 116 can be moved to a display area for thumbnails with different scene names by operation input from the user terminal 50. Moving the thumbnail corresponds to moving the scene. When a scene editing instruction indicating movement of a scene is input, the editing unit 34 changes the scene name of the video frame corresponding to the moved scene by editing the metadata stored in the storage unit 35. . The editing unit 34 then stores the edited metadata in the video storage device 40.

Further, the scene edited image 110 may include a time width edited image 119. The time width editing image 119 is an image for making it possible to edit the start position and end position of a scene. When the time width editing image 119 is selected, the start position and end position of the scene in the scene display image 112 can be edited. When a scene editing instruction indicating editing of the start position or end position of a scene is input, the editing unit 34 edits the metadata stored in the storage unit 35 to edit the video frame whose scene name has been changed. Change the scene name. The editing unit 34 then stores the edited metadata in the video storage device 40.

Further, the editing unit 34 displays the label edited image 120 on the user terminal 50 in response to the label edited image display instruction. The label edited image display instruction is an instruction to display the label edited image 120. The label editing image 120 is an image for editing a label. Furthermore, the editing unit 34 generates a label edited image 120 to be displayed on the user terminal 50 based on the acquired video and metadata.

FIG. 8 shows an example of the label edited image 120.

The label edited image 120 includes an original moving image display image 121. The original moving image display image 121 is the same as the original moving image display image 111, so a description thereof will be omitted.

Furthermore, the label editing image 120 includes a scene display image 122. The scene display image 122 is similar to the scene display image 112, so the description thereof will be omitted.

The label edited image 120 also includes a cutout image 123. The cutout image 123 is similar to the cutout image 113, so the description thereof will be omitted.

The label edited image 120 also includes a genre image 124. The genre image 124 shows genre information of the video to be edited. The editing unit 34 displays genre information included in the metadata of the video to be edited on the genre image 124.

Furthermore, in the genre image 124, the displayed genre information is changed in response to an operation input from the user terminal 50. When an operation input to select "Register" for the genre image 124 is made, the editing section 34 edits the metadata stored in the storage section 35 to update the genre information displayed on the genre image 124. Change to genre information.

The first learning model and the second learning model are learning models based on genre information. Therefore, if the genre information is changed and reanalysis is performed, the scene name or label of the video frame may be changed. Therefore, when the genre information is changed, the editing device 30 may instruct the video analysis device 20 to re-analyze the video to be edited. Then, the editing unit 34 may acquire the metadata after the reanalysis from the video storage device 40 and display the scene edited image 110 and the label edited image 120 based on the new metadata on the user terminal 50.

Furthermore, the label editing image 120 includes a label display image 125. The label display image 125 is an image for editing a label. The label display image 125 displays the label of the video frame selected by the cursor displayed on the scene display image 122. The editing unit 34 refers to the metadata of the video to be edited and displays the label set for the video frame in the label display image 125. Further, the label displayed on the label display image 125 can be deleted by operation input from the user terminal 50.

Further, the label display image 125 includes similar labels. Similar labels are candidates for labels to be added.

The similar label may be, for example, a word similar to the set label. For example, the editing unit 34 refers to a similar word dictionary and displays words similar to the label set in the video frame on the label display image 125 as similar labels. In this case, the storage unit 35 stores a similar word dictionary.

Further, the similar label may be, for example, a label set to a similar video. For example, the editing unit 34 searches for a video stored in the video storage device 40 for a video that has the same genre information as the video to be edited, and acquires metadata of the searched video. Then, the editing unit 34 displays the label included in the metadata as a similar label on the label display image 125. Furthermore, for example, the editing unit 34 may search for videos with similar video characteristics from among the videos stored in the video storage device 40. In this case, the output unit 23 of the video analysis device 20 calculates the feature amount of the video during analysis and stores it in the video storage device 40. Then, the editing unit 34 searches the video storage device 40 for a video having a feature amount similar to the feature amount of the video to be edited.

When a label editing instruction to add a similar label is input, the editing unit 34 adds the selected similar label as a label of the video frame by editing the metadata. More specifically, when the editing unit 34 inputs an operation to select "Add" in the label addition image 127, the editing unit 34 adds similar labels whose check boxes next to the similar labels are checked to the video frame. Add as a configured label.

Additionally, the label edited image 120 includes an unlabeled scene image 126. The unlabeled scene image 126 displays thumbnails of scenes for which no labels have been set. A scene with no label set is a scene where there is no video frame with a label set. The editing unit 34 causes the unlabeled scene image 126 to display thumbnails of scenes for which no labels have been set, based on the metadata of the video to be edited. Further, when an operation input to select "Add" of the label added image 127 is performed and a label is set for a scene that was displayed as a scene without a label, the editing unit 34 selects the corresponding scene from the unlabeled scene image 126. Delete a scene. Note that the editing unit 34 may delete the labeled scene from the unlabeled scene image 126 at the timing when an operation input to select the storage image 129 or the registration image 130 is made.

The label edited image 120 also includes a label added image 127. The label addition image 127 is an image for adding a label. The editing unit 34 displays the label addition image 127 when an operation input to select "+" displayed on the label display image 125 is made. Further, the editing unit 34 displays the label added image 127 when an operation input for selecting a thumbnail displayed in the unlabeled scene image 126 is performed. In this case, the editing unit 34 moves the cursor displayed on the scene display image 122 to the right or left to the position of the scene corresponding to the selected thumbnail.

When a character string is input to the label addition image 127 through an operation input and an operation input to select "add" is performed, the editing unit 34 uses the input character string as a label set to the video frame. It is displayed on the display image 125. At this time, if there is a similar label whose checkbox is checked, that similar label is also added to the label display image 125 as a label set in the video frame.

The label edit image 120 also includes a label box image 128. Label box image 128 displays a box indicating area information about the label. The box indicates area information, that is, in which area of the video frame the subject corresponding to the label is captured. The label box image 128 displays a box with a label set for the video frame indicated by the cursor displayed on the scene display image 122. At this time, the label box image 128 is displayed as a box superimposed on the image of the video frame indicated by the cursor displayed on the scene display image 122. The label box image 128 also displays a label corresponding to the box. The editing unit 34 displays each label box in the label box image 128 based on the area information included in the metadata. Further, the box is moved and transformed by operation input from the user terminal 50.

Furthermore, when a label set for a video frame is added to the label display image 125, the editing unit 34 displays a box for the added label in the label box image 128. Note that the initial position and initial size of the box displayed at this time are arbitrary. Furthermore, when the label set for the video frame is deleted from the label display image 125, the editing unit 34 deletes the box of the deleted label from the label box image 128.

The label edited image 120 also includes a storage image 129. The storage image 129 is an image for temporarily storing a label. When the editing unit 34 receives an operation input to select the storage image 129, the editing unit 34 edits the metadata stored in the storage unit 35 to edit the video frame indicated by the cursor displayed on the scene display image 122. The label is changed to the label displayed in the label display image 125. At this time, the editing unit 34 also edits the metadata stored in the storage unit 35 to change the area information of the label of the video frame indicated by the cursor displayed in the scene display image 122 to the label box image. The area information of the label box displayed in 128 is changed. Further, at this time, the editing unit 34 may change the label in the same way for a video frame that has the same scene name as the video frame and also has the same label before the change.

Furthermore, the label editing image 120 includes a registration image 130. The registration image 130 is an image for registering metadata. When an operation input to select the registration image 130 is made, the editing unit 34 first edits the metadata stored in the storage unit 35 in the same way as when an operation input to select the storage image 129 is made. do. The editing unit 34 then stores the metadata stored in the storage unit 35 in the video storage device 40.

Note that the scene editing image 110 and the label editing image 120 are displayed on the user terminal 50 by, for example, an operation input to the editing menu image 140. FIG. 9 shows an example of the edit menu image 140.

Edit menu image 140 includes video information display image 141. The video information display image 141 displays identification information of the video to be edited, for example, the file name.

Further, the editing menu image 140 includes a moving image selection image 142. The moving image selection image 142 is an image for selecting a moving image to be edited. When an operation input to select the moving image selection image 142 is input, the editing unit 34 displays a list of moving images and the like. Further, when a moving image to be edited is selected, the editing unit 34 causes the identification information of the selected moving image to be displayed on the video information display image 141. The editing unit 34 also acquires the selected video and the metadata of the video, and stores the acquired video and metadata in the storage unit 35.

The edit menu image 140 also includes a scene edit image display image 143. The scene edited image display image 143 is an image for displaying the scene edited image 110. When the scene edited image display image 143 is selected, a scene edited image display instruction is input to the editing section 34. When the scene edited image display instruction is input, the editing unit 34 displays the scene edited image 110 on the user terminal 50.

The edit menu image 140 also includes a label edit image display image 144. The label edited image display image 144 is an image for displaying the label edited image 120. When the label edited image display image 144 is selected, a label edited image display instruction is input to the editing unit 34. When the label edited image display instruction is input, the editing unit 34 displays the label edited image 120 on the user terminal 50.

In this way, the editing device 30 of this embodiment can edit metadata and change scene names and labels. Further, the learning device 60 can perform re-learning using the edited metadata. In this way, by performing relearning using learning information based on the edited metadata, it is possible to achieve scene segmentation with the precision the user expects and labeling with the granularity the user expects. become.

For example, assume that "batting" is displayed in the label display image 125 as a label set to a video frame. At this time, the user can check the cutout image 123 and the like and add more detailed labels (for example, "no-step batting method" and "one-leg batting method"). Further, the learning device 60 can perform re-learning using learning information including added labels. When retraining is performed using learning information that includes added labels and a new second learning model is generated, the added labels will be automatically set by the second learning model. Become. In this way, by performing relearning using learning information including edited labels, it becomes possible to realize labeling with the granularity expected by the user.

Next, FIG. 10 shows an example of the operation flow of the learning device 60 of this embodiment. The learning device 60 performs the operations shown in FIG. 10 for each of generation of the first learning model and generation of the second learning model. Further, the learning device 60 performs the operation shown in FIG. 10 at predetermined intervals or when learning is instructed from the user terminal 50.

The learning information input unit 61 receives the learning information and stores it in the learning information storage unit 62 (step S201). The model generation unit 63 generates and outputs a learning model using the learning information (step S202).

Next, FIG. 11 shows an example of the operation flow of the video analysis device 20 of this embodiment. The video analysis device 20 performs the operation shown in FIG. 11 at predetermined intervals or when an analysis is instructed from the user terminal 50.

Further, the video analysis device 20 may perform the operation shown in FIG. 11 when a new video to be analyzed is added to the video storage device 40. For example, the video analysis device 20 can detect that a new video has been added to the video storage device 40 by acquiring a video list from the video storage device 40 at predetermined time intervals and comparing the old and new video lists. Can be done. Alternatively, a notification indicating that the new video has been added may be transmitted from the device that added the new video to the video storage device 40 to the video analysis device 20.

The scene dividing unit 21 acquires a video to be analyzed and metadata of the video from the video storage device 40 (step S301). Furthermore, the scene dividing unit 21 divides the moving image to be analyzed into scenes based on the genre information. Furthermore, the scene dividing unit 21 determines a scene name for each video frame included in the video (step S302). The scene division unit 21 uses the first learning model generated by the learning device 60 to perform scene division and scene name determination. The scene dividing unit 21 inputs a video to be analyzed and genre information of the video to the first learning model. The first learning model outputs a scene name for each video frame included in the video.

The label setting unit 22 sets a label for each video frame included in the moving image to be analyzed (step S303). The label setting unit 22 uses the second learning model generated by the learning device 60 to set a label. Furthermore, the label setting unit 22 uses the second learning model to determine area information for each label. The label setting unit 22 inputs the same scene frame, the scene name of the same scene frame, and the genre information of the video including the same scene frame to the second learning model. The second learning model outputs a label for each video frame included in the same scene frame and label area information.

The output unit 23 outputs the scene name determined by the scene dividing unit 21 and the label set by the label setting unit 22. More specifically, the output unit 23 includes the scene name and label for each video frame, and the area information for each label in the metadata, and stores the metadata in the video storage device 40 (step S304). .

Next, FIG. 12 shows an example of the operation flow of the editing device 30 of this embodiment.

The editing unit 34 acquires a video to be edited and metadata of the video from the video storage device 40 in response to a video acquisition instruction from the user terminal 50 (step S401). Furthermore, the editing unit 34 causes the storage unit 35 to store the acquired moving image and metadata.

The editing unit 34 displays the scene edited image 110 on the user terminal 50 in response to the scene edited image display instruction. Furthermore, the editing unit 34 displays the label edited image 120 on the user terminal 50 in response to the label edited image display instruction (step S402).

The editing unit 34 then updates the scene edited image 110 in accordance with the scene editing instruction. The editing unit 34 also edits the metadata stored in the storage unit 35 in response to a scene editing instruction. The editing unit 34 also updates the label editing image 120 in response to the label editing instruction. Further, the editing section 34 edits the metadata stored in the storage section 35 in accordance with the label editing instruction. Further, the editing unit 34 stores the metadata stored in the storage unit 35 in the video storage device 40 in accordance with the instruction (step S403).

As described above, in the second embodiment of the present invention, the video analysis device 20 includes a scene dividing section 21, a label setting section 22, and an output section 23. The scene dividing unit 21 divides the video into scenes based on genre information. Further, the scene dividing unit 21 determines a scene name for each scene. The scene dividing unit 21 uses the first learning model to divide scenes and determine scene names. The first learning model divides the video into scenes based on the video and the genre information of the video, and determines a scene name for each scene. The label setting unit 22 sets a label for each video frame included in a scene based on the scene name. The output unit 23 outputs, for each video frame, the scene name and label of the scene in which the video frame is included.

In this way, the video analysis device 20 divides scenes and determines scene names based on genre information. Thereby, the video analysis device 20 increases the possibility that the scene will be analyzed as a scene that frequently appears in the genre indicated by the genre information. As a result, the video analysis device 20 can reduce the possibility that a scene with similar video feature amounts will be analyzed as an incorrect scene. Therefore, the reliability of scene analysis can be improved. Further, since the video analysis device 20 sets the label based on the scene name, the reliability of the scene analysis is improved, and the reliability of the subject analysis is also improved. Therefore, it is possible to further improve the reliability of scene and subject analysis.

The label setting unit 22 uses the second learning model to set a label. The second learning model sets a label for each video frame included in the same scene frame based on the same scene frame, the scene name of the same scene frame, and genre information of the video that includes the same scene frame. . The same scene frames are video frames with the same scene name. As a result, the genre information is also used for label setting, making it possible to further improve the reliability of subject analysis. Furthermore, by using the second learning model for label setting, variations in the set labels can be reduced compared to when labels are set manually.

Furthermore, the second learning model further outputs region information for each label. The region information is information indicating in which region of the video frame the subject corresponding to the label is imaged. Furthermore, the label setting unit 22 uses the second learning model to determine area information for each label. The output unit 23 further outputs area information. This makes it possible to visualize the area information, improving convenience for the user.

Furthermore, the output unit 23 includes the scene name in the metadata and stores it in the video storage device. Metadata is information about a video. The video storage device stores videos and metadata. This allows the user to use the metadata at the timing the user needs.

The editing device 30 also includes an editing section 34. The editing unit 34 acquires a video to be edited and metadata regarding the video from the video storage device 40. Furthermore, in response to the scene edited image display instruction, the editing unit 34 causes the user terminal 50 to display the scene edited image based on the acquired video and metadata. The scene edited image display instruction is an instruction to display a scene edited image. The scene editing image is an image for editing a scene name. The editing unit 34 also edits metadata in response to scene editing instructions. The scene editing instruction is an instruction regarding editing a scene name. Further, the editing unit 34 edits the scene name included in the metadata in accordance with the scene editing instruction, and stores the edited metadata in the video storage device 40. This allows the user to edit the scene name.

Furthermore, the scene edited image includes a scene name and a thumbnail for each scene included in the video to be edited. This allows the user to easily check the result of scene division.

Further, when a scene editing instruction indicating movement of a scene is input by moving a thumbnail to a display area of thumbnails with a different scene name, the editing unit 34 edits the metadata to edit the moved scene. Change the scene name of the video frame corresponding to . This makes it possible to easily change the scene name.

The editing unit 34 also acquires a video to be edited and metadata regarding the video from the video storage device 40. In addition, the editing unit 34 causes the user terminal 50 to display the label edited image based on the acquired video and metadata in response to the label edited image display instruction. The label edited image display instruction is an instruction to display a label edited image. The label editing image is an image for editing a label. Further, the editing unit 34 edits the metadata according to the label editing instruction. The label editing instruction is an instruction regarding label editing. Further, the editing unit 34 edits the label included in the metadata in accordance with the label editing instruction, and stores the edited metadata in the video storage device 40. This allows the label to be edited by the user.

The label edited image also includes similar labels that are candidates for labels to be added. When a label editing instruction to add a similar label is input, the editing unit 34 adds the selected similar label as a label of the video frame by editing the metadata. This makes it easy to add labels.

Further, the similar label is a word similar to the label set for the video frame. This makes it easy to add similar words to the label.

Further, the similar label is a label set to a similar video. A similar video is a video that has the same genre information as the video to be edited, or a video that has similar feature amounts. The editing unit 34 searches for similar videos from the videos stored in the video storage device 40, and sets a label included in the metadata of the searched similar video as a similar label. This makes it easy to add labels set to similar videos.

Furthermore, the label edited image includes thumbnails of scenes to which no labels have been set. This allows the user to easily know the existence of scenes for which no labels have been set.

Furthermore, the label setting unit 22 determines area information for each label. The region information is information indicating in which region of the video frame the subject corresponding to the label is imaged. The output unit 23 further includes the region information in the metadata and stores it in the video storage device 40. The label edit image includes a box that indicates area information about the label. This allows the user to easily understand the area corresponding to the label.

Further, the learning device 60 uses the metadata edited by the editing device 30 as learning information to generate a first learning model. Thereby, the reliability of the first learning model can be further improved.

The label setting unit 22 uses the second learning model to set a label. The second learning model sets a label for each video frame included in the same scene frame from the same scene frame, the scene name of the same scene frame, and the genre information of the video that includes the same scene frame. do. The same scene frames are video frames with the same scene name. The learning device 60 uses the metadata edited by the editing device 30 as learning information to generate a second learning model. As a result, the results of editing by the user are used for learning, making it possible to bring the automatically assigned labels closer to the granularity desired by the user.

[Hardware configuration example]
The video analysis device (10, 20), editing device 30, or learning device 60 (hereinafter referred to as video analysis device, etc.) in each embodiment of the present invention described above is realized using one information processing device (computer). An example of the configuration of hardware resources will be explained. Note that the video analysis device and the like may be physically or functionally realized using at least two information processing devices. Further, the video analysis device and the like may be realized as a dedicated device. Further, only some functions of the video analysis device or the like may be realized using an information processing device.

FIG. 13 is a diagram schematically showing an example of the hardware configuration of an information processing device that can implement the video analysis device and the like of each embodiment of the present invention. Information processing device 90 includes a communication interface 91, an input/output interface 92, an arithmetic device 93, a storage device 94, a nonvolatile storage device 95, and a drive device 96.

For example, the scene dividing section 11 and label setting section 12 in FIG. 1 can be realized by the arithmetic device 93. Further, the output unit 13 can be realized by the communication interface 91 and the arithmetic device 93.

The communication interface 91 is a communication means through which the video analysis device and the like of each embodiment communicates with an external device by wire and/or wirelessly. Note that when a video analysis device or the like is implemented using at least two information processing devices, these devices may be connected to each other via the communication interface 91 so that they can communicate with each other.

The input/output interface 92 is a man-machine interface such as a keyboard as an example of an input device and a display as an output device.

The arithmetic unit 93 is realized by an arithmetic processing unit such as a general-purpose CPU (Central Processing Unit) or a microprocessor, and a plurality of electric circuits. The arithmetic device 93 can, for example, read various programs stored in the nonvolatile storage device 95 into the storage device 94 and execute processing according to the read programs.

The storage device 94 is a memory device such as a RAM (Random Access Memory) that can be referenced by the arithmetic device 93, and stores programs, various data, and the like. Storage device 94 may be a volatile memory device.

The nonvolatile storage device 95 is a nonvolatile storage device such as a ROM (Read Only Memory) or a flash memory, and is capable of storing various programs, data, and the like.

The drive device 96 is, for example, a device that reads and writes data to and from a recording medium 97, which will be described later.

The recording medium 97 is any recording medium capable of recording data, such as an optical disk, a magneto-optical disk, or a semiconductor flash memory.

In each embodiment of the present invention, for example, a video analysis device or the like is configured by the information processing device 90 illustrated in FIG. This may be realized by supplying.

In this case, the embodiment can be realized by the arithmetic device 93 executing a program supplied to the video analysis device or the like. Further, it is also possible to configure not all but some functions of the video analysis device and the like with the information processing device 90.

Furthermore, the above program can also be recorded on the recording medium 97. The program may be configured to be stored in the non-volatile storage device 95 as appropriate during the shipping stage of the video analysis device or the like, or during the operational stage. In this case, the above-mentioned program may be supplied by installing it into the video analysis device or the like using an appropriate jig at the manufacturing stage or operation stage before shipping. Further, as the method for supplying the program, a general procedure such as a method of downloading the program from an external source via a communication line such as the Internet may be adopted.

Part or all of the above embodiments may be described as in the following additional notes, but are not limited to the following.

(Additional note 1)
a scene dividing unit that divides the video into scenes based on genre information indicating a genre of the video, and determines a scene name indicating a classification of the scene for each of the scenes;
a label setting unit that sets, for each of the video frames included in the scene, a label that is information about a subject imaged in the video frame based on the scene name;
an output unit that outputs, for each of the video frames, the scene name of the scene in which the video frame is included and the label;
The scene dividing unit classifies the video into each scene based on the video and the genre information of the video, and uses a first learning model that determines the scene name for each of the scenes. dividing the scene and determining the scene name;
Video analysis device.

(Additional note 2)
The label setting unit selects the same scene frame which is the video frame having the same scene name, the scene name of the same scene frame, and the genre information of the video including the same scene frame. Setting the labels using a second learning model that sets the labels for each of the video frames included in a scene frame;
The video analysis device described in Appendix 1.

(Additional note 3)
The second learning model further outputs, for each of the labels, area information that is information indicating in which area of the video frame the subject corresponding to the label is imaged;
The label setting unit determines the area information for each of the labels using the second learning model,
The output unit further outputs the area information.
The video analysis device described in Appendix 2.

(Additional note 4)
The output unit includes the scene name in metadata that is information regarding the video, and stores the scene name in a video storage device that stores the video and the metadata.
The video analysis device described in Appendix 1.

(Appendix 5)
The video analysis device described in Appendix 1,
A video analysis system comprising: a learning device that generates the first learning model.

(Appendix 6)
The video analysis device described in Appendix 2 or 3,
A video analysis system comprising: a learning device that generates the first learning model and the second learning model.

(Appendix 7)
Comprising the video analysis device and editing device described in Appendix 4,
The editing device includes:
an editing unit that edits the metadata in accordance with a scene editing instruction that is an instruction regarding editing the scene name;
The editorial department is
Obtaining a video to be edited and metadata regarding the video from the video storage device,
In response to a scene editing image display instruction that instructs displaying a scene editing image that is an image for editing the scene name, the scene editing image is displayed on the user terminal based on the acquired video and the metadata. display,
Editing the scene name included in the metadata in response to the scene editing instruction, and storing the edited metadata in the video storage device.
Video analysis system.

(Appendix 8)
The scene editing image includes the scene name and thumbnail for each of the scenes included in the video to be edited.
The video analysis system described in Appendix 7.

(Appendix 9)
The editing unit edits the metadata when the scene editing instruction indicating movement of the scene is input by moving the thumbnail to a display area of the thumbnail with a different scene name. changing the scene name of the video frame corresponding to the moved scene;
The video analysis system described in Appendix 8.

(Appendix 10)
Comprising the video analysis device and editing device described in Appendix 4,
The editing device includes:
The video analysis device includes an editing unit that edits the metadata in accordance with a label editing instruction that is an instruction regarding editing the label,
The editorial department is
acquiring a video to be edited and the metadata regarding the video from the video storage device;
Displaying the label editing image on the user terminal based on the acquired video and the metadata in response to a label editing image display instruction that instructs displaying a label editing image that is an image for editing the label. let me,
editing the label included in the metadata in response to the label editing instruction, and storing the edited metadata in the video storage device;
Video analysis system.

(Appendix 11)
The label editing image includes similar labels that are candidates for labels to be added,
When the label editing instruction to add the similar label is input, the editing unit adds the selected similar label as the label of the video frame by editing the metadata.
The video analysis system described in Appendix 10.

(Appendix 12)
The similar label is a word similar to the label set on the video frame.
The video analysis system described in Appendix 11.

(Appendix 13)
The similar label is the label set to a similar video,
The similar video is a video that has the same genre information as the video to be edited, or a video that has similar feature amounts,
The editing unit searches for the similar video from among the videos stored in the video storage device, and sets a label included in the metadata of the searched similar video as the similar label.
The video analysis system described in Appendix 11.

(Appendix 14)
The label edited image includes a thumbnail of the scene to which the label is not set.
The video analysis system described in Appendix 10.

(Appendix 15)
The label setting unit determines, for each of the labels, area information that is information indicating in which area of the video frame the subject corresponding to the label is imaged;
The output unit further includes the area information in the metadata and stores it in the video storage device,
the label editing image includes a box indicating the area information about the label;
The video analysis system described in Appendix 10.

(Appendix 16)
further comprising a learning device that generates the first learning model;
The learning device generates the first learning model by using the metadata edited by the editing device as learning information.
The video analysis system according to any one of appendices 10 to 15.

(Appendix 17)
The label setting unit selects the same scene frame which is the video frame having the same scene name, the scene name of the same scene frame, and the genre information of the video including the same scene frame. Setting the labels using a second learning model that sets the labels for each of the video frames included in a scene frame;
Further, a learning device that generates the second learning model by using the metadata edited by the editing device as learning information,
The video analysis system according to any one of appendices 10 to 15.

(Appendix 18)
An editing device in the video analysis system according to any one of appendices 7 to 15.

(Appendix 19)
A learning device in the video analysis system according to appendix 5.

(Additional note 20)
A learning device in the video analysis system described in Supplementary Note 6.

(Additional note 21)
A learning device in the video analysis system according to appendix 16.

(Additional note 22)
A learning device in the video analysis system according to appendix 17.

(Additional note 23)
Using a first learning model that divides the video into scenes based on the video and genre information indicating the genre of the video, and determines a scene name indicating the classification of the scene for each of the scenes, dividing the video into the scenes and determining the scene name for each of the scenes;
setting a label, which is information about a subject imaged in the video frame, for each video frame included in the scene, based on the scene name;
outputting, for each of the video frames, the scene name and the label of the scene in which the video frame is included;
Video analysis method.

(Additional note 24)
to the computer,
a scene dividing function that divides the video into scenes based on genre information indicating a genre of the video, and determines a scene name indicating a classification of the scene for each of the scenes;
a label setting function for setting a label, which is information about a subject imaged in the video frame, for each video frame included in the scene, based on the scene name;
realizing, for each of the video frames, an output function that outputs the scene name and the label of the scene in which the video frame is included;
The scene division function divides the video into scenes based on the video and the genre information of the video, and uses a first learning model that determines the scene name for each of the scenes. dividing the scene and determining the scene name;
Video analysis program.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The configuration and details of the present invention can be modified in various ways that can be understood by those skilled in the art within the scope of the present invention.

10, 20 Video analysis device 11, 21 Scene dividing section 12, 22 Label setting section 13, 23 Output section 30 Editing device 34 Editing section 35 Storage section 40 Video storage device 50 User terminal 60 Learning device 61 Learning information input section 62 Learning Information storage section 63 Model generation section 80 Video analysis system 90 Information processing device 91 Communication interface 92 Input/output interface 93 Arithmetic device 94 Storage device 95 Nonvolatile storage device 96 Drive device 97 Recording medium

Claims (10)

a scene dividing unit that divides the video into scenes based on genre information indicating a genre of the video, and determines a scene name indicating a classification of the scene for each of the scenes;
a label setting unit that sets, for each of the video frames included in the scene, a label that is information about a subject imaged in the video frame based on the scene name;
an output unit that outputs, for each of the video frames, the scene name of the scene in which the video frame is included and the label;
The scene dividing unit divides the video into scenes based on the video and the genre information of the video, and uses a first learning model that determines the scene name for each of the scenes. dividing the scene and determining the scene name;
Video analysis device. The label setting unit selects the same scene frame which is the video frame having the same scene name, the scene name of the same scene frame, and the genre information of the video including the same scene frame. Setting the labels using a second learning model that sets the labels for each of the video frames included in a scene frame;
The video analysis device according to claim 1. The output unit includes the scene name in metadata that is information regarding the video, and stores the scene name in a video storage device that stores the video and the metadata.
The video analysis device according to claim 1. A video analysis device according to claim 1;
A video analysis system comprising: a learning device that generates the first learning model. A video analysis device according to claim 2;
A video analysis system comprising: a learning device that generates the first learning model and the second learning model. Comprising the video analysis device according to claim 3 and an editing device,
The editing device includes:
an editing unit that edits the metadata in accordance with a scene editing instruction that is an instruction regarding editing the scene name;
The editorial department is
Obtaining a video to be edited and metadata regarding the video from the video storage device,
In response to a scene edit image display instruction that instructs display of a scene edit image that is an image for editing the scene name, the scene edit image is displayed on the user terminal based on the acquired video and the metadata. display,
Editing the scene name included in the metadata in response to the scene editing instruction, and storing the edited metadata in the video storage device.
Video analysis system. An editing device in a video analysis system according to claim 6. A learning device in a video analysis system according to claim 4. Using a first learning model that divides the video into scenes based on the video and genre information indicating the genre of the video, and determines a scene name indicating the classification of the scene for each of the scenes, dividing the video into the scenes and determining the scene name for each of the scenes;
setting a label, which is information about a subject imaged in the video frame, for each video frame included in the scene, based on the scene name;
outputting, for each of the video frames, the scene name and the label of the scene in which the video frame is included;
Video analysis method. to the computer,
a scene dividing function that divides the video into scenes based on genre information indicating a genre of the video, and determines a scene name indicating a classification of the scene for each of the scenes;
a label setting function that sets a label, which is information about a subject imaged in the video frame, for each video frame included in the scene, based on the scene name;
realizing, for each of the video frames, an output function that outputs the scene name and the label of the scene in which the video frame is included;
The scene division function divides the video into scenes based on the video and the genre information of the video, and uses a first learning model that determines the scene name for each of the scenes. dividing the scene and determining the scene name;
Video analysis program.

Images