KR102558504B1 - Scene-based video organization method - Google Patents

Abstract

A scene-based video organizing method is disclosed. A video organizing method performed in a computing device according to an aspect of the present invention includes the steps of storing image content information based on input information including information about a plurality of scenes; generating recognition information for scene recognition by analyzing the screen and sound source of the target image; and determining a scene corresponding to the recognition information based on the image content information.

Description

Scene-based video organization method {Scene-based video organization method}

The present invention relates to a method for arranging a video based on a scene.

Video contents such as movies and dramas are composed of a plurality of scenes. Conventionally, a method in which hitting a slate or a clapperboard is filmed together, and through this, an editor directly recognizes which scene the video is about with the naked eye is used. That is, information about the date, scene number, cinematographer, etc. is written on the slate or clapper board, and the scene in which the filming staff manipulates the slate or clapper board is filmed at the beginning and/or end of the video content scene, so that the editor can distinguish which scene was filmed.

According to this method, since the editor has to check the captured images one by one and recognize the scenes, much effort is required to identify and classify the scenes of the captured images. In addition, there is a hassle in that the slate or clapper board must be continuously inserted during the shooting.

Korean Patent Publication No. 10-2019-0087711, image pre-processing method, device and computer program

Therefore, the present invention has been made to solve the above problems, and is to provide a scene-based video organizing method and apparatus for classifying scenes by image analysis using deep learning.

Other objects of the present invention will become clearer through preferred embodiments described below.

According to one aspect of the present invention, in a video organizing method performed on a computing device, the step of storing content information based on input information including information on a plurality of scenes constituting video content; generating recognition information for recognizing a scene by analyzing at least one of a screen and a sound source of a target image; and determining, based on the content information, a scene corresponding to the recognition information as a confirmation scene for the target video, a scene-based moving picture arrangement method and a recording medium recording a program for executing the method.

Here, the step of assigning identification information including information about the confirmation scene to the target video, wherein a take number is given as the identification information according to the existence and number of videos determined to be the same as the confirmation scene among pre-stored videos.

In addition, based on the content information, shots corresponding to the recognition information are divided into subdivided scenes to check the target image.

In addition, based on the information about the confirmed scene of the target video, when the target video is determined to be short of a section, the target video is deleted or managed as an NG video.

In addition, if the target video is determined to be the NG video, it is checked whether the target video contains a sound of laughter, and classified according to the check result.

The method further includes generating NG video collection contents by dividing and managing the level of the laughing sound, and extracting and merging a plurality of NG videos using the level.

According to the present invention, by automatically recognizing and classifying scenes of captured images, it is possible to minimize not only the convenience of shooting but also the post-work of classifying images.

1 is a configuration diagram schematically illustrating an entire system for arranging a scene-based video according to an embodiment of the present invention.
2 is a table illustrating identification information for distinguishing scenes of a captured image according to an embodiment of the present invention.
3 is a flowchart illustrating a process of determining a scene of a target image according to an embodiment of the present invention;
4 is a flowchart illustrating a process of assigning identification information for scene, shot, and take numbers according to an embodiment of the present invention.
5 and 6 are flowcharts illustrating a process of identifying an NG image according to each embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

It should be understood that when an element is referred to as being “connected” or “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, terms such as a first threshold and a second threshold, which will be described later, may be substantially different from each other or some of them may be pre-designated as thresholds having the same value.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but it should be understood that the presence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof is not excluded in advance.

In addition, the components of the embodiments described with reference to each drawing are not limitedly applied to the corresponding embodiment, and may be implemented to be included in other embodiments within the scope of maintaining the technical idea of the present invention, and even if a separate description is omitted, it is natural that a plurality of embodiments may be re-implemented as one integrated embodiment.

In addition, in the description with reference to the accompanying drawings, the same or related reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted. In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

1 is a configuration diagram schematically illustrating an entire system for organizing a scene-based video according to an embodiment of the present invention, FIG. 2 is a table illustrating identification information for distinguishing scenes of a captured image according to an embodiment of the present invention, and FIG. 3 is a flowchart illustrating a process of determining a scene of a target video according to an embodiment of the present invention.

Referring first to FIG. 1 , the entire system according to the present embodiment includes an image capturing device 10 , an image processing device 20 and a content information DB 30 .

The image capture device 10 is a device capable of generating an image, and may include a camera. The image capturing device 10 may be a single device including one or more cameras. The image capture device 10 may be connected to the image processing device 20 through a wired or wireless connection.

The image processing device 20 classifies and classifies images captured by the image capturing device 10, such as one or more cameras, into scenes, shots, and takes, generates identification information, and stores it in the content information DB 30. For example, the image processing device 20 may receive data about a photographed image from the connected image capturing device 10 . Also, when a captured image is received, the image processing device 20 may generate identification information corresponding to the captured image according to a preset method. At this time, the image processing device 20 may use content information stored in the content information DB 30 to generate identification information corresponding to the captured image.

The content information DB 30 is a device connected to the image processing device 20 by wire or wirelessly, and information on a plurality of scenes constituting image content may be stored therein as content information. The content information may include character information (gender, number of people, etc.), shooting location information, props information, dialogue information, etc. for each scene.

Accordingly, the image processing device 20 may automatically assign identification information about scenes, shots (or cuts), and takes to each captured image. 2 illustrates a scene classification process by analyzing a photographed image performed by the image processing device 20 .

Referring to FIG. 2 , the image processing device 20 may store content information (S310). Content information is information based on input information, and may include information about a plurality of scenes constituting video content. The content information may be information stored in the content information DB 30 and downloaded from the content information DB 30 by the image processing device 20 . Alternatively, the content information may be received from another external device (not shown) connected to the image processing device 20 . Alternatively, the content information may be information automatically generated by the image processing device 20 . For example, when text information corresponding to a movie scenario is input, the image processing device 20 analyzes the text or image of the scenario to extract content information such as characters, places, lines, props, etc. of each scene and store it as content information.

Then, when a captured image (hereinafter referred to as 'object image') to be arranged is input, the image processing device 20 analyzes the screen of the target video (each frame image constituting the video) and the sound source (effect sound, lines, etc.) to generate recognition information for scene recognition (S320). For example, the number of characters in the target image, the character's gender, the character's age, the character's "emotion, main" background and props, "text" information through voice, etc. can be used as recognition information.

Here, deep learning technology may be used to increase the accuracy of extracting recognition information by image analysis.

That is, the image processing device 20 may analyze the number of characters for each cut in the target image. At this time, the image processing device 20 may use a face recognition algorithm. A preset face recognition algorithm may be stored in the image processing device 20, and the image processing device 20 may use the face recognition algorithm to analyze the number, age, gender, etc. of characters photographed in each cut in the target image.

Also, the image processing device 20 may analyze the types of objects appearing in each cut in the target image. At this time, the image processing device 20 may use an object detection algorithm. A preset object recognition algorithm may be stored in the image processing device 20, and the image processing device 20 may analyze objects (eg, tables, chairs, clocks, beds, etc.) captured in each cut in the target image using the object recognition algorithm.

Also, the image processing device 20 may analyze the audio included in the target image to analyze the type of audio (wind, bird sound, water sound, musical instrument sound, etc.). At this time, the image processing device 20 may use an audio detection deep-learning algorithm for sound recognition.

Also, if the audio included in the target image is a human voice, the image processing device 20 may extract the dialogue as text.

A technology for analyzing each image in a video and recognizing a subject included in the image is still used in various fields (e.g., YOLO algorithm, etc.), and a technology for converting voice into text and a voice recognition technology that analyzes emotions as well as gender and age by analyzing voice waveforms are also widely used. Therefore, the implementation method of the above-described deep learning algorithm will be obvious to those skilled in the art. Therefore, a detailed description thereof will be omitted.

Then, the image processing device 20 may compare the pre-stored content information with the recognition information generated in S320, determine a scene corresponding to the recognition information, and set the determined scene as a confirmation scene for the target image (S330).

Additionally, when the scene of the target video is confirmed, the image processing device 20 can classify and manage which shot the corresponding scene was filmed in, and if the same scene (and shot or cut) already exists, it can be managed with different take numbers. 4 is a flowchart illustrating a process of assigning identification information for scene, shot, and take numbers according to an embodiment of the present invention.

Referring to FIG. 4 , the image processing device 20 may subdivide a confirmation scene for a target image by classifying not only scenes but also shots using recognition information (S410). That is, the image processing device 20 may assign identification information including a confirmation scene and a confirmation shot to the target image (S420), and check whether an image having the same scene and shot as identification information is stored in advance (S430).

If it does not exist, the image processing device 20 may assign the take number of the target image to #1 (S440). On the other hand, if a video having the same scene and shot as identification information already exists, the video processing device 20 may assign a number next to the highest take number as the take number of the corresponding target video (S450).

Meanwhile, when producing video content such as a movie, some of the captured videos need to be classified as NG (No Good) videos. In the case of NG videos, there will be many cases where the scene has a shorter section than the normal composition time (eg, 1 minute or 10 lines). This is because NG is often stopped before all the lines constituting the scene are progressed. Accordingly, the image processing device 20 may use this to discriminate the NG image.

5 and 6 are flowcharts illustrating a process of identifying an NG image according to each embodiment of the present invention.

Referring to FIG. 5 , the image processing device 20 may check a section (composition time or entire dialogue) of a scene corresponding to a confirmed scene of a target image (S510).

Then, the image processing device 20 may compare the identified section with the section of the target image to determine whether the section is insufficient (S520). For example, if the checked section is 5 minutes, but the section of the target video is 2 minutes, the image processing device 20 may determine this as 'insufficient'.

If not enough, the image processing device 20 may assign a take number to the target image (S530). Conversely, if insufficient, the image processing device 20 may delete the target image or classify and manage it as an NG image (S540).

Referring to FIG. 6 showing a specific embodiment of this, if it is determined that the section of the target video is insufficient, it can be additionally checked whether there is a sound of laughter in the target video (S541).

If there is no laughing sound, the image processing device 20 may classify the NG image as a normal NG image (S543). Conversely, if there is a sound of laughter, the image processing device 20 may classify and manage the corresponding target image as a utilized NG image (S545).

In particular, the image processing device 20 may classify the level of laughter (eg, loudness of laughter) and manage them together. Accordingly, the image processing device 20 may automatically generate NG video collection contents by extracting and merging those having a high laughing sound level among NG videos to be used in the future.

Here, it is natural that the classification of target images stored as NG images can be changed by the administrator. That is, after automatically classifying captured images, the image processing device 20 may provide classified information to a manager and provide an interface allowing the manager to change the classified information.

The above-described method for arranging a video based on a scene according to the present invention can be implemented as computer readable code on a computer readable recording medium. Computer-readable recording media includes all types of recording media in which data that can be decoded by a computer system is stored. For example, there may be read only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, and the like. In addition, the computer-readable recording medium may be distributed to computer systems connected through a computer communication network, and stored and executed as readable codes in a distributed manner.

In addition, although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the present invention described in the claims below.

10: video recording device
20: image processing device
30: Content Information DB

Claims (6)

In the video organizing method performed on a computing device,
Storing content information based on input information including information on a plurality of scenes constituting video content;
generating recognition information for recognizing a scene by analyzing at least one of a screen and a sound source of the target image; and
comparing the content information with the recognition information, determining a scene corresponding to the recognition information, and determining the determined scene as a confirmation scene for the target image;
Including,
The target image corresponds to some or all candidates of an arbitrary scene of the image content,
The content information includes at least one of character information, prop information, and dialogue information for each of the plurality of scenes,
The recognition information includes at least one of the number of characters in the target image, character information, prop information, and text information through voice.
According to claim 1,
assigning identification information including information about the confirmation scene to the target image;
Including more,
A scene-based video organizing method of assigning a take number as the identification information according to the existence and number of images corresponding to the confirmed scene among pre-stored images.
According to claim 2,
Subdividing the verification scene into verification shots using the recognition information;
Further comprising a scene-based video organizing method.
According to claim 1,
deleting or managing the target video as an NG video when it is determined that the target video is short of a section based on the content information corresponding to the recognition information;
Further comprising a scene-based video organizing method.
According to claim 4,
if the target video is determined to be the NG video, checking whether a laughing sound is included in the target video; and
Classifying and classifying according to the confirmation result;
Further comprising a scene-based video organizing method.
According to claim 5,
Classifying the level of the laughter; and
automatically generating NG video collection contents by extracting a plurality of NG videos using the level;
Further comprising a scene-based video organizing method.