WO2021095959A1

WO2021095959A1 - System and method for inserting scene information or shot information into video file

Info

Publication number: WO2021095959A1
Application number: PCT/KR2019/015971
Authority: WO
Inventors: 양창모; 김경원
Original assignee: 전자부품연구원
Priority date: 2019-11-15
Filing date: 2019-11-20
Publication date: 2021-05-20
Also published as: KR20210059409A

Abstract

A method for inserting scene information or shot information into an MPEG-4 video file, according to the present invention, comprises the steps of: opening the video file and a file, which includes the scene information or the shot information; analyzing location information and size information about file type box (ftyp) data, media data box (mdat) data, and movie box (moov) data for the video file; changing, according to the analysis result, the mdat data on the basis of whether the scene information or the shot information is inserted between chunk data; and correcting the moov data on the basis of whether the change is performed.

Description

System and method for inserting scene information or shot information into a video file

The present invention relates to a system and method for inserting scene information or shot information into a video file, and more particularly, to MPEG-4 video scene information or shot information without affecting playback of MPEG-4 standard video. 4 It relates to a system and method for embedding into standard standard video.

Video can be composed of individual video frames that can be grouped into multiple shots.

For example, a shot means a sequence of frames captured at a predetermined visual angle of the camera.

And a scene means a collection of shots that can be related in action, place, context and/or time.

For example, in some examples of programs such as comedy or soap operas or dramas, a scene may be characterized as a continuous set of shots that capture certain actions taking place at a particular location.

Meanwhile, recent attempts to generate revenue by processing and distributing a part of a video file, such as easy editing of a video or reuse for each clip of a video, have been actively conducted.

In the case of such a video file, as described above, it is common to classify each part of a video in units of scenes or shots, and recently, studies for detecting a scene or shot of a video based on a new technology such as AI have been actively conducted.

In the case of the prior art using video divided by scene or shot units, a method of creating scene or shot information as separate metadata, and then transmitting metadata including scene or shot information together when transmitting the video is a general method. Was used as.

However, this method has a disadvantage in that it is difficult to manage a file due to separate management of a video file and a metadata file, and is vulnerable to data omission or truncation when transmitting a video or metadata.

In order to solve the above-described problem, an embodiment of the present invention inserts scene information or shot information between chunk data, which is a continuous set of one MPEG-4 standard video file data, without affecting the reproduction of the video. At the same time, there is provided a system and method for generating a video file that allows the scene information or shot information of the transmitted video to be preserved as much as possible even if data is missing or truncated during video transmission.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, a method of inserting scene information or shot information into an MPEG-4 video file according to the first aspect of the present invention includes the video file and the scene information. Or opening a file including shot information; Analyzing location information and size information of ftyp (file type box) data, mdat (media data box) data, and moov (movie box) data for the video file; And changing the mdat data based on whether the scene information or shot information is inserted between chunk data according to the analysis result, and modifying the moov data based on the change.

In an embodiment, changing the mdat data may include inserting the scene information or shot information between the chunk data included in the mdat data, and modifying the size information of the mdat data. .

In an embodiment, the inserting of the scene information or shot information between the chunks includes: determining whether the scene information or shot information to be inserted is present; Analyzing start and end frame information of the scene information or shot information when the scene information or shot information to be inserted is present; Determining whether to insert the scene information or shot information into an end of the first chunk data; And inserting the scene information or shot information into an end of the first chunk data.

In an embodiment, the determining whether to insert the scene information or shot information into the end of the first chunk data includes the first chunk data including video data and the first chunk data. When the video frame number is greater than the last frame number of the scene information or shot information, it may be determined to be inserted.

In one embodiment, the step of inserting the scene information or shot information between the chunks is, when it is not determined to be inserted into the first chunk data, at the end of the second chunk data positioned after the first chunk data. It may include determining whether to insert the scene information or shot information.

In one embodiment, the inserting of the scene information or shot information between the chunks includes: reading scene information or shot information to be inserted next according to the insertion into the first chunk data; And reading the read scene information or second chunk data positioned after the first chunk data into which the shot information is to be inserted.

In an embodiment, in the modifying the moov data, stco (chunk offset box) information including location information of the chunk data included in the moov data may be modified.

In addition, a system including a memory storing a program for inserting scene information or shot information into an MPEG-4 video file according to the second aspect of the present invention and a processor executing a program stored in the memory , As the processor executes the program, it opens the video file and a file including the scene information or shot information, and ftyp (file type box) data, mdat (media data box) for the video file After analyzing the location information and size information of data and moov (movie box) data, the mdat data is changed based on whether the scene information or shot information is inserted between chunk data according to the analysis result, and the Modifies moov data based on whether it has changed.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

According to any one of the above-described problem solving means of the present invention, there is an advantage in that file management is more efficient than a method of managing video files and scene/shot data as separate files.

In addition, by inserting scene information or shot information between chunk data, there is an effect that the reproduction of MPEG-4 video is not affected.

In addition, by inserting scene information or shot information at the end of the chunk data, it is possible to preserve the scene information or shot information of the transmitted video as much as possible even if data is missing or truncated during video transmission.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a diagram for describing a system according to an embodiment of the present invention.

2 is a flowchart of an insertion method according to an embodiment of the present invention.

3 is a flowchart illustrating a method of changing mdat data.

4 is a flowchart illustrating a method of inserting scene information or shot information.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts not related to the description are omitted in order to clearly describe the present invention.

In the entire specification, when a certain part "includes" a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated.

1 is a diagram illustrating a system 100 according to an embodiment of the present invention.

The system 100 according to an embodiment of the present invention includes a memory 110 and a processor 120.

A program for inserting scene information or shot information into an MPEG-4 video file is stored in the memory 110.

In this case, the memory 110 collectively refers to a nonvolatile storage device and a volatile storage device that continuously maintains stored information even when power is not supplied. For example, the memory 120 may be a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD), and a micro SD card. Magnetic computer storage devices such as NAND flash memory, hard disk drives (HDD), such as cards, and optical disc drives such as CD-ROM, DVD-ROM, etc. I can.

As the processor 120 executes the program stored in the memory 110, it opens a video file and a file including scene information or shot information.

In addition, the processor 120 analyzes the location information and size information of ftyp (file type box) data, mdat (media data box) data, and moov (movie box) data for the video file, and then chunks ( Chunk) The mdat data is changed based on whether the scene information or shot information is inserted between data, and the moov data is modified based on the change.

For reference, the constituent elements shown in FIG. 1 according to an embodiment of the present invention may be implemented in software or in hardware such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and perform predetermined roles. can do.

However,'components' are not meant to be limited to software or hardware, and each component may be configured to be in an addressable storage medium or configured to reproduce one or more processors.

Thus, as an example, components are components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, and sub- Includes routines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables.

Components and functions provided within the components can be combined into a smaller number of components or further divided into additional components.

Hereinafter, a method performed in the system 100 will be described in more detail with reference to FIGS. 2 to 4.

2 is a flowchart of an insertion method according to an embodiment of the present invention. 3 is a flowchart illustrating a method of changing mdat data. 4 is a flowchart illustrating a method of inserting scene information or shot information.

According to an embodiment of the present invention, first, a file including scene information or shot information and an MPEG-4 video file (hereinafter referred to as a video file) to which the scene information or shot information is to be inserted are opened (S110). .

Next, the structure of the opened video file is analyzed (S120).

In this step, location information and size information of ftyp (file type box) data, mdat (media data box) data, and moov (movie box) data based on the start of the video file are analyzed.

An MPEG-4 video file consists of ftyp data, which is a file type box that checks the compatibility of files, moov data, which is a movie box that stores all metadata of the media, and mdat data, which is a media data box that stores actual media.

As an embodiment, in this step, among information included in moov data, stco (chunk offset box) information including location information of each chunk data may be analyzed.

For reference, moov data is composed of mvhd (movie header box), and a plurality of traks (track boxes), which are boxes defining a single track in a movie. And trak includes a sample table box (stbl) that describes information on media sample data.

In addition, stbl includes table box information such as stsd (sample description box), stts (time-to-sample box), stsz (sample size box), stsc (sample to chunk box), and stco (chunk offset box). .

Next, the mdat data is changed based on whether scene information or shot information is inserted between chunk data according to the analysis result (S130).

In one embodiment, the step of changing mdat data includes inserting scene information or shot information between chunk data included in mdat data (S131), and correcting size information of mdat data (S132). I can.

First, the step of inserting scene information or shot information between chunk data included in mdat data (S131) will be described as follows.

First, it is determined whether there is scene information or shot information to be inserted (S1311).

Here, when the scene information or shot information to be inserted no longer exists (S1311-N), step S132 is performed.

In contrast, if there is scene information or shot information to be inserted (S1311-Y), the scene information or shot information is analyzed. In this step, the process of grasping the start frame and end frame information of the scene information or shot information is performed. It is performed (S1312).

Thereafter, the first chunk data is analyzed (S1313). In the step of analyzing the first chunk data, a type of data included in the chunk data and a number of a video frame included in the chunk data when the chunk data includes a video are analyzed.

At this time, in order to read the chunk data, location information and size information of the current chunk data are required.

Meanwhile, the first and second chunk data mentioned herein do not mean the chunk data at a specific location. That is, the current chunk data of any one of the chunk data at an arbitrary position is referred to as first chunk data, and the chunk data at the next position is referred to as second chunk data.

In one embodiment, the location information of the first chunk data is the index value of the current chunk data described in stco, and the size information is a value obtained by subtracting the index value of the first chunk data from the index value of the second chunk data. I can.

Next, it is determined whether to insert scene information or shot information into the end of the first chunk data (S1314).

In an embodiment, if the type of data included in the first chunk data is video data, and the video frame number included in the first chunk data is greater than the last frame number of the scene information or the shot information, insertion is determined (S1314). -Y).

According to this determination, scene information or shot information is inserted into the end of the first chunk data (S1315).

On the contrary, when it is not determined that scene information or shot information is inserted into the first chunk data (S1314-N), the second chunk data positioned after the first chunk data is subject to the scene information or the same as the first chunk data. Whether to insert shot information is determined (S1316).

Accordingly, after a process of analyzing the second chunk data is performed, a process of determining whether to insert or not is determined is performed.

Next, after scene information or shot information is inserted into the first chunk data, the next scene information or shot information to be inserted is read (S1317).

Then, the read scene information or second chunk data positioned after the first chunk data into which the shot information is to be inserted is read (S1318). If scene information or shot information is inserted into the second chunk data in the above step, the third chunk data is read, and this process is repeated until all scene information or shot information is inserted.

Referring back to FIG. 3, when insertion of scene information or shot information between chunk data is completed, size information of mdat data is corrected (S132).

The mdat data is composed of 4 bytes of size information, 4 bytes of type information, and a plurality of chunk data.

In this process, size information of mdat data is corrected by summing the size of the additional data inserted between the chunk data and the size of the chunk data.

When the modification of the mdat size information is completed, the moov data is modified (S140).

Even after inserting scene information or shot information, moov data information must be modified for normal video playback.

To this end, an embodiment of the present invention may modify stco information including location information of chunk data included in moov data.

Since stco information has position information of each chunk data, the offset information of stco is updated by summing the scene information or shot information inserted between the chunk data from the existing offset information.

According to an embodiment of the present invention, insertion of scene information or shot information between chunk data in an MPEG-4 standard video is completed.

In the above description, steps S110 to S140 may be further divided into additional steps or may be combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order between steps may be changed. In addition, even if other contents are omitted, the contents already described in Fig. 1 are also applied to the insertion method in Figs.

An embodiment of the present invention may be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transmission mechanism, and includes any information delivery media.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention do.

Claims

In the method of inserting scene information or shot information into an MPEG-4 video file,

Opening the video file and a file including the scene information or shot information;

Analyzing location information and size information of ftyp (file type box) data, mdat (media data box) data, and moov (movie box) data for the video file;

Changing the mdat data based on whether the scene information or shot information is inserted between chunk data according to the analysis result; and

And modifying moov data based on the change.
The method of claim 1,

The step of changing the mdat data,

Inserting the scene information or shot information between the chunk data included in the mdat data, and

And modifying the size information of the mdat data.
The method of claim 2,

Inserting the scene information or shot information between the chunks,

Determining the presence or absence of the scene information or shot information to be inserted;

Analyzing start and end frame information of the scene information or shot information when the scene information or shot information to be inserted is present;

Determining whether to insert the scene information or shot information into an end of the first chunk data; And

Inserting the scene information or shot information into an end of the first chunk data.
The method of claim 3,

The step of determining whether to insert the scene information or shot information at the end of the first chunk data,

And determining to insert when the first chunk data includes video data and the video frame number included in the first chunk data is greater than the last frame number of the scene information or shot information.
The method of claim 3,

Inserting the scene information or shot information between the chunks,

And determining whether to insert the scene information or shot information at an end of second chunk data positioned after the first chunk data when it is not determined to be inserted into the first chunk data.
The method of claim 3,

Inserting the scene information or shot information between the chunks,

Reading scene information or shot information to be inserted next as it is inserted into the first chunk data; And

And reading second chunk data positioned after the first chunk data into which the read scene information or shot information is to be inserted.
The method of claim 1,

The step of modifying the moov data,

The insertion method of modifying stco (chunk offset box) information including position information of the chunk data included in the moov data.
A system comprising a memory storing a program for inserting scene information or shot information into an MPEG-4 video file, and a processor executing a program stored in the memory,

As the processor executes the program, it opens the video file and a file including the scene information or shot information, and ftyp (file type box) data and mdat (media data box) data for the video file And after analyzing location information and size information of moov (movie box) data, changing the mdat data based on whether the scene information or shot information is inserted between chunk data according to the analysis result, and the change The system that modifies the moov data based on whether or not.