KR20210064587A - High speed split device and method for video section - Google Patents

Abstract

According to one embodiment of the present invention, provided is a video section high-speed segmentation device, which includes at least one processor. The at least one processor determines I frames in the video, and extracts visual features for each of the determined I frames.

Description

High-speed video segmentation device and high-speed segmentation method {HIGH SPEED SPLIT DEVICE AND METHOD FOR VIDEO SECTION}

The following embodiments relate to a high-speed division apparatus and a high-speed division method for a video section.

A codec (English: codec) is hardware or software that can encode, decode, or both for any data stream or signal. In general, a codec refers to a technology for compressing media information such as video and sound.

Frames are largely divided into independently encoded frames and predicted frames. The former is an I-frame (Intra-coded frame), and the latter is divided into a P-frame (Predictive-coded frame) and a B-frame (Bidirectional-coded frame). I-frames (Intra-coded frames) can come anywhere in the data stream, are used for random access to data, and are coded without reference to other images. A P-frame (Predictive-coded frame) uses the previous I-frame information and the previous P-frame information when encoding and decoding. A B-frame (Bidirectional-coded frame) uses both the I-frame and the P-frame before and after performing encoding and decoding.

Machine learning is a field of artificial intelligence that has evolved from the study of pattern recognition and computer learning theory, and refers to a field that develops algorithms and technologies that allow computers to learn.

Deep learning is defined as a set of machine learning algorithms that attempt high-level abstraction through a combination of several nonlinear transformation methods. field can be said.

According to an embodiment of the present invention, it is possible to provide a video section high-speed segmentation apparatus and a high-speed segmentation method capable of high-speed segmentation of a video section based on visual features for each of the I frames.

Further, according to another embodiment of the present invention, it is to provide a high-speed segmentation apparatus and a high-speed segmentation method for a moving image section capable of rapidly segmenting a segment of a moving image based on language features for each of the I frames can

In addition, according to another embodiment of the present invention, high-speed segmentation of a video section capable of rapidly segmenting a section of a video based on visual features and language features for each of the I frames A device and a high-speed partitioning method can be provided.

In addition, according to another embodiment of the present invention, the section of the video is determined based on the length of visual features, language features, and data units for each of the I frames. It is possible to provide a high-speed division apparatus and a high-speed division method for a video section capable of high-speed division.

In addition, according to another embodiment of the present invention, a video section capable of rapidly dividing a section of a video based on visual features for each of the I frames and the length of a data unit (date unit) A high-speed division apparatus and a high-speed division method can be provided.

In addition, according to another embodiment of the present invention, a video section capable of rapidly dividing a section of a video based on language features for each of the I frames and the length of a data unit (date unit). A high-speed division apparatus and a high-speed division method can be provided.

According to an embodiment of the present invention, in an apparatus for high-speed segmentation of a video section, the apparatus includes at least one processor, wherein the at least one processor determines I frames in the video, and the determined I frames ) to extract the visual features for each.

Also, the at least one processor may divide the section of the video based on the extracted visual features.

Also, the at least one processor may generate an image caption for each of the determined I frames, and extract language features from the generated image caption.

Also, the at least one processor may divide the section of the video based on the extracted visual features or the extracted language features.

Also, the at least one processor may calculate the length of a data unit for each of the determined I frames.

In addition, the at least one processor may divide the section of the video based on the extracted visual features, the extracted language features, or the calculated length of the data unit. .

Also, the at least one processor may calculate the length of a data unit for each of the determined I frames.

Also, the at least one processor may divide the section of the video based on the extracted visual features or the calculated length of the data unit (date unit).

According to another embodiment of the present invention, in an apparatus for dividing a video section at a high speed, the apparatus includes at least one processor, wherein the at least one processor determines I frames in the video, and the determined I frames ), an image caption is generated for each, and language features are extracted from the generated image caption.

Also, the at least one processor may divide the section of the video based on the extracted language features.

In addition, the at least one processor calculates a length of a data unit for each of the determined I frames, and uses the extracted language features or the calculated date unit. ) can be divided into sections of the video based on the length of the video.

According to another embodiment of the present invention, an operation of determining I frames from a moving picture and an operation of extracting visual features for each of the determined I frames are included.

In addition, the high-speed segmentation method of the video section may further include dividing the section of the video based on the extracted visual features.

In addition, the high-speed segmentation method of the video section may further include generating an image caption for each of the determined I frames and extracting language features from the generated image caption. have.

In addition, the high-speed segmentation method of the video section may further include dividing the section of the video based on the extracted visual features or the extracted language features.

Also, the high-speed segmentation method of the video section may further include calculating the length of a data unit for each of the determined I frames.

In addition, the high-speed segmentation method of the video section includes dividing the section of the video based on the extracted visual features, the extracted language features, or the calculated length of the data unit (date unit). may further include.

Also, the high-speed segmentation method of the video section may further include calculating the length of a data unit for each of the determined I frames.

In addition, the high-speed segmentation method of the video section may further include dividing the section of the video based on the extracted visual features or the calculated length of the data unit (date unit).

According to another embodiment of the present invention, the operation of determining I frames in a video, the operation of generating an image caption for each of the determined I frames, and the language feature ( language features).

In addition, the high-speed segmentation method of the video section may further include dividing the section of the video based on the extracted language features.

In addition, the high-speed segmentation method of the video section includes an operation of calculating the length of a data unit for each of the determined I frames, and the extracted language features or the calculated data unit ( The operation of dividing the section of the video based on the length of the date unit) may be further included.

According to an embodiment of the present invention, there is an effect of high-speed segmentation of a section of a video based on visual features of each of the I frames.

In addition, there is an effect of high-speed division of a section of a video based on language features for each of the I frames.

In addition, there is an effect of high-speed segmentation of a section of a video based on visual features and language features of each of the I frames.

In addition, there is an effect of high-speed segmentation of a section of a video based on the length of visual features, language features, and data units for each of the I frames.

In addition, there is an effect of high-speed segmentation of a section of a video based on the length of a data unit and visual features for each of the I frames.

In addition, there is an effect of high-speed division of a section of a video based on the length of a data unit and language features for each of the I frames.

1 is a diagram illustrating a configuration of a video section high-speed segmentation apparatus according to an embodiment.
2 is a flowchart illustrating a method for high-speed segmentation of a video section according to an exemplary embodiment.
3 is a flowchart illustrating a method for high-speed segmentation of a video section according to another exemplary embodiment.
4 is a diagram illustrating a state in which sections of a video are divided at high speed based on visual features of each of I frames according to an embodiment.
5 is a diagram illustrating a state in which sections of a video are divided at high speed based on visual features and language features of each of I frames according to another embodiment.
6 is a diagram illustrating a video section and a frame constituting the video.

Specific structural or functional descriptions for the embodiments according to the concept of the present invention disclosed in this specification are only exemplified for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are It may be implemented in various forms and is not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention may have various changes and may have various forms, the embodiments will be illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all modifications, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one element from another element, for example, without departing from the scope of rights according to the concept of the present invention, a first element may be called a second element, and similarly The second component may also be referred to as the first component.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is mentioned that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Other expressions describing the relationship between elements, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

As used herein, terms such as “comprise” or “have” are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers. , it is to be understood that it does not preclude the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

In the following description, the same identification symbols mean the same configuration, and unnecessary redundant descriptions and descriptions of well-known technologies will be omitted.

In an embodiment of the present invention, 'communication', 'communication network' and 'network' may be used as the same meaning. The above three terms mean a wired/wireless short-distance and wide-area data transmission/reception network capable of transmitting and receiving files between a user terminal, terminals of other users, and a download server.

Hereinafter, the present invention will be described in detail by describing preferred embodiments of the present invention with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of a video section high-speed segmentation apparatus according to an embodiment.

Referring to FIG. 1 , the high-speed video section division apparatus 100 includes a processor 110 , an input/output interface module 120 , and a memory 130 .

The processor 110, the input/output interface module 120, and the memory 130 of the video section high-speed division apparatus 100 are interconnected, and it is possible to transmit data to each other.

The processor 110 may execute programs or instructions stored in the memory 130 . In this case, the memory 130 may store an operation program (eg, OS) for operating the high-speed video section division apparatus 100 .

The processor 110 may execute a program for managing information on the high-speed segmentation apparatus 100 of a video section.

The processor 110 may execute a program for managing the operation of the high-speed segmentation apparatus 100 for video sections.

The processor 110 may execute a program for managing the operation of the input/output interface module 120 .

The processor 110 may acquire a moving picture (a compressed video stream and a compressed bit stream) through the input/output interface module 120 .

The processor 110 may determine I frames, P frames, or B frames from the obtained video.

The processor 110 may select and decode only I frames from among the determined frames.

i) visual features

The processor 110 may extract visual features for each of the decoded I frames. In this case, the processor 110 may use a learned deep neural network to extract visual features for each of the decoded I frames, but the processor 110 may What can be used to extract visual features for each of the decoded I frames is not limited thereto.

The processor 110 may divide the section of the video based on the visual features extracted from each of the decoded I frames.

The processor 110 may set a point at which the visual features change abruptly as a reference point for dividing the video.

The processor 110 may set the interval between the set reference point and the reference point as a section having the same or similar visual features.

The processor 110 may divide the video into a plurality of sections based on the set section.

ii) language features

The processor 110 may generate an image caption for each of the decoded I frames by applying image captioning to each of the decoded I frames.

The processor 110 uses at least one learned deep neural network (eg, a convolutional neural network, CNN) to obtain an image corresponding to each of the decoded I frames. At least one learned deep neural network (Deep Neural Network) different from the at least one trained deep neural network (eg, convolutional neural network, CNN) that processed the image ) (eg, a recurrent neural network (RNN)) may be used to generate a sentence (image caption) for each of the processed images.

The processor 110 may extract language features for each sentence (image caption) generated for each of the processed images by using the learned deep neural network. However, what the processor 110 can use to extract language features for each sentence (image caption) generated for each of the processed images is not limited thereto.

The processor 110 may divide the section of the video based on the language features extracted from each sentence (image caption) generated for each of the processed images.

The processor 110 may set a point at which the language features change abruptly as a reference point for dividing the video.

The processor 110 may set a section having the same or similar language features between the set reference point and the reference point.

The processor 110 may divide the video into a plurality of sections based on the set section.

iii) visual features or language features;

The processor 110 may divide the video into a plurality of sections based on the extracted visual features or the extracted language features.

The processor 110 divides the video into a plurality of sections based on the extracted visual features or the extracted language features. The extracted visual features and the extracted language features features) may be assigned a weight to at least one of them.

The processor 110 may set a point at which the extracted visual features and the extracted language features change abruptly at the same time as a reference point for dividing the video.

The processor 110 may set a point at which a weighted feature among the extracted visual features or the extracted language features changes abruptly as a reference point for segmenting the video.

The processor 110 may set the interval between the set reference point and the reference point as a section in which the extracted visual features or the extracted language features are the same or similar.

The processor 110 may divide the video into a plurality of sections based on the set section.

iv) the length of the visual feature, the linguistic feature or the data unit (date unit);

The processor 110 may calculate the length of a data unit for each of the decoded I frames.

The processor 110 may divide the video into a plurality of sections based on the extracted visual features, the extracted language features, or the calculated length of the data unit (date unit).

The processor 110 divides the video into a plurality of sections based on the extracted visual features, the extracted language features, or the calculated length of the data unit (date unit). A weight may be assigned to at least one of visual features, the extracted language features, and the calculated length of the data unit (date unit).

The processor 110 selects a point at which the extracted visual features and the extracted language features abruptly change at the same time or a point at which the length of the calculated date unit sharply increases as the moving picture. It can be set as a reference point for division.

The processor 110 determines whether the weighted feature or the length of the data unit among the extracted visual features, the extracted language features, or the calculated length of the data unit is determined. A point that changes rapidly may be set as a reference point for dividing the video.

The processor 110 may set the interval between the set reference point and the reference point as a section in which the extracted visual features or the extracted language features are the same or similar.

The processor 110 may divide the video into a plurality of sections based on the set section.

v) length of visual features or data units

The processor 110 may divide the video into a plurality of sections based on the extracted visual features or the calculated length of the data unit (date unit).

The processor 110 divides the video into a plurality of sections based on the extracted visual features or the calculated length of the data unit, the extracted visual features and the calculated A weight may be assigned to at least one of the lengths of a data unit.

The processor 110 may set a point at which the extracted visual features abruptly change or a point at which the calculated data unit length rapidly increases as a reference point for dividing the moving picture.

The processor 110 divides the video at a point where the weighted feature or the length of the data unit changes abruptly among the extracted visual features or the calculated length of the data unit. It can be set as a reference point for

The processor 110 may set the interval between the set reference point and the reference point as a section in which the extracted visual features are the same or similar.

The processor 110 may divide the video into a plurality of sections based on the set section.

vi) length of language features or data units

The processor 110 may divide the video into a plurality of sections based on the extracted language features or the calculated length of the data unit (date unit).

The processor 110 divides the video into a plurality of sections based on the extracted language features or the calculated length of the data unit, the extracted language features and the calculated A weight may be assigned to at least one of the lengths of a data unit.

The processor 110 may set a point at which the extracted language features abruptly change or a point at which the calculated length of the data unit rapidly increases as a reference point for dividing the video.

The processor 110 divides the video at a point where the weighted feature or the length of the data unit changes rapidly among the extracted language features or the calculated length of the data unit. It can be set as a reference point for

The processor 110 may set the interval between the set reference point and the reference point as a section having the same or similar language features.

The processor 110 may divide the video into a plurality of sections based on the set section.

The processor 110 may output a plurality of sections obtained by dividing the video through the input/output interface module 120 .

The input/output interface module 120 may be connected to an external device (eg, a server) through a network.

The input/output interface module 120 may obtain data from an external device.

The input/output interface module 120 may acquire a moving picture (compressed video stream, compressed bit stream) from an external device.

The input/output interface module 120 may obtain a user's input.

The input/output interface module 120 may output a plurality of sections of the video divided by the processor 110 .

The input/output interface module 120 may be provided integrally with the video section high-speed division apparatus 100 .

The input/output interface module 120 may be provided separately from the video section high-speed division apparatus 100 .

The input/output interface module 120 may be a separate device to be communicatively connected to the video section high-speed division apparatus 100 .

The input/output interface module 120 may include a port (eg, a USB port) for connecting to an external device.

The input/output interface module 120 may include a monitor, a touch screen, a mouse, an electronic pen, a microphone, a keyboard, a speaker, an earphone, a headphone, or a touch pad.

The memory 130 may store data acquired through the input/output interface module 120 .

The memory 130 may store moving images (compressed video stream, compressed bit stream) acquired through the input/output interface module 120 .

The memory 130 may store the frames determined by the processor 110 .

The memory 130 may store I frames decoded by the processor 110 .

The memory 130 may store a reference point set by the processor 110 .

The memory 130 may store visual features extracted by the processor 110 .

The memory 130 may store a section of the video divided by the processor 110 based on visual features.

The memory 130 may store language features extracted by the processor 110 .

The memory 130 may store the image caption generated by the processor 110 .

The memory 130 may store the image processed by the processor 110 .

The memory 130 may store a sentence (image caption) generated by the processor 110 .

The memory 130 may store a weight assigned by the processor 110 .

The memory 130 may store the length of the data unit calculated by the processor 110 .

As used herein, the term 'module' refers to a logical structural unit, and it is obvious to those skilled in the art that the present invention is not necessarily a physically separate component.

2 is a flowchart illustrating a method for high-speed segmentation of a video section according to an exemplary embodiment.

Referring to FIG. 2 , the high-speed video section division apparatus determines I frames in the video ( 200 ).

In this case, the high-speed segmentation apparatus of the moving picture may decode only the I frames determined from the moving picture.

The video section high-speed segmentation apparatus extracts visual features for each of the determined I frames ( 210 ).

In this case, the high-speed segmentation apparatus of the video section may use a deep neural network learned to extract visual features for each of the determined I frames.

The high-speed video section division apparatus extracts language features for each of the determined I frames (220).

In this case, the high-speed segmentation apparatus of the video section may process an image corresponding to each of the decoded I frames using at least one learned convolutional neural network (CNN).

In addition, the high-speed segmentation apparatus of the video section may generate a sentence (image caption) for each of the processed images by using a recurrent neural network (RNN).

In addition, the high-speed segmentation apparatus of the video section may extract language features for each sentence (image caption) generated for each of the processed images by using a learned deep neural network. .

The high-speed video section division apparatus calculates the length of a data unit for each of the determined I frames ( 230 ).

The high-speed video section segmentation apparatus divides the video section based on the extracted visual features, the extracted language features, or the calculated length of the data unit (240).

In this case, the video section high-speed segmentation apparatus divides the video into a plurality of sections based on the extracted visual features, the extracted language features, or the calculated length of the data unit (date unit). For this purpose, a weight may be assigned to at least one of the extracted visual features, the extracted language features, and the calculated length of the data unit (date unit).

In addition, the high-speed segmentation apparatus for the video section determines a point at which the extracted visual features or the extracted language features abruptly change or a point at which the calculated length of the data unit rapidly increases. It can be set as a reference point for dividing the video.

In addition, the high-speed segmentation apparatus of the video section may set a section between the set reference point and the reference point as one section.

Also, the high-speed video section division apparatus may divide the video into a plurality of sections based on the set section.

The video section high-speed division apparatus may output a plurality of sections in which the video is divided ( 250 ).

In this case, the high-speed segmentation apparatus of the video section may output images corresponding to the decoded I frames in order to display the segmented section.

Although operations 210, 220, and 230 are illustrated as being sequentially performed in FIG. 2, operations 210, 220 and 230 may be performed simultaneously, in parallel, or overlapping, and operation 210 may be performed later than operations 220 and 230. can Also, operation 220 may be performed later than operation 230 .

Also, in FIG. 2 , after operations 210, 220, and 230 are completed, it is exemplified that the video section is divided based on the results of operations 210, 220, and 230. A section of the video may be divided by combining the results of at least any two of the operations, and according to another embodiment, the section of the video may be divided based on the results of each of operations 210, 220, and 230.

3 is a flowchart illustrating a method for high-speed segmentation of a video section according to another exemplary embodiment.

Referring to FIG. 3 , the high-speed video section division apparatus determines I frames in the video ( 300 ).

In this case, the high-speed segmentation apparatus of the moving picture may decode only the I frames determined from the moving picture.

The high-speed video section division apparatus generates image captions for each of the determined I frames ( 310 ).

In this case, the high-speed segmentation apparatus of the video section may process an image corresponding to each of the decoded I frames using at least one learned convolutional neural network (CNN).

In addition, the high-speed segmentation apparatus of the video section may generate a sentence (image caption) for each of the processed images by using a recurrent neural network (RNN).

The high-speed segmentation apparatus extracts language features from the generated sentence (image caption) ( 320 ).

In this case, the high-speed segmentation apparatus of the video section may extract language features for each sentence (image caption) generated for each of the processed images by using a learned deep neural network. .

The high-speed video section division apparatus calculates the length of a data unit for each of the determined I frames ( 330 ).

The high-speed video section segmentation apparatus divides the video section based on the extracted language features or the calculated length of the data unit (S340).

In this case, the high-speed segmentation apparatus for video section divides the video into a plurality of sections based on the extracted language features or the calculated length of the data unit (language features). and a weight may be assigned to at least one of the calculated lengths of the data unit (date unit).

In addition, the high-speed segmentation apparatus of the moving picture section sets a point at which the extracted language features abruptly change or a point at which the length of the calculated date unit rapidly increases as a reference point for dividing the moving picture. can

In addition, the high-speed segmentation apparatus of the video section may set a section between the set reference point and the reference point as one section.

Also, the high-speed video section division apparatus may divide the video into a plurality of sections based on the set section.

The high-speed video section division apparatus may output a plurality of sections in which the video is divided ( S350 ).

In this case, the high-speed segmentation apparatus of the video section may output images corresponding to the decoded I frames in order to display the segmented section.

Although it is exemplified in FIG. 3 as being performed before operations 300 to 320 and 330, operations 300 to 320 and 330 may be performed simultaneously, in parallel, or overlapping, and operations 300 to 320 are performed later than operation 330 can be

In addition, in FIG. 3 , the video section is divided based on the results of operations 320 and 330 after operations 300 to 330 are completed, but according to another embodiment, based on the results of operations 320 and 330, respectively You can divide a section of a video.

4 is a diagram illustrating a state in which sections of a video are divided at high speed based on visual features of each of I frames according to an exemplary embodiment.

Referring to FIG. 4 , the high-speed video section division apparatus may determine frames 400 , 410 , 420 , 430 , 440 , 450 , 460 , and 470 constituting the acquired video.

The apparatus for high-speed segmentation of a moving picture may determine I frames 400 , 420 , 440 , and 460 that occur one by one every 32 frames in the obtained moving picture according to an embodiment.

The high-speed video section division apparatus can decode only the I-frames 400 , 420 , 440 , and 460 that are determined to divide the acquired video section at high speed.

The video section high-speed segmentation apparatus may extract visual features for each of the decoded I frames 400 , 420 , 440 , 460 using a learned deep neural network. .

The high-speed video section division apparatus may rapidly divide the obtained video section based on visual features of each of the extracted I frames 400 , 420 , 440 , and 460 .

The high-speed video section division apparatus may output images corresponding to decoded I frames in order to display the divided section.

According to an embodiment, in the case of FIG. 4 , the high-speed video section division apparatus may output images corresponding to 400, 420, and 460 among decoded I frames to display the divided section.

5 is a diagram illustrating a state in which sections of a video are divided at high speed based on visual features and language features of each of I frames according to another embodiment.

Referring to FIG. 5 , the high-speed segmentation apparatus for a moving image may determine frames 500 , 510 , 520 , 530 , 540 , 550 , 560 and 570 constituting the obtained moving image.

The apparatus for high-speed segmentation of a moving picture may determine I frames 500 , 520 , 540 , and 560 that are generated one by one every 32 frames in the obtained moving picture according to an embodiment.

The high-speed video section division apparatus can decode only the I-frames 500 , 520 , 540 , and 560 that are determined to divide the acquired video section at high speed.

The video section high-speed segmentation apparatus may extract visual features for each of the decoded I frames 500 , 520 , 540 , and 560 using a learned deep neural network. .

The video section high-speed segmentation apparatus can process an image corresponding to each of the decoded I frames (500, 520, 540, 560) using at least one learned convolutional neural network (CNN). have.

The video section high-speed segmentation apparatus uses a recurrent neural network (RNN) to process a sentence (image caption) for each image corresponding to each of the decoded I frames (500, 520, 540, 560). ) (501, 521, 541, 561).

The video section high-speed segmentation apparatus may extract language features from the generated sentences (image captions) 501 , 521 , 541 , and 561 using a learned deep neural network.

The high-speed video section segmentation apparatus may divide the obtained video section based on the extracted visual features and the extracted language features.

The video section high-speed segmentation apparatus divides the obtained video into a plurality of sections based on the extracted visual features or the extracted language features, the extracted visual features and the extracted language At least one of language features may be weighted.

The high-speed segmentation apparatus of a moving picture may set a point at which the extracted visual features or the extracted language features abruptly change as a reference point for segmenting the obtained moving image.

The high-speed video section division apparatus may set a section between the set reference point and the reference point as one section.

The high-speed video section division apparatus may divide the obtained video into a plurality of sections based on a preset section.

The high-speed video section division apparatus may output images corresponding to decoded I frames in order to display the divided section.

According to an embodiment, in the case of FIG. 5 , the high-speed video section division apparatus may output images corresponding to 500, 520, and 660 among decoded I frames to display the divided section.

6 is a diagram illustrating a video section and a frame constituting the video.

Referring to FIG. 6 , a moving picture (a compressed video stream and a compressed bit stream) is formed by combining a plurality of sections 600 and 650 .

A video compression stream (movie, compressed bit stream) is composed of a raw header (NAL Header, NH) 611 and a data unit (Data Unit) 612 . Here, a Network Abstraction Layer (NAL) header 611 is data-related information in units of video frames, and a data unit 612 is an actual data part.

A NAL Header (NH) 611 and a data unit 612 are formed in units of frames 610 , 620 , 630 , 640 , and 650 and include information for configuring one frame.

When a scene change occurs between I frames, it is difficult to accurately predict motion, so residual information increases, and the bit amount increases due to the increase of residual information, and finally This increases the length of the data unit.

The apparatus for high-speed segmentation of a moving picture may determine I-frames 610 and 660 that occur one by one every 32 frames, according to an embodiment, in the obtained moving picture.

The high-speed video section division apparatus may decode only the I frames 610 and 660 determined to rapidly divide the obtained video section.

The video section high-speed segmentation apparatus may extract visual features for each of the decoded I frames 610 and 660 using a learned deep neural network.

The video section high-speed segmentation apparatus may process an image corresponding to each of the decoded I frames 610 and 660 using at least one learned convolutional neural network (CNN).

The video section high-speed segmentation apparatus generates a sentence (image caption) for each image corresponding to each of the decoded I frames 610 and 660 processed above using a Recurrent Neural Network (RNN). can

The high-speed segmentation apparatus for video sections may extract language features from the generated sentences (image captions) using a learned deep neural network.

The high-speed video section division apparatus may calculate the length of a data unit 612 for each of the determined I frames 610 and 660 .

The video section high-speed segmentation apparatus includes a data unit 612 for each of the extracted visual features, the extracted language features, and the calculated I frames 610 and 660. A section of the obtained video may be divided based on the length of .

The video section high-speed segmentation apparatus includes a data unit 612 for each of the extracted visual features, the extracted language features, or the calculated I frames 610 and 660. Each of the extracted visual features, the extracted language features, and the calculated I frames 610 and 660 to divide the obtained video into a plurality of sections based on the length of A weight may be assigned to at least one of the lengths of the data unit 612 for .

The video section high-speed segmentation apparatus is a data unit (date) for a point at which the extracted visual features or the extracted language features change abruptly or the calculated I frames 610 and 660, respectively. A point at which the length of the unit 612 rapidly increases may be set as a reference point for dividing the acquired video.

The high-speed video section division apparatus may set a section between the set reference point and the reference point as one section.

The high-speed video section division apparatus may divide the obtained video into a plurality of sections based on a preset section.

The high-speed video section division apparatus may output images corresponding to decoded I frames in order to display the divided section.

In the above, even though all the components constituting the embodiment of the present invention are described as being combined or operating in combination, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining at least one.

In addition, although all of the components may be implemented as one independent hardware, some or all of the components are selectively combined to perform some or all functions of the combined components in one or a plurality of hardware program modules It may be implemented as a computer program having Codes and code segments constituting the computer program can be easily deduced by those skilled in the art of the present invention.

Such a computer program is stored in a computer readable storage medium (Computer Readable Media), read and executed by the computer, thereby implementing the embodiment of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, and the like.

In addition, terms such as "comprises", "comprises" or "have" described above mean that the corresponding component may be embedded, unless otherwise specified, excluding other components. Rather, it should be construed as being able to further include other components.

All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless otherwise defined. Terms commonly used, such as those defined in the dictionary, should be interpreted as being consistent with the meaning in the context of the related art, and are not interpreted in an ideal or excessively formal meaning unless explicitly defined in the present invention.

The methods disclosed herein include one or more acts or steps for achieving the method described above. Method acts and/or steps may be interchanged with each other without departing from the scope of the claims. In other words, unless a specific order for acts or steps is specified, the order and/or use of specific acts and/or steps may be modified without departing from the scope of the claims.

As used herein, a phrase referring to “at least one of” in a list of items refers to any combination of these items, including single members. As an example, "at least one of a, b, or c:" means a, b, c, ab, ac, bc, and abc, as well as any combination with multiples of the same element (e.g., aa , aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc or any other ordering of a, b, and c).

As used herein, the term “determining” encompasses a wide variety of operations. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (eg, looking up in a table, database, or other data structure), ascertaining, etc. . Also, “determining” may include receiving (eg, receiving information), accessing (accessing data in a memory), and the like. Also, “determining” may include resolving, choosing, choosing, establishing, and the like.

The above description is merely illustrative of the technical idea of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100... high-speed segmentation device for video sections

Claims (22)

In the video section high-speed segmentation apparatus,
at least one processor;
the at least one processor,
Determining I frames in the video,
A video section high-speed segmentation apparatus for extracting visual features for each of the determined I frames. According to claim 1,
the at least one processor,
A high-speed segmentation apparatus for segmenting a segment of the moving image based on the extracted visual features. According to claim 1,
the at least one processor,
Generate an image caption for each of the determined I frames,
Extracting language features from the generated image caption.
is a high-speed segmentation device for video sections. 4. The method of claim 3,
the at least one processor,
A high-speed segmentation apparatus for segmenting a segment of the moving image based on the extracted visual features or the extracted language features. 4. The method of claim 3,
the at least one processor,
A video section high-speed division apparatus for calculating the length of a data unit for each of the determined I frames. 6. The method of claim 5,
the at least one processor,
A high-speed segmentation apparatus for segmenting a segment of the moving image based on the extracted visual features, the extracted language features, or the calculated length of the data unit. According to claim 1,
the at least one processor,
A video section high-speed division apparatus for calculating the length of a data unit for each of the determined I frames. 8. The method of claim 7,
the at least one processor,
A video section high-speed segmentation apparatus for dividing the section of the video based on the extracted visual features or the calculated length of the data unit (date unit). In the video section high-speed segmentation apparatus,
at least one processor;
the at least one processor,
Determining I frames in the video,
Generate an image caption for each of the determined I frames,
A high-speed segmentation apparatus for video section extracting language features from the generated image caption. 10. The method of claim 9,
the at least one processor,
A high-speed segmentation apparatus for segmenting a segment of the moving image based on the extracted language features. 10. The method of claim 9,
the at least one processor,
Calculate the length of the data unit (date unit) for each of the determined I frame (frame),
A video section high-speed segmentation device for dividing the section of the video based on the extracted language features or the calculated length of the data unit (date unit). Determining I frames in a moving picture; and
An operation of extracting visual features for each of the determined I frames
A video section high-speed segmentation method comprising a. 13. The method of claim 12,
The video section high-speed segmentation method,
Segmenting a section of the video based on the extracted visual features
A video section high-speed segmentation method further comprising a. 13. The method of claim 12,
The video section high-speed segmentation method,
generating an image caption for each of the determined I frames; and
Extracting language features from the generated image caption
A video section high-speed segmentation method further comprising a. 15. The method of claim 14,
The video section high-speed segmentation method,
Segmenting a section of the video based on the extracted visual features or the extracted language features
A video section high-speed segmentation method further comprising a. 15. The method of claim 14,
The video section high-speed segmentation method,
Calculating the length of a data unit for each of the determined I frames
A video section high-speed segmentation method further comprising a. 17. The method of claim 16,
The video section high-speed segmentation method,
dividing the section of the video based on the extracted visual features, the extracted language features, or the calculated length of the data unit (date unit)
A video section high-speed segmentation method further comprising a. 13. The method of claim 12,
The video section high-speed segmentation method,
Calculating the length of a data unit for each of the determined I frames
A video section high-speed segmentation method further comprising a. 19. The method of claim 18,
The video section high-speed segmentation method,
dividing the section of the video based on the extracted visual features or the calculated length of the data unit (date unit)
A video section high-speed segmentation method further comprising a. Determining I frames in a moving picture;
generating an image caption for each of the determined I frames; and
Extracting language features from the generated image caption
A video section high-speed segmentation method comprising a. 21. The method of claim 20,
The video section high-speed segmentation method,
Segmenting a section of the video based on the extracted language features
A video section high-speed segmentation method further comprising a. 21. The method of claim 20,
The video section high-speed segmentation method,
calculating a length of a data unit for each of the determined I frames; and
dividing the section of the video based on the extracted language features or the calculated length of the data unit (date unit)
A video section high-speed segmentation method further comprising a.

Images