KR101087194B1 - Encoding System and Method of Moving Picture - Google Patents

Abstract

The disclosed video encoding system includes a scene divider and a plurality of encoding modules for receiving a video file including an audio file and a video file, extracting a video file from an encoded video file, and dividing the video file into a plurality of scenes. And an encoding unit for distributedly encoding the video file divided by the scene division unit.

Video, encoding, distribution

Description

Video encoding system and method {Encoding System and Method of Moving Picture}

The present invention relates to an image processing system, and more particularly to a video encoding system and method.

There are many kinds of services provided through communication networks, and video services are a service that demand is increasing recently. This video service started with a service using a desktop computer that can be connected to the Internet, and expanded to a service through a portable communication terminal. Recently, an IPTV (Internet Protocol Television) service, which is a two-way television service, has been introduced, which is convenient for viewers. You can select and watch a show you want to watch in time.

A video service provider such as an IPTV service collects a video from a video producer (producer) and provides the video to a user. In this case, the video file collected by the service provider may be created in various formats. Therefore, the video service provider must perform a process of changing the encoding type of the collected video file.

The process of encoding the original video, or the transcoding process of decoding the encoded video file and encoding the encoded video file again, requires a time proportional to the capacity of the video file. In other words, the larger the size of the video file, the longer it takes to encode.

If it takes a long time to encode a video, efficient service cannot be provided, and the cost increases because a higher specification system is required to improve encoding speed.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned disadvantages, and there is a technical problem to provide a video encoding system and method capable of dividing and encoding a video.

Another technical problem of the present invention is to provide a video encoding system and method that can improve the video encoding speed.

According to an aspect of the present invention, a video encoding system according to an embodiment of the present invention receives a video file including an audio file and a video file, extracts a video file from an encoding target video file, and outputs a plurality of video files. A scene divider for dividing into scenes; And an encoding unit including a plurality of encoding modules to distribute-encode the video file divided by the scene dividing unit.

Meanwhile, a video encoding method according to an embodiment of the present invention includes a video encoding method in a video encoding system including a scene divider and an encoding unit, including: extracting a video file from an encoding target video file; Dividing the video file into a plurality of scenes; And simultaneously encoding a video file divided into the plurality of scenes.

According to the present invention, after dividing one video into independent scenes, distributed encoding is performed for each divided scene. Therefore, parallel encoding is possible for each scene of the divided video, thereby greatly reducing the time required for encoding the video.

In addition, since the video file to be distributed encoded is divided into independent scenes, appropriate advertisements can be inserted between the scenes, thereby maximizing the advertisement effect.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is a view for explaining a connection relationship of a video encoding system according to the present invention.

As illustrated, the video encoding system 10 may be connected to a plurality of Content Provider (CP) servers 30-1 to 30-n through the communication network 20. In addition, the database 40 may store a video file. The video files transmitted from the CP servers 30-1 to 30-n or the video files stored in the database 40 are encoded, but each video is divided into independent scenes and distributedly encoded.

2 is a block diagram of a video encoding system according to an embodiment of the present invention.

Referring to FIG. 2, the video encoding system 10 may include a controller 110, a file manager 120, a scene divider 130, an encoder 140, a merger 150, and a storage controller for controlling overall operations. 160, and may further include a decoding unit 170 as needed.

The file management unit 120 stores the title of the video file, the video file, and the audio file for each video file collected from the CP servers 30-1 to 30-n or each video file held in the own video DB 40. Manage the overall information of the video file, etc.

The scene dividing unit 130 divides the video file of the video to be encoded into a plurality of independent scenes. In addition, the scene dividing unit 130 includes reproduction time information, that is, a reproduction time stamp, in the metadata of each divided scene so that the scene divider 130 may be referred to in the subsequent processing.

When splitting a scene, various scene splitting techniques may be used to divide each divided scene so that there is no correlation. In particular, in the present invention, a scene segmentation technique using energy of a frame, motion analysis and estimation of a camera may be used. Such a scene segmentation technique has been filed by the present applicant with the application number 10-2009-0078545, which will be described later with reference to FIGS.

The encoding unit 140 performs distributed encoding on each scene of the video divided by the scene dividing unit 130 and the audio file of the corresponding video. To this end, the encoding unit 140 may include a plurality of encoding modules. In addition, each segmented video after encoding is also stored in the play time information, that is, the time stamp as metadata so that it can be referred to in subsequent processing.

In an embodiment of the present disclosure, the number of scenes divided by the scene dividing unit 130 may be appropriately adjusted in consideration of the number of encoding modules and processing power. That is, as a result of dividing a single video into a plurality of independent scenes, when the number of divided scenes is larger than the number of encoding modules, the number of divided scenes and the encoding module may be divided by merging adjacent scenes. Can match the numbers. Alternatively, when the number of scenes divided by the scene dividing unit 130 is greater than the number of encoding modules, a method of performing first-order distributed encoding on each scene by the number of encoding modules and performing second-order distributed encoding on the remaining scenes may also be applied. have.

In the present invention, a plurality of encoding modules constituting the encoding unit 140 performs encoding independently without mutual communication. Accordingly, a plurality of encoding modules may be configured using a plurality of independent computing devices, or a plurality of encoding modules may be configured using a parallel system such as a multi-core.

The merger 150 collects the plurality of scenes encoded in the encoding unit 140 and the encoded audio file, generates a single video file, and stores the plurality of scenes in the storage unit 160. At this time, one completed video file is generated with reference to the time stamp included in the metadata of each divided scene.

Meanwhile, when the video file to be distributedly encoded is received in the encoded state, the decoding unit 170 performs a function of decoding the corresponding video in advance.

When decoding a video for performing distributed encoding, the decoding unit 170 performs scene link information on each frame, that is, a color feature, a shape feature, and a motion feature. ) Can be filtered and managed as cache data. When the scene divided by the scene dividing unit 130 is transmitted to the encoding unit 140, the control unit 110 refers to cache data including the scene link information, and encodes a scene having similar scene link information in the same encoding module. Can be sent to. In this case, since encoding for similar scenes is performed in the same encoding module, encoding speed can be improved. That is, in the case of similar scenes, since color and motion information are easily predicted, encoding may be performed by the same encoding module to increase processing speed.

As described above, in the present invention, one video file is divided into a plurality of scenes that are independent of each other. In addition, by encoding the divided scenes in parallel, the time required for encoding can be greatly reduced. In addition, since the divided scenes are independent of each other, that is, there is no correlation between the previous scene and the subsequent scene, the image quality of the video can be maintained at the same level as the original even when distributed encoding is decoded and played back.

In another embodiment of the present invention, when the encoding unit 140 performs distributed encoding, an encoding error for a specific scene may occur. If an encoding error occurs, the encoding unit 140 reports this to the control unit 110, and the control unit 110 re-encodes only the scene in which the error occurs. That is, since each scene is independently encoded, even if an encoding error occurs for a specific scene, only the scene is re-encoded, and there is no need to re-encode a scene in which no encoding error occurs. Thus, the time required for encoding error recovery and re-encoding can be greatly reduced.

Next, referring to FIGS. 3 to 5, a scene segmentation technique applied to the present invention will be described.

3 is a configuration diagram illustrating the scene divider shown in FIG. 2.

As shown in FIG. 3, the scene divider 130 includes a frame extractor 132, a coefficient filter 134, a motion analyzer 136, and a split position determiner 138.

The frame constituting the video may be divided into an intra frame (I-frame), a predictive frame (P-frame), and a bidirectional frame (B-frame). An I-frame is an independent frame that has little or no correlation with previous or subsequent images, and the I-frame can know spatial correlation. The P-frame means a frame that depends on the previous picture, the B-frame means a frame that depends on the previous picture and the after picture, and the correlation in time can be known as the P-frame or the B-frame. However, the B-frame is a frame predicted from past and future frames with respect to the moving time of the video, and may not be considered because the camera movement cannot be found over time.

Therefore, the I-frame and the P-frame are separated from the segmentation target video by using the frame extractor 132.

In general, P-frames are highly dependent on I-frames, but in some cases there may be frames that are less dependent on I-frames. Since the P-frame may be treated as an independent scene, the coefficient filtering unit 134 may have an energy difference, that is, an alternating current (AC) coefficient difference from a neighboring frame among the P-frames, higher than a preset threshold or direct current (DC). Current) Selects a frame having a higher coefficient difference than a preset threshold as a meaningful scene. In addition, I-frames having a high correlation with neighboring I-frames may exist among the I-frames, and the coefficient filtering unit 134 may set a threshold value in which an energy difference, that is, an AC coefficient difference, is preset among the I-frames. The frame that is higher or the DC coefficient difference is higher than the preset threshold is selected as a meaningful scene. That is, I-frames having high correlation between neighboring I-frames are treated as one scene.

To this end, the frame extractor 132 includes an I-frame extraction module 1321 and a P-frame extraction module 1323 as shown in FIG. 4. Meanwhile, the coefficient filtering unit 134 includes a comparison module 1341 and a selection module 1343.

Frames are extracted by the I-frame extraction module 1321 and the P-frame extraction module 1323 of the frame extraction unit 132, respectively, for the split target frame. In addition, the comparison module 1341 compares the difference between the AC coefficient and the DC coefficient between neighboring I-frames with the preset threshold, respectively, and compares the difference between the AC coefficient and the DC coefficient between the P-frames with the preset threshold, respectively. do. In addition, the selection module 1343 selects a frame estimated as a meaningful scene change position, that is, a frame having a DC coefficient difference or an AC coefficient difference higher than a threshold value, as the scheduled division position, according to the comparison result of the comparison module 1341.

However, since it is not possible to accurately determine whether to change the scene only by comparing the energy, the motion analyzer 136 checks whether the background or the foreground of the frame has been changed from the previous frame with respect to the frames determined as the scheduled division positions.

That is, the motion analyzer 136 refers to the frames selected as the split positions in the selection module 1343 of the coefficient filtering unit 134 and determines temporal / spatial continuity through comparison between neighboring frames. Temporal / spatial continuity is achieved by analyzing global motion or local motion of a frame. For this purpose, the motion analyzer 136 performs a global motion analysis module 1361 as shown in FIG. 5. ) And local motion analysis module 1363.

The global motion analysis module 1361 determines whether the background image has changed when the corresponding frame is compared with the previous frame through the camera motion analysis. That is, it is determined whether two-dimensional motion (temporal continuity) by a shooting technique such as panning, yawing, zooming, or three-dimensional motion (spatial continuity) by spatial movement of the camera. To this end, the global motion analysis module 1361 analyzes a distribution pattern of residual coefficients, that is, AC coefficients, for each frame at the position to be divided, from which the difference in energy between the frames is large, but the energy due to a simple imaging technique. It can be determined whether the change or the energy change due to the actual scene change.

The local motion analysis module 1363 checks whether the foreground image, that is, the video object, has changed when the frame is compared with the previous frame. This is to check whether the person or the person's motion has changed. After separating the video object for each frame at the position to be divided, analyzing the distribution pattern of the surplus coefficients, or AC coefficients, between neighboring frames, the temporal continuity and the spatial continuity are analyzed. Judge.

Various feature extraction algorithms can be used for global motion analysis and local motion analysis, for example, principal component analysis for motion vectors, independent component analysis for surplus coefficients, and matching tracking analysis. Matching Pursuit Analysis and the like can be used. The principal component analysis of the motion vector can determine the two-dimensional movement of the camera (background) or video object (foreground), and the independent component analysis of the surplus coefficients determines the three-dimensional movement of the camera or video object. can do. In addition, in the case of a matching tracking analysis method, there is an advantage that a temporal feature and a spatial feature can be simultaneously extracted.

Meanwhile, the split position determiner 138 divides the video by determining a frame having a low temporal / spatial continuity as the split position according to the analysis result of the motion analyzer 136, and stores the split video in the storage 160. do.

As described above, the scene dividing unit 130 applied to the present invention extracts a frame having a large energy difference from a neighboring frame among the frames constituting the divided target moving picture to determine a dividing scheduled position. At this time, the accuracy of scene division is improved by considering not only I-frames independent of neighboring frames but also P-frames.

Then, temporal / spatial continuity is determined from the frames before and after the segmentation scheduled position through motion analysis. As a result, by extracting a frame having low temporal / spatial continuity, and determining a split position therefrom, it is possible to accurately distinguish between frames in which a scene change is made so that the divided scenes can be mutually independent.

Accordingly, in the present invention, the "scene" finally divided from the video may be defined as a scene divided based on the temporal / spatial continuity of the camera or video object according to the playback time of the video.

After the final division of the scene of the moving image as described above, the division position determiner 138 may include the reproduction time information, that is, the time stamp, in the metadata of each divided scene.

Meanwhile, the encoding method of the video encoding system 10 described above will be described below.

6 is a flowchart illustrating a video encoding method according to an embodiment of the present invention.

The video encoding system 10 receives an encoding target video file from a CP server or a database held by itself (S10).

Then, the scene dividing unit 130 extracts an audio file and a video file from the received video file (S20), and divides the extracted video file into a plurality of independent scenes (S30). At this time, the reproduction time information, that is, the time stamp, is recorded in the metadata of each divided scene so that it can be referred to in subsequent processes.

Next, the encoding unit 140 distributedly encodes each scene divided in step S30, and encodes the audio file extracted in step S30 (S40). That is, encoding is performed at a high speed by simultaneously encoding video files divided into scene units in a plurality of encoding modules.

When the encoding is completed, the merger 150 merges the audio file encoded in step S40 and the plurality of independently encoded video files to generate one encoded video file (S50). In this case, a plurality of video files encoded respectively may be merged according to a playback order by using time stamp information included in metadata in a scene segmentation process.

FIG. 7 is a flowchart for describing a scene division process illustrated in FIG. 6.

First, the frame extractor 132 of the scene divider 130 extracts I-frames and P-frames with respect to the video file of the corresponding video, and the coefficient filtering unit 134 performs energy difference and threshold values between the frames. The division scheduled position is determined by comparing (S310). In a preferred embodiment of the present invention, the coefficient filtering unit 134 calculates an AC coefficient and a DC coefficient of the I-frame, an AC coefficient and a DC coefficient of the P-frame, and a difference between the AC coefficient and the DC coefficient with the neighboring frame, respectively. An I-frame larger than a threshold of P, and a P-frame having a difference in AC coefficient and DC coefficient with a neighboring frame larger than each threshold may be selected as a division scheduled position.

Thereafter, the motion analyzer 136 determines the temporal / spatial continuity by comparing frames before and after the split scheduled position with reference to the split scheduled position determined in step S310 (S320). That is, the feature is extracted from the frames before and after the segmentation scheduled position and compared to determine whether there is a two-dimensional or three-dimensional change.

Then, according to the determination result of step S320, the split position determiner 138 determines a frame having a low temporal / spatial continuity as the actual scene change position of the video (S330), and divides the video at the corresponding position (S340).

Referring to FIG. 8, the process of determining the scheduled dividing position shown in FIG. 7 will be described. First, the I-frame extraction module 1321 and the P-frame extraction module 1323 of the frame extractor 132 are included in the corresponding video. The extracted I / P frames are extracted (S311).

The comparison module 1341 of the coefficient filtering unit 134 calculates the energy of each frame, that is, the AC coefficient and the DC coefficient (S313), and compares the energy difference between the frames with a preset threshold (S315).

Thereafter, the selection module 1343 determines a frame in which the energy difference value between the frames of the comparison result of the step S315 is larger than the threshold value as the scheduled division position (S317).

After determining the division scheduled position as described above, a motion analysis process is performed as shown in FIG. 9.

That is, in the global motion analysis module 1361 of the motion analysis unit 136, by analyzing the movement of the camera through the comparison between the frames before and after the segmentation scheduled position (S321), whether the scene change is made by the movement of the camera, that is, It is determined whether the motion is global (S323).

When it is determined that the motion is global, the background feature of the corresponding frame is extracted (S325), and the temporal / spatial continuity is determined by comparing the feature with the neighboring frame (S329). That is, it is determined whether two-dimensional motion (temporal continuity) by a shooting technique such as panning, yawing, zooming, or three-dimensional motion (spatial continuity) by spatial movement of the camera. To this end, the global motion analysis module 1361 analyzes a distribution pattern of residual coefficients, that is, AC coefficients, for each frame at a predetermined position to be divided. It is determined whether it is energy change or energy change due to actual scene change.

In operation S330, the neighboring frame and the frame having low temporal / spatial continuity are determined as the split positions.

On the other hand, when the determination result of step S323 is not the global motion, that is, in the case of local motion, the local motion analysis module 1363 separates the video object from the frame, and then the distribution pattern of the surplus coefficient, that is, AC coefficient, between neighboring frames is determined. Analyze (S327), and determine the temporal and spatial continuity (S329). Similarly, when temporal / spatial continuity with respect to the foreground feature is low, the corresponding frame is determined as the split position (S330).

As described above, the present invention selects a preliminary division position based on an energy difference value between neighboring frames. The temporal / spatial continuity is determined by comparing the frames before and after the segmentation scheduled position, and a frame having low continuity is determined as the segmentation position.

Therefore, the part where the scene change is performed on the contents of the moving picture can be easily detected at high speed, and the accuracy and speed of scene division can be improved.

In the above, the scene segmentation technique using the energy of the frame, the motion analysis and the estimation of the camera motion has been described. However, the method of segmenting the scene of the video in order to distribute-encode the video is not limited thereto. That is, in addition to the above-described method, any method may be applied as long as it is a scene segmentation technique capable of segmenting each scene of the video so that there is no correlation.

Regardless of the technique, the video file of the moving picture is divided into a plurality of independent scenes, and then parallel encoding is performed on the divided video files, thereby greatly reducing the time required for encoding.

Those skilled in the art to which the present invention described above belongs will understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. Therefore, the above-described embodiments are to be understood as illustrative in all respects and not as restrictive. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

1 is a view for explaining a connection relationship of a video encoding system according to the present invention;

2 is a block diagram of a video encoding system according to an embodiment of the present invention;

3 is a configuration diagram of a scene divider shown in FIG. 2;

4 is a block diagram of a frame extractor and a coefficient filter shown in FIG. 3;

5 is a configuration diagram of a motion analyzer shown in FIG. 3;

6 is a flowchart illustrating a video encoding method according to an embodiment of the present invention;

7 is a flowchart for explaining a scene division process shown in FIG. 6;

FIG. 8 is a flowchart for explaining a division scheduled position determination process shown in FIG. 7;

9 is a flowchart for explaining a motion analysis process illustrated in FIG. 7.

Description of the Related Art [0002]

10: video encoding system 110: control unit

120: file management unit 130: scene divider

140: encoding unit 150: merge unit

160: storage unit 170: decoding unit

Claims (25)

A scene dividing unit which receives a video file including an audio file and a video file, extracts a video file from an encoding target video file, and divides the video file into a plurality of scenes; And An encoding unit having a plurality of encoding modules and distributed encoding the video file divided by the scene dividing unit; Including, The scene dividing unit may include: a frame extracting unit extracting an intra frame (I-frame) and a predictive frame (P-frame) constituting a segmentation target video; Computing the energy of each of the I-frame and P-frame, and compares the energy difference with the neighboring homogeneous frame with a predetermined threshold and the comparison module of the comparison module as a result of comparing the frame with the energy greater than the threshold A coefficient filtering unit including a selection module that determines a division scheduled position; A motion analysis unit for determining the continuity between the frames before and after the split scheduled position determined by the coefficient filtering unit; And A split position determiner configured to determine a scene split position of the video based on the continuity determined by the motion analyzer; Video encoding system comprising a. The method of claim 1, The scene dividing unit divides the plurality of scenes to have a mutually independent relationship. The method of claim 2, And the scene dividing unit includes reproduction time information in metadata of each divided scene. The method of claim 1, And the scene dividing unit extracts an audio file from the encoding target moving image file, and the encoding unit further comprises an encoding module for encoding the extracted audio file. The method of claim 1, And a merging unit for merging the video file and the audio file encoded by the encoding unit. The method of claim 1, The scene dividing unit determines a number of scenes to be divided by referring to the number of encoding modules. delete delete delete The method of claim 1, The comparison module calculates at least one of an alternating current (AC) coefficient and a direct current (DC) coefficient from the I-frame, and compares the AC coefficient difference and the DC coefficient difference with adjacent I-frames with respective thresholds. Video encoding system, characterized in that. The method of claim 1, The comparison module calculates at least one of an alternating current (AC) coefficient and a direct current (DC) coefficient from the P-frame, and compares an AC coefficient difference and a DC coefficient difference with an adjacent P-frame with respective thresholds. Video encoding system, characterized in that. The method of claim 1, And the global motion analysis module including a global motion analysis module that analyzes the motion of the camera for each of frames before and after the split scheduled position determined by the coefficient filtering unit to determine whether the background image is changed temporally or spatially. 13. The method of claim 12, The motion analysis unit further comprises a local motion analysis module for analyzing whether the foreground image is changed temporally or spatially by analyzing the change of the video object with respect to each of the frames before and after the split scheduled position determined by the coefficient filtering unit. The method of claim 1, And the encoding unit encodes a plurality of scenes divided by the scene dividing unit in the same encoding module in the case of similar scenes by referring to similarities of scenes. The method of claim 14, The similarity of the scene is determined by referring to the color characteristics, shape characteristics and motion characteristics of each scene. A video encoding method in a video encoding system including a scene splitter including a frame extractor, a coefficient filter, a motion analyzer, and a split position determiner, and an encoder, Extracting a video file from an encoding target video file by the scene divider; Dividing the video file into a plurality of scenes by the scene dividing unit; And Simultaneously encoding, by the encoding unit, a video file divided into the plurality of scenes; Including, The process of dividing the video file into a plurality of scenes by the scene dividing unit may include extracting an intra frame (I-frame) and a predictive frame (P-frame) from the video to be divided by the frame extractor. ; For each of the extracted frames, determining, by the coefficient filtering unit, a predetermined scheduled position by comparing an energy difference value and a threshold value between neighboring frames; Determining, by the motion analyzer, temporal / spatial continuity between frames before and after the segmentation scheduled position with reference to the segmentation scheduled position; And And determining, by the division position determiner, a frame having a low temporal / spatial continuity as a division position. The determining of the scheduled division position includes: calculating at least one of an alternating current (AC) coefficient and a direct current (DC) coefficient from the I-frame; Comparing the AC coefficient difference and the DC coefficient difference with neighboring I-frames with respective thresholds; Determining an I-frame in which the AC coefficient difference value and the DC coefficient difference value are larger than the respective threshold values as the scheduled division positions; Calculating at least one of an Alternate Current (AC) coefficient and a Direct Current (DC) coefficient from the P-frame; Comparing the AC coefficient difference value and the DC coefficient difference value between neighboring P-frames with respective threshold values; And Determining a P-frame in which the AC coefficient difference value and the DC coefficient difference value are larger than the respective threshold values as the scheduled division positions; Video encoding method comprising a. The method of claim 16, And dividing the video file into a plurality of scenes comprises splitting the divided scenes to have independent relationships with each other. The method of claim 16, The extracting of the video file further includes extracting an audio file from the encoding target video file, and the encoding process further comprises encoding the audio file. The method of claim 18, And after the encoding, merging the encoded video file and the audio file. delete delete delete The method of claim 16, The determining of the temporal / spatial continuity may include: determining whether a background change has been made by analyzing the movement of the camera; And Determining temporal / spatial continuity by comparing features with neighboring frames when a background change is made as a result of the determination; Video encoding method comprising a. The method of claim 23, wherein Determining whether a foreground change is made when the background change is not performed as a result of the determination; And Determining temporal / spatial continuity by comparing features with neighboring frames when the foreground change is made; Video encoding method further comprising. The method of claim 16, After the determining of the dividing position, dividing the scene of the video according to the dividing position by the dividing position determining unit; And Storing division order information in metadata of the divided scenes by the division position determiner; Video encoding method further comprising.

Images