KR20230053229A - Device and Method for Performing Distributed Parallel-Transcoding - Google Patents

Abstract

Disclosed is a distributed parallel transcoding method and device. According to one aspect of the present invention, a computer-implemented method for parallel transcoding video content containing key frames at given frame intervals, comprising: a process of dividing the playback time of video content into a plurality of time sections; a process of adjusting at least one of a start point or an end point of one of the plurality of time sections based on the position of the key frame; a process of decoding a video content part corresponding to the adjusted time interval using one transcoder among a plurality of transcoders; and a process of encoding the decoded video content part in the range of the one time interval using the one transcoder. Accordingly, the present invention divides video content into a plurality of data chunks and assigns them to a plurality of transcoders to transcode the divided video content in parallel, thereby reducing the time required for transcoding.

Description

Distributed parallel transcoding method and apparatus {Device and Method for Performing Distributed Parallel-Transcoding}

Embodiments of the present invention relate to a distributed transcoding method and apparatus.

The information described in this section simply provides background information on the present invention and does not constitute prior art.

Recently, media services such as video on demand (VOD) systems and real-time streaming services are expanding.

In a system that provides media services, a server compresses media content according to a user's request and provides the compressed media content to a user terminal. The user terminal restores the original media content from the compressed media content.

In general, a compression algorithm such as MPEG2 or H.264 is applied to media content by placing a separate encoding server in addition to a server that manages media content. In particular, since encoding and decoding of video content among media content consumes a lot of computational resources, one media content has conventionally been allocated and encoded to a single dedicated server with high performance.

However, there is a limit to providing a high-definition VOD service or real-time streaming service in a system in which a dedicated server performing transcoding and content are mapped 1:1. That is, in the conventional system, there is a problem in that a smooth service cannot be provided due to an increase in the time required to perform encoding and decoding.

Furthermore, demand for media services of consistent quality is increasing among users in various environments. In order to meet these demands, it is necessary to compress original media contents in consideration of each user's environment. That is, it is necessary to transcode media content in consideration of various environments such as performance of a user terminal, display information, network conditions, and service scenarios.

Here, transcoding means converting original media content into a different format to suit a user's environment or changing the resolution, frame rate, bitrate, etc. of original media content. That is, transcoding refers to a process of converting video data and audio data created by a producer into a form that a user can view.

As such, a system in which a dedicated server performing transcoding and contents are mapped 1:1 has limitations in providing various environments for users and high-quality media services.

Embodiments of the present invention are mainly aimed at providing a method and apparatus for transcoding the divided video contents in parallel by dividing the video contents into a plurality of data chunks and assigning them to a plurality of transcoders. .

Another embodiment of the present invention is a method for preventing frame loss in a decoding process when a frame interval, which is an interval between key frames in video content, and a time interval divided from a playback time of video content do not match, and One object is to provide a device.

According to one aspect of the present invention, in a computer implemented method for parallel transcoding video contents including key frames at given frame intervals, a process of dividing a reproduction time of video contents into a plurality of time intervals, the key frames A process of adjusting at least one of the starting point or the ending point of one of the plurality of time intervals based on the position of, and corresponding to the adjusted time interval using one transcoder among a plurality of transcoders. decoding a video content part of the video content, and encoding the decoded video content part in a range of the one time interval using the one transcoder.

According to another aspect of the present embodiment, a memory for storing instructions, and at least one processor, wherein the at least one processor executes the instructions to reduce the playback time of video content including key frames at given frame intervals. Dividing into a plurality of time intervals, adjusting at least one of a start point or an end point of one of the plurality of time intervals based on the position of the key frame, and using one transcoder among a plurality of transcoders. An apparatus for decoding a video content part corresponding to the adjusted time interval by using a coder and encoding the decoded video content part in a range of the one time interval by using the one transcoder.

As described above, according to an embodiment of the present invention, video content is divided into a plurality of data chunks and allocated to a plurality of transcoders to transcode the divided video contents in parallel, thereby reducing the time required for transcoding. can be reduced

According to another embodiment of the present invention, it is possible to provide an optimal media service to a user by transcoding video content in consideration of various environments, such as a user's video playback environment and a network condition between a user terminal and a server.

According to another embodiment of the present invention, when a frame interval, which is an interval between key frames in video content, and a time interval divided from the reproduction time of video content do not match, frame loss in the decoding process can be prevented from occurring. .

1 is a diagram showing a transcoding process according to an embodiment of the present invention.
2 is a diagram showing the configuration of a transcoding system according to an embodiment of the present invention.
3 is a block diagram showing a distributed parallel transcoding system according to an embodiment of the present invention.
4 is a flowchart illustrating a parallel transcoding process according to an embodiment of the present invention.
5 is a flowchart illustrating an operation process of an analyzer according to an embodiment of the present invention.
6 is a flowchart illustrating an operation process of a transcoding unit according to an embodiment of the present invention.
7 is a flowchart illustrating an operation process of a merging unit according to an embodiment of the present invention.
8 is a flowchart illustrating an operation process of a control unit according to an embodiment of the present invention.
9 is a diagram illustrating a frame-by-frame configuration of video content.
10 is a diagram showing the configuration of frame groups of video content.
11A and 11B are diagrams illustrating transcoding sections of frames.
12 is a diagram showing transcoding sections of frames according to an embodiment of the present invention.
13 is a flowchart illustrating a parallel transcoding process according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

Also, terms such as first, second, A, B, (a), and (b) may be used in describing the components of the present invention. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. Throughout the specification, when a part 'includes' or 'includes' a certain component, it means that it may further include other components without excluding other components unless otherwise stated. . In addition, terms such as '~unit' and 'module' described in the specification refer to a unit that processes at least one function or operation, and may be implemented by hardware, software, or a combination of hardware and software.

1 is a diagram showing a transcoding process according to an embodiment of the present invention.

Referring to FIG. 1 , a de-multiplexer 100, a video transcoder 110, an audio transcoder 120, and a multiplexer 130 are shown. Components shown in FIG. 1 may be included in a cloud server, which is a cloud computing environment.

The input image is data including video content and audio content, and is received in an encoded state. The input video is input to the demultiplexer 100 and divided into video content and audio content. Video content is input to the video transcoder (110) and audio content is input to the audio transcoder (120).

Since the input video content is in an encoded state, the video transcoder 110 decodes the encoded video content using a decoder. The video transcoder 110 sets encoding parameters for the decoded video content in consideration of the user terminal's environment and network conditions. For example, the video transcoder 110 converts the codec of the input image into a different type of codec or converts the resolution and frame rate of the input image in consideration of the performance of the user terminal, the display environment, the network state, and the service scenario. rate), bitrate, file format, media container, etc., to set encoding parameters. Here, a frame is an image constituting video content or a representation of an image, and may be used interchangeably with a picture. The video transcoder 110 encodes the decoded video content using the set encoding parameters and encoder.

Like video content, audio content is decoded by audio transcoder 120 and then re-encoded.

The encoded video content and the encoded audio content are input to the multiplexer 130 and merged into one output image.

The user terminal does not receive the input video input to the server as it is, but receives the transcoded output video in consideration of the environment and network conditions of the user terminal. Through this, the user can watch the output video transcoded to be suitable for the user terminal.

2 is a diagram showing the configuration of a transcoding system according to an embodiment of the present invention.

Referring to FIG. 2 , a server 200, video content 202, server video codec 204, audio content 206, server audio codec 208, container 210, user terminal 220, terminal video A codec 212 , a terminal audio codec 214 , and an output image 230 are shown.

Video content 202 is video data captured by a camera, and audio content 2206 is audio data collected by a microphone. In the following, media content is referred to as a composition that includes both video content 202 and audio content 206 .

The server 200 may store the video content 202 in advance or receive it in real time through a network. Server 200 decodes and encodes video content 202 using video codec 204 . The server 200 compresses and encodes the video content 202 .

Here, as an example of the type of codec, the codec format used to compress the video content 202 is MPEG-1, MPEG-2, MPEG-4, and MPEG-7 standardized by the Moving Picture Experts Group (MPEG). , MPEG-21, etc. This codec format compresses video content 202 by removing overlapping pixels in two-dimensional space. To this end, a discrete cosine transform (DCT) is applied that transforms an image signal on a time axis into a frequency axis. In addition, a compression method using correlation between frames as well as frame-by-frame compression is applied. In this case, a motion compensation method may be further applied to eliminate duplication on the time axis of video sequences. The motion compensation method means performing bi-directional prediction using two frame memories. In addition, compression is performed using a Group of Pictures (GOP) structure to enable random access.

The server 200 may store the audio content 206 in advance or receive it in real time through a network. Server 200 decodes and encodes audio content 206 using audio codec 208 . The server 200 compresses and encodes the audio content 206 .

Here, the audio codec may use Modified Discrete Cosine Transform (MDCT) for transcoding of audio content. At this time, in order to normally decode the first pulse-code modulation (PCM) sample, a bundled sample of a certain length may be added and encoded. That is, audio content may be encoded in association with video content.

The server 200 creates the container 210 by combining the encoded video content and the encoded audio content.

Container 210 contains content information and details about video content 202 or audio content 206 . Content information includes video data of video content 202 or audio data of audio content 206 . Detailed information includes codec information, bit rate, frame rate, audio track, audio channel information, recording date, recording location, number of streams, length of playback time of media content, stream location information, or other metadata. includes

The container 210 includes encoded media content in the form of a stream and is transmitted from the server 200 to the user terminal 220 .

User terminal 220 receives container 210 and decodes the encoded media content using terminal video codec 222 and terminal audio codec 224 . The user terminal 220 obtains decoded video content and decoded audio content through decoding. The user terminal 220 generates an output image 230 that the user can watch from the decoded video content and the decoded audio content, and provides the output image 230 to the user.

On the other hand, when the server 200 is a single server, it takes a lot of time for the server 200 to transcode the video content 202 and the audio content 206. In particular, the server 200 consumes a lot of time to process high-resolution content and video technologies such as high dynamic range (HDR).

According to an embodiment of the present invention, a codec requiring a large amount of computation is applied to video content or high-quality video content is divided and allocated to a plurality of transcoders to perform parallel transcoding, thereby reducing the cost required for transcoding. can save time

3 is a block diagram showing a distributed parallel transcoding system according to an embodiment of the present invention.

Referring to FIG. 3 , a transcoding system 30 , a storage unit 310 , a control unit 320 , an analysis unit 330 , a transcoding unit 340 and a merging unit 350 are illustrated.

The transcoding system 30 may include a hardware device dedicated to parallel processing for a plurality of transcoders, or a plurality of software-based encoder devices.

Components within the transcoding system 30 may include an independent memory and at least one processor. That is, the components within transcoding system 30 may be separate computing devices.

Components in the transcoding system 30 are connected through a network. Components in the transcoding system 30 transmit or receive media data, detailed information about the media data, or control signals in the form of Internet Protocol (IP) packets. To this end, each component may include a communication module.

Meanwhile, the user terminal receives encoded video content from the transcoding system 30 . A user terminal may be an electronic device having a decoder. For example, the user terminal includes a smart phone, a mobile phone, a navigation device, a computer, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet PC, a game console, It may correspond to a wearable device, an internet of things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, and the like.

Here, the media data includes video content and may further include audio content. Hereinafter, video content will be described, but audio content may be processed the same as video content.

Also, detailed information on media data includes detailed information on video content or detailed information on audio content. Detailed video information includes offset information of video content, video codec information, video bit rate, video frame rate (Frame Per Second, FPS), GOP information, total playback time, resolution, aspect ratio, average bit rate, number of streams , stream location information, or metadata. Detailed information about audio content includes audio codec information, audio track information, audio channel information, audio bit rate, total playback time, offset information of audio content, sampling rate, language information or meta data. The control signal is a signal that the control unit 320 transmits to each component for controlling the transcoding system 30 .

Video content received by transcoding system 30 may be in an encoded state.

Hereinafter, video content may be divided into a plurality of data chunks, and division of video content means logical division, not physical division. Specifically, the transcoding system 30 divides the reproduction time of video content into a plurality of time intervals, and stores the start point and end point of each time interval. The divided time intervals may have the same size. Components in the transcoding system 30 may load a part of video content corresponding to a time interval using the start point and end point of each time interval, instead of loading all of the video content. In this way, setting offset information of each data chunk for video content is called logical division, and data divided by time interval is called a data chunk.

The storage unit 310 is configured to store video content and information necessary for parallel transcoding of the video content.

Specifically, the storage unit 310 stores video content, detailed information of video content, offset information of video content, or encoded video content. Also, the storage unit 310 may store audio content, detailed information of audio content, offset information of audio content, or encoded audio content.

When the video content is uploaded, the storage unit 310 may notify the control unit 320 of completion of the upload of the video content or an event according to the completion of the upload.

The control unit 320 is configured to generate a control signal for parallel transcoding and transmit the control signal to each component.

When receiving a transcoding request for video content, the control unit 320 transmits a video content analysis request to the analyzer 330 . At this time, the analysis unit 330 generates offset information that is a boundary value of a time interval for dividing the video content into a plurality of data chunks according to time.

The control unit 320 sets transcoding parameters according to the analysis result, and transmits the transcoding parameters and offset information to the transcoding unit 9340. Here, the transcoding parameters include decoding parameters and encoding parameters. Decoding parameters refer to parameters necessary for decoding video content among information included in detailed information. Encoding parameters refer to parameters necessary for encoding video content among information included in detailed information. For example, encoding parameters include video codec information, video bit rate, video frame rate (Frame Per Second, FPS), average bit rate, number of streams, location information of streams, and the like.

According to an embodiment of the present invention, transcoding parameters are set in consideration of at least one of a playback environment of a user terminal and a network condition. To this end, a configuration for receiving information about a reproduction environment of a user terminal or a configuration for measuring a network state may be further used.

According to another embodiment of the present invention, the transcoding parameters include a time interval between key frames in the transcoded video content. Each transcoder generates a key frame by considering a time interval between key frames in transcoded video content during an encoding process. Key frames serve as reference points for playback such as video playback, reverse playback, and skipping.

Meanwhile, video content is divided into a plurality of data chunks according to offset information and transcoded for each data chunk.

Then, the control unit 320 requests the merging unit 350 to merge the transcoded data chunks. The control unit 320 stores the transcoded video content in the storage unit 310 .

The analyzer 330 is configured to load the video content stored in the storage 310 and obtain detailed information and offset information about the video content.

Here, the offset information represents information about a start point and an end point of each time interval when the reproduction time of video content is divided into a plurality of time intervals. That is, the offset information is a boundary value of a plurality of data chunks divided from video content. For example, the offset information may indicate a start point and an end point of each time interval when video content having a reproduction time of 30 seconds is divided into time intervals in units of 0.5 ms.

According to an embodiment of the present invention, the analyzer 330 decodes the video content to determine whether decoding is possible, and compares detailed information of the video content with a decoding result of the video content to determine an error in the video content. It is explained in detail in FIG. 4 .

The transcoder 340 generates a plurality of video transcoders and allocates a plurality of data chunks divided from video content to the transcoders to perform parallel transcoding. Here, a plurality of video transcoders may be composed of independent processors.

Specifically, the transcoder 340 generates as many video transcoders as the number of divided data chunks according to the playback time of video content and a given time interval. The transcoding unit 340 receives transcoding parameters and offset information, and allocates the transcoding parameters and offset information to a plurality of video transcoders.

Each video transcoder loads a data chunk for a time interval corresponding to offset information among video contents based on the allocated offset information. Each video transcoder decodes a chunk of data based on the details of the video content. Then, each video transcoder encodes the decoded data chunk using the transcoding parameters.

According to an embodiment of the present invention, the transcoding unit 340 evaluates the quality of the transcoded data chunk, adjusts transcoding parameters for the decoded data chunk according to the quality of the transcoded data chunk, and encodes again. can do. In particular, encoding may be re-encoded by adjusting encoding parameters. This is explained in detail in FIG. 6 .

According to another embodiment of the present invention, the transcoding parameters include a time interval between key frames in the transcoded video content. That is, each video transcoder performs encoding so that finally transcoded video content includes key frames at regular intervals. Key frames at regular intervals allow the user to skip while watching media content, play at double speed, and the like.

Meanwhile, the transcoder 340 may transcode audio content using a plurality of audio transcoders. Since transcoding of the audio content requires a small amount of computation, the transcoding unit 340 may transcode the audio content as a whole or perform distributed parallel transcoding.

The transcoding unit 340 stores a plurality of transcoded data chunks in the storage unit 310 .

The merging unit 350 is configured to generate transcoded video content by merging the plurality of transcoded data chunks. The merging unit 350 determines the order of the plurality of transcoded data chunks based on the offset information, and merges the plurality of transcoded data chunks in order. At this time, a key frame is located at the start position of each transcoded data chunk.

According to an embodiment of the present invention, the merging unit 350 may verify the integrity of the transcoded video content by comparing the length information of the transcoded video content with the length information of the transcoded audio content. That is, when the two pieces of length information do not match, the merging unit 350 may detect frame loss or duplication. This is explained in detail in FIG. 7 .

4 is a flowchart illustrating a parallel transcoding process according to an embodiment of the present invention.

Referring to FIG. 4 , the storage unit receives video content (S400).

The storage unit may store input video content in advance or receive input video content in real time.

When the storage of the video content is completed, the storage unit notifies the control unit of uploading the video content (S402).

The storage unit transmits identification information (ID) of the uploaded video content to the control unit along with an upload notification. In addition, the storage unit may generate a transcoding request event and transmit it to the control unit when the storage of the video content is completed.

The control unit requests the analysis unit to analyze the video contents according to the upload notification of the video contents (S404).

The controller transmits the video content ID and analysis request to the analyzer.

The analysis unit loads the video contents from the storage unit into the local storage using the ID of the video contents (S406).

The analysis unit may identify the location of the video content in the storage unit using uniform resource locator (URL) information including an ID of the video content. The analysis unit loads video content in an encoded state into a local storage.

The analysis unit performs error checking on the loaded video content and divides the video content into a plurality of data chunks (S408).

The analysis unit may divide the video content into a plurality of data chunks by dividing the reproduction time of the video content into predetermined time intervals. The analysis unit stores the starting point and ending point for each time interval as offset information of the corresponding data chunk. Also, the analysis unit may extract detailed information of video content.

The analysis unit notifies the control unit of completion of analysis of the video contents together with detailed information of the video contents, error check results, and offset information (S408).

The control unit generates identification information (ID) of transcoders according to the number of data chunks (S412).

The control unit requests transcoding to the transcoding unit (S414).

The control unit transmits offset information and transcoding parameters for a plurality of data chunks to the transcoding unit.

The transcoding unit loads data chunks from the storage unit according to the transcoding request (S416).

The transcoding unit generates transcoders according to the number of the plurality of data chunks, allocates computing resources to the transcoders, and transcodes the plurality of data chunks in parallel using the transcoders, offset information, and transcoding parameters ( S418). Each transcoder transcodes a data chunk corresponding to the assigned offset information.

According to an embodiment of the present invention, each transcoder transcodes allocated data chunks, evaluates the quality of the transcoded data chunks, and re-encodes data chunks that do not meet the quality standard. In this case, encoding may be performed by changing an encoding parameter.

After the transcoding of the data chunks is completed, the transcoding unit notifies the controller of the completion of transcoding (S420).

The transcoding unit stores the transcoded data chunks in the storage unit (S422).

The control unit requests the merging unit to merge the transcoded data chunks (S424).

The merging unit loads the transcoded data chunks from the storage unit according to the merging request (S426).

The merging unit loads the transcoded data chunks into memory and merges them (S428).

Specifically, the merging unit may generate transcoded video content by sequentially merging transcoded data chunks using key frames.

The merging unit notifies the control unit of merging completion (S430).

The merging unit stores the transcoded video contents in the storage unit (S432).

Meanwhile, the transcoding system may equally apply a transcoding process for video content to audio content. However, since the amount of transcoding operation for audio content is smaller than that for video content, the transcoding system may transcode the audio content at once or divide the data chunk of the video content by an integer multiple of the size and perform parallel transcoding.

5 is a flowchart illustrating an operation process of an analyzer according to an embodiment of the present invention.

Referring to FIG. 5 , the analyzer receives a video content analysis request from the control unit (S500).

The analysis request may include an ID of video content to be analyzed in a URL format. The analysis unit may access video content stored in the storage unit using a URL.

The analysis unit copies the video content from the storage unit to a local storage within the analysis unit (S502).

The analysis unit extracts detailed information of video content and generates offset information of data chunks (S504).

Specifically, the analyzer may extract detailed information of video content for error checking of video content and encoding parameter setting of the control unit.

The analyzer may divide the video content into a plurality of data chunks according to time. The analysis unit may divide the video content into a plurality of data chunks by dividing the reproduction time of the video content into predetermined time intervals. On the other hand, the analysis unit may variably set the time interval in consideration of the complexity of video content.

The analysis unit stores the starting point and ending point for each time interval as offset information of the corresponding data chunk. That is, the starting point and ending point for each time interval may be the starting point and ending point of the data chunk. Specifically, the analyzer decodes frames included in video content. If the current frame being decoded is the frame closest to the starting point of the next data chunk, the analyzer sets the current frame as the starting point of the next data chunk and sets the current frame as the ending point of the current data chunk.

The analysis unit decodes the video content frame by frame (S506).

Since the video contents stored in the storage unit are stored in an encoded state, the analysis unit searches for key frames of the video contents and decodes them frame by frame based on the key frames.

The analysis unit determines whether or not there is an error in the frame during decoding (S508).

Specifically, according to an embodiment of the present invention, the analysis unit checks the first error in the video content by attempting to decode the video content and checking whether decoding is possible. Meanwhile, the analyzer checks for a second error in the video content by determining whether the detailed information of the video content matches the decoded video content. In addition to this, the analyzer may check a third error by checking a syntax error of the video content.

If it is determined that the frame of the video content includes an error, the analysis unit notifies the controller of the error in the video content (S510).

If it is determined that the frame of the video content does not contain an error, the analysis unit determines whether it is the last frame of the video content (S512).

If the current frame is not the last frame of the video content, the analysis unit determines whether the current playback time of the video content is an offset boundary value of the data chunk (S514).

If the current frame being decoded is the frame closest to the starting point of the next data chunk, the analyzer sets the current frame as the starting point of the next data chunk and sets the current frame as the ending point of the current data chunk (S516).

If the current frame being decoded is not the frame closest to the starting point of the next data chunk, the analysis unit decodes the next frame.

Meanwhile, when the current frame is the last frame of the video content, the analysis unit records the current frame as the end point of the data chunk (S518).

Finally, the analysis unit transmits the detailed information and offset information of the video contents to the storage unit (S520).

6 is a flowchart illustrating an operation process of a transcoding unit according to an embodiment of the present invention.

Referring to FIG. 6 , the transcoding unit receives a transcoding request and generates transcoders according to the number of data chunks calculated by the analysis unit (S600).

The transcoding request includes transcoding parameters and offset information for each data chunk. Meanwhile, the number of data chunks is calculated by dividing the reproduction time of video content by a time interval of a specific size.

The transcoding unit loads data chunks according to the offset information for each transcoder (S602).

Specifically, the transcoding unit allocates a plurality of data chunks to a plurality of transcoders. Each transcoder loads a chunk of data that is part of the video content corresponding to the assigned offset information.

The transcoding unit transcodes the data chunks using the transcoding parameters included in the transcoding request (S604).

Data chunks are converted into data suitable for the user's environment and network conditions by being transcoded. The plurality of data chunks can reduce the time required for transcoding compared to a single transcoder method by transcoding the data chunks in parallel.

The transcoding unit evaluates the quality of each transcoded data chunk (S606).

The transcoding unit may evaluate the quality of each transcoded data chunk based on the video content. Specifically, the transcoding unit may evaluate the quality of each transcoded data chunk using a video quality measurement algorithm such as VMAF (Video Multimethod Assessment Fusion).

The transcoding unit determines whether each transcoded data chunk passes the quality standard (S608).

If the transcoded data chunk does not meet the quality standard, the transcoding unit changes the transcoding parameter of the data chunk (S610).

Specifically, when the transcoded data chunk does not meet the quality criterion, the transcoding unit identifies a data chunk corresponding to the transcoded data chunk as a target data chunk. That is, the target data chunk is data before transcoding of the transcoded data chunk.

The transcoding unit calculates transcoding parameter values for transcoding so that the target data chunk satisfies a target quality criterion. The transcoding unit changes transcoding parameters for the target data chunk. The transcoding unit may change the transcoding parameters of the target data chunk with high quality. For example, the transcoding unit may determine the distribution of compressed data of the video content, maintain the average bit rate, and adjust the encoding bit rate for each data chunk. As another example, the transcoding unit may achieve a target quality by adjusting a Constant Rate Factor (CRF) value among transcoding parameters. The transcoding unit transcodes the target data chunk again using the adjusted transcoding parameters and evaluates the quality. If the re-transcoded target data chunk does not satisfy the quality criterion, the transcoder may re-transcode the video content by increasing the average bit rate. As such, the transcoding unit may evaluate the quality of the transcoded data chunk, change transcoding parameters according to the quality, and perform the transcoding again, thereby guaranteeing the quality of the video content viewed by the user.

The transcoding unit determines whether the currently transcoded data chunk is the last data chunk (S612).

If the currently transcoded data chunk is not the last data chunk, the transcoding unit evaluates the quality of the next transcoded data chunk.

If the currently transcoded data chunk is the last data chunk, the transcoding unit stores each transcoded data chunk and offset information (S614).

Meanwhile, the transcoding unit may perform the same process as video content for audio content. The transcoding unit may integrally transcode the audio content or divide the audio content into a plurality of audio data chunks and perform parallel transcoding. In this case, the size of each audio data chunk may be an integer multiple of the video data chunk. That is, both video content and audio content can be distributedly encoded, but in some cases, it is also possible to process audio content with single encoding and process only video content with distributed encoding.

7 is a flowchart illustrating an operation process of a merging unit according to an embodiment of the present invention.

First, the merging unit receives a merging request for a plurality of transcoded data chunks from the control unit.

Referring to FIG. 7 , the merging unit sequentially merges a plurality of transcoded data chunks using offset information (S700).

Transcoded video content is created by merging a plurality of transcoded data chunks in offset order.

The merging unit loads the transcoded audio content (S702).

Here, the transcoded audio content may exist as one piece of data or in the form of a plurality of data chunks.

The merging unit determines whether length information of the transcoded video content matches length information of the audio content (S704).

Specifically, the merging unit determines whether the total playback time of the transcoded video content and the total playback time of the transcoded audio content are the same.

When the length information of the transcoded video content matches the length information of the audio content, the merging unit merges the video content and the audio content (S708).

The merging unit may temporally interleave the transcoded video content and the transcoded audio content for each segment into one encoding file.

The merging unit stores the merged media contents (S710).

Meanwhile, when the length information of the transcoded video content and the length information of the audio content do not match, the merging unit notifies a transcoding error (S706).

The merging unit determines that frame loss or frame duplication has occurred during the transcoding of the transcoded video content. That is, the merging unit determines that the integrity of the transcoded video content has not been verified. Transcoding of the video content may be attempted again.

8 is a flowchart illustrating an operation process of a control unit according to an embodiment of the present invention.

First, the control unit receives an upload completion notification of video content or a transcoding request event from the storage unit.

Referring to FIG. 8 , the control unit requests the analysis unit to extract detailed information about video content and divide data chunks (S800).

In addition, the control unit may request an error check on the video content to the analysis unit. After analysis by the analysis unit is completed, the control unit is notified of completion of the analysis from the analysis unit.

The control unit requests the transcoding unit to transcode the video content and the audio content (S802).

The control unit transmits transcoding parameters and offset information together with the transcoding request.

The controller requests the merging unit to merge the transcoded video content and the transcoded audio content (S804).

The controller delivers the transcoded media content to the streaming server (S806).

In addition to this, the controller may deliver the transcoded media content to a server for storage.

9 is a diagram illustrating a frame-by-frame configuration of video content.

A codec for compressing a video extracts information capable of restoring a frame in a receiving device from a frame, and transmits the extracted information to the receiving device. Since the amount of extracted information is less than that of the frame itself, the codec can reduce the amount of transmission through information extraction.

The codec can extract encoding data for each frame, but it can also use information about the change between two frames for further compression. Specifically, the codec extracts first information for reconstructing the first frame from the first frame, extracts information about a difference between the first frame and the second frame as second information, and transmits it to the receiving device. The receiving device may reconstruct the first frame from the first information and reconstruct the second frame from the first information and the second information. This is called motion compensation.

Referring to FIG. 9 , consecutive frames in an image are shown. Each frame represents a still image.

In the motion compensation method, the codec can compress an image using three frames. The three frames are an Infra frame (I-frame), a Bi-directional predicted frame (B-frame), and a Predicted frame (P-frame).

An I-frame has a key frame role and can be independently encoded and decoded with only a single frame without other frames. Since I-frames can be independently encoded and decoded, they can be used as reference points for random access. For example, when a user designates a point in time at which an arbitrary frame is located while watching a video as a reproduction start point, it may be reproduced from an I-frame located near the designated point. Meanwhile, I-frames may be compressed using a joint photographic coding experts group (JPEG) compression method.

A P-frame can be encoded using either an I-frame or a P-frame that precedes the P-frame. For example, a difference between a P-frame and an I-frame positioned before the P-frame may be calculated as change information, and the change information may be DCT-converted and compressed. The receiving device may reconstruct the I-frame and predict the P-frame from the I-frame using change information.

A B-frame is a frame that is bidirectionally predicted by referring to either an I-frame or a P-frame located before the B-frame and either an I-frame or a P-frame located after the B-frame.

As such, I-frames can be encoded and decoded independently, and P-frames and B-frames can be encoded and decoded with reference to other frames.

P-frames and B-frames do not refer to other frames, but the amount of information required to reconstruct the frame is smaller than that of I-frames. The size of a P-frame corresponds to one third of the size of an I-frame, and the size of a B-frame corresponds to one-third of the size of a P-frame.

A set of frames compressed through the aforementioned compression process may be referred to as a Group of Pictures (GOP). In detail, GOP means an interval between one I-frame and one I-frame located after one I-frame, and represents the period of the I-frame.

10 is a diagram showing the configuration of frame groups of video content.

A frame of video content may consist of only I-frames or may further include P-frames or B-frames.

The number of GOPs is the number of frames located between one I-frame and the next I-frame. In other words, the number of GOPs may mean the sum of the numbers of P-frames and B-frames within one I-frame period. As the number of GOPs increases, the compression rate increases, but the image quality deteriorates. Conversely, as the number of GOPs decreases, the compression rate decreases but image quality improves. The number of GOPs can be set to any value.

One GOP includes one or more I-frames, and the I-frames serve as reference points for reproduction. The smaller the number of GOPs, the higher the density of I-frames, and the larger the number of GOPs, the lower the density of I-frames. A smaller number of GOPs is suitable for random access, high-speed playback, and reverse playback.

Meanwhile, the number of GOPs may be equal to the number of frames corresponding to about 0.5 seconds based on frames per second of video content. A GOP generated according to the number of frames corresponding to about 0.5 seconds is referred to as a long GOP. On the other hand, a GOP generated every 6 frames is referred to as a short GOP.

Meanwhile, a transport stream (TS) file, which is a file recorded from a broadcasting source, includes the number of GOPs and the number of consecutive B-frames.

Referring to FIG. 10, the number of GOPs (N) and the number of consecutive B-frames (N) are illustrated. The frames located at the top have a GOP number of 12 and the number of consecutive B-frames is 3. On the other hand, the lower frames have 15 GOPs and 5 consecutive B-frames.

GOP information including the number of GOPs and the number of consecutive B-frames may be transmitted along with a video stream behind a sequence header. Here, the sequence header may further include information such as screen resolution information, aspect ratio information, number of frames per second, bit rate, or buffer size.

Through the above configurations, when a receiving device randomly accesses a specific position in a video, it can check the position of the sequence header and refer to the I-frame in the GOP to restore the video. The receiving device may reconstruct the P-frame and the B-frame using the restored I-frame.

Meanwhile, according to an embodiment of the present invention, the transcoding system may perform parallel transcoding by logically dividing video content into a plurality of data chunks according to time and allocating the data chunks to transcoders. To this end, the transcoding system may divide the reproduction time of video content into a plurality of time intervals. Each transcoder performs decoding and encoding based on the allocated time interval.

However, video content is composed of a plurality of frames, and the video content can be decoded using an I-frame as a reference point. That is, video content is decoded based on I-frames arranged at frame intervals according to the number of GOPs.

Therefore, when each transcoder of the transcoding system decodes a data chunk according to an allocated time interval, an error may occur between the start point of the allocated time interval and the position of the I-frame.

11A and 11B are diagrams illustrating transcoding sections of frames.

Referring to FIG. 11A , frames within video content are shown.

I-frames are placed at given frame intervals. A given frame interval is determined by the playback time of video content, the number of GOPs, the number of frames per second, and the like.

Video content may be divided into time sections having the same size. For example, each time interval may have a size of 0.5 seconds. In this case, the T-1 th time interval corresponds to the T-1 th data chunk, and the T th time interval corresponds to the T th data chunk.

In FIG. 11A, frame intervals between I-frames in video content coincide with divided time intervals. That is, the starting point of the frame interval coincides with the starting point of the time interval.

The T-th transcoder may load a data chunk corresponding to the T-th time interval and perform decoding from an I-frame located at a decoding start point of the T-th time interval.

The Tth transcoder encodes the decoded data chunk according to the encoding parameters. Here, the T-th transcoder may encode the frame located at the decoding start point of the T-th time interval by setting it as an I-frame.

For example, when the size of the time interval is designated as 4 seconds and the number of frames per second of video content is 24, the start point of the frame interval and the start point of the time interval may coincide. The first time interval includes 96 frames, and the second time interval starts from the 97th frame.

On the other hand, referring to FIG. 11B, the frame interval between I-frames in video content and the divided time intervals do not match. The start point of the T-th frame section and the start point of the T-th time section do not coincide.

For example, when the size of the time interval is 4 seconds and the number of frames per second is 23.96, the start point of the second time interval is located between the 96th frame and the 97th frame. That is, the starting point of the second time interval may be different from the starting point of the second frame interval.

The T-th transcoder loads the data chunk based on the B-frame located at the start point of the T-th time interval. That is, the first frame of the data chunk loaded by the Tth transcoder is a B-frame. In this case, the T-th transcoder cannot perform decoding at the start point of the T-th time interval.

The T-th transcoder searches for an I-frame within the loaded data chunk and starts decoding based on the first I-frame.

Accordingly, frames located between the start point of the T-th time interval and the decoding start point of the T-th interval are lost in the transcoding process even though they are loaded into the transcoder. This becomes a factor in deteriorating image quality for users who watch video content.

The transcoding system may adjust the number of frames per second to match the starting point of the time interval with the starting point of the frame interval, but it is not effective because it may affect the quality of video content.

12 is a diagram illustrating transcoding sections of frames according to an embodiment of the present invention.

Referring to FIG. 12, the transcoding system obtains the positions of key frames located in each given frame period. A transcoding system divides video content into a plurality of time intervals.

Then, the transcoding system determines whether the starting point of the T-th time interval coincides with the starting point of the T-th frame interval.

The transcoding system adjusts the start point of the T-th time interval based on the location of the key frame when the start point of the T-th time interval does not coincide with the start point of the T-th frame interval.

The transcoding system may adjust the start point of the T-th time interval to the position of the key frame closest to the start point of the T-th time interval.

Specifically, the transcoding system may adjust the start point of the T-th time interval to a key frame closest to the start point of the T-th time interval among key frames located before the start point of the T-th time interval.

Similarly, the transcoding system may adjust the end point of the T-th time interval to the position of the key frame closest to the end point of the T-th time interval.

Specifically, the transcoding system may adjust the end point of the T-th time interval to a key frame closest to the end point of the T-th time interval among key frames located after the end point of the T-th time interval.

Meanwhile, the transcoding system can be used as it is without adjusting the start and end points of video content.

The transcoding system transmits information about the unadjusted time interval or the adjusted time interval to transcoders. Upon receiving the adjusted time interval, the transcoder loads and decodes the video content part corresponding to the adjusted time interval within the video content. Since the key frame is located at the start point of the time interval allocated to the transcoders, the transcoders can decode frames without frame loss for the allocated time interval.

The T-th transcoder performs decoding by loading frames from the decoding start point of the T-th time interval. The T-th transcoder performs decoding until the adjusted end point of the T-th time interval.

Thereafter, the transcoder performs encoding not according to the adjusted time interval but according to the non-adjusted time interval. The video content part decoded according to the adjusted time interval is encoded according to the existing time interval.

The T-th transcoder performs encoding from a frame located at the start point of the T-th time interval, not the decoding start point of the T-th time interval. That is, the frame located at the start point of the Tth time interval is encoded as a key frame.

Data chunks corresponding to each time interval are transcoded without frame loss.

Then, the transcoder stores the transcoded data chunks according to the given time interval. Transcoded data chunks are merged into transcoded video content.

13 is a flowchart illustrating a parallel transcoding process according to an embodiment of the present invention.

A parallel transcoding method according to an embodiment of the present invention is applied to video content including key frames at given frame intervals.

Referring to FIG. 13, the computing device for parallel transcoding divides the playback time of video content into a plurality of time intervals (S1300).

According to an embodiment of the present invention, the computing device may determine the size of the time intervals in consideration of at least one of a computing resource, a playback environment of a user terminal, or a network state.

Also, the computing device generates a plurality of transcoders based on the number of time intervals.

Based on the location of the key frame, the computing device may adjust at least one of a starting point or an ending point of one time interval among a plurality of time intervals.

The computing device adjusts the starting point of one time interval to the position of a key frame closest to the starting point among key frames located before the starting point (S1302).

The computing device adjusts the end point of one time interval to a position of a key frame closest to the end point among key frames located after the end point (S1304).

In addition to this, the computing device may adjust the start point to the position of the key frame closest to the start point, or adjust the end point to the position of the key frame closest to the end point.

The computing device decodes the video content part corresponding to the adjusted time interval using one of the plurality of transcoders (S1306).

The computing device may decode the video content part by sending the transcoding parameters together with the adjusted time interval to one transcoder.

The transcoder loads and decodes the video content part corresponding to the adjusted time interval within the video content. Since the key frame is located at the starting point of the adjusted time interval, the transcoder can perform decoding from the starting point of the adjusted time interval. In addition, since the end point of the adjusted time interval corresponds to the start point of the next time interval and the start point of the next frame interval, the transcoder can decode up to the start point of the next time interval.

Meanwhile, according to another embodiment of the present invention, transcoding parameters include a time interval between key frames in transcoded video content. Each transcoder generates a key frame by considering a time interval between key frames in transcoded video content during an encoding process. Key frames serve as reference points for playback such as video playback, reverse playback, and skipping.

The computing device encodes the decoded video content part in the range of the one time interval by using a transcoder (S1308).

The transcoder encodes the decoded video content part within a range corresponding to the one time interval. Encoded video content parts become transcoded data chunks.

Finally, transcoded data chunks are generated according to each time interval without frame loss. The transcoded data chunks are merged to become transcoded video content and transmitted to the user terminal.

4 to 8 and 13 describe that each process is sequentially executed, but this is merely an example of the technical idea of an embodiment of the present invention. In other words, those skilled in the art to which an embodiment of the present invention pertains may change and execute the order described in FIGS. 4 to 8 and 13 without departing from the essential characteristics of the embodiment of the present invention, or each process 4 to 8 and 13 are not limited to a time-sequential order, since it will be possible to apply various modifications and variations by executing one or more of these processes in parallel.

Meanwhile, the processes shown in FIGS. 4 to 8 and 13 can be implemented as computer readable codes on a computer readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. That is, such a computer-readable recording medium includes non-transitory media such as ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium may be distributed to computer systems connected through a network to store and execute computer-readable codes in a distributed manner.

In addition, components of the present invention may use an integrated circuit structure such as a memory, a processor, a logic circuit, a look-up table, and the like. These integrated circuit structures execute each of the functions described herein through the control of one or more microprocessors or other control devices. In addition, the components of the present invention may be specifically implemented by a program or part of code that includes one or more executable instructions for performing a specific logical function and is executed by one or more microprocessors or other control devices. In addition, the components of the present invention may include or be implemented by a central processing unit (CPU), a microprocessor, etc. that perform each function. Also, components of the present invention may store instructions executed by one or more processors in one or more memories.

The above description is merely an example of the technical idea of the present embodiment, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of this embodiment should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of this embodiment.

Claims (10)

A computer-implemented method for parallel transcoding video content containing key frames at given frame intervals, comprising:
Dividing a reproduction time of video content into a plurality of time intervals;
adjusting at least one of a starting point and an ending point of one of the plurality of time intervals based on the position of the key frame;
decoding a video content part corresponding to the adjusted time interval by using one of a plurality of transcoders; and
Encoding the decoded video content part in a range of the one time interval using the one transcoder
How to include. According to claim 1,
The process of adjusting at least one of the start point and the end point of the one time interval,
Adjusting the start point to the position of a key frame closest to the start point or adjusting the end point to the position of a key frame closest to the end point
How to include. According to claim 1,
The process of adjusting the starting point of the one time interval,
Adjusting the starting point to a position of a key frame closest to the starting point among key frames located before the starting point
How to include. According to claim 1,
The process of adjusting the end point of the one time interval,
Adjusting the end point to a position of a key frame closest to the end point among key frames located after the end point
How to include. According to claim 1,
The one transcoder,
and loading and decoding a video content part corresponding to the adjusted time interval in the video content. According to claim 1,
The process of encoding the decoded video content part,
Setting a frame located at a start point of the one time interval among frames included in the decoded video content part as a key frame
How to include. According to claim 1,
Determining the size of the time intervals in consideration of at least one of a computing resource, a playback environment of a user terminal, or a network state
How to include more. According to claim 1,
Process of generating a plurality of transcoders based on the number of time intervals
How to include more. memory for storing instructions; and
including at least one processor;
By the at least one processor executing the instructions,
Dividing a reproduction time of video content including a key frame into a plurality of time intervals for each given frame interval;
Adjusting at least one of a start point or an end point of one time interval among the plurality of time intervals based on the position of the key frame;
decoding a video content part corresponding to the adjusted time interval using one transcoder among a plurality of transcoders;
and encodes the decoded video content part in a range of the one time interval using the one transcoder. A computer-readable recording medium recording a computer program for executing the method of any one of claims 1 to 8.

Images